base on Packaging metasploit-framework with omnibus metasploit-framework Omnibus project
==========================
This project creates full-stack platform-specific packages for
`metasploit-framework`. This is not the same as the Metasploit Community
edition. It only contains the framework command-line interface and the
associated tools and modules.
Installing the package
------------
If you just want to install this package, we provide a number of pre-built binaries for Metasploit that are rebuilt every night. See https://github.com/rapid7/metasploit-framework/wiki/Nightly-Installers for installation information.
Building the package
------------
## Prerequisites
This project has a package cache that should be pulled in before building. Run 'git submodule update -i' to download the git submodule that contains these packages. We cache these both for performance, and because occasionally upstream locations go away, and this allows the build to continue without broken dependencies.
## Building on Docker
The Dockerfiles for `metasploit-omnibus` are located within the `docker` directory of this repository.
You can build images yourself:
```shell
git clone https://github.com/rapid7/metasploit-omnibus.git
cd metasploit-omnibus
docker build --tag metasploit-omnibus-builder - < ./docker/ubuntu1204-x86/Dockerfile
```
Or on OSX you can use the following script to build all images following the latest Docker image naming convention:
```shell
export BUILD_DATE=$(date "+%Y_%m"); ls ./docker | xargs -I IMAGE_NAME /bin/bash -x -c "docker build --tag rapid7/msf-IMAGE_NAME-omnibus:$BUILD_DATE -f ./docker/IMAGE_NAME/Dockerfile ./docker/IMAGE_NAME"
```
Pushing
```shell
export BUILD_DATE=$(date "+%Y_%m"); ls ./docker | xargs -I IMAGE_NAME /bin/bash -x -c "docker push rapid7/msf-IMAGE_NAME-omnibus:$BUILD_DATE"
```
You can then run a new container using the above tagged image, whilst mounting the current directory as a volume:
```shell
docker run -it --rm --volume $(pwd):$(pwd) --workdir $(pwd) --user jenkins metasploit-omnibus-builder /bin/bash --login
```
Or you can run a new container using pre-built images from [Rapid7's Docker Hub account](https://hub.docker.com/u/rapid7):
```shell
docker run -it --rm --volume $(pwd):$(pwd) --workdir $(pwd) --user jenkins rapid7/msf-ubuntu1204-x86-omnibus:2024_04 /bin/bash --login
```
By default, `metasploit-omnibus` will download the latest version of Metasploit framework from Github, but also supports building with local copies from `/metasploit-framework` - [full details](https://github.com/rapid7/metasploit-omnibus/blob/9cd575bcdd19d8fedf4a94c4ca2d1d6c253628c2/config/software/metasploit-framework.rb#L2-L8).
To build omnibus with a local version of Metasploit framework, you can mount your framework repository as a volume to `/metasploit-framework` within the container. The following command assumes that the repository exists within the parent directory:
```shell
docker run -it --rm --volume $(pwd):$(pwd) --volume=$(pwd)/../metasploit-framework:/metasploit-framework --workdir $(pwd) --user jenkins rapid7/msf-ubuntu1204-x86-omnibus:2024_04 /bin/bash --login
```
When running inside the container, you can perform a normal ommibus build:
```
# Download the git submodule that contains cached packages
git submodule update -i
# install omnibus' dependencies
bundle install
bundle binstubs --all
# build the metasploit-framework package
bin/omnibus build metasploit-framework
```
When complete, there will be a new installable `.deb` file under the 'pkg' directory. Note that the use of Docker volumes may cause builds to run slower.
To test the `.deb` file, install it - and then open msfconsole:
```
# install
sudo dpkg -i pkg/metasploit-framework_6.3.39~20231017232715.git.3.47e0cd3~1rapid7-1_amd64.deb
# Run to verify
msfconsole
```
## Building on Ubuntu / Debian systems
In general, a build environment needs a working C/C++ compiler, Ruby 1.9 or higher and the ruby development headers, bundler, git, bison and flex. A quad-core CPU and 4GB of ram are recommended.
The following steps should produce a successful build with Ubuntu and other Debian derivatives, starting from a fresh installation:
```shell
# install required packages to build on Ubuntu / Debian systems
sudo apt-get -y install build-essential git ruby bundler ruby-dev bison flex autoconf automake
```
Configure the omnibus cache and target directories if you want to build as non-root user (recommended).
```shell
# setup build directories you can write to
sudo mkdir -p /var/cache/omnibus
sudo mkdir -p /opt/metasploit-framework
sudo chown `whoami` /var/cache/omnibus
sudo chown `whoami` /opt/metasploit-framework
```
Next setup git if you need to.
```shell
# setup git (ignore if you already have it configured)
git config --global user.name "Nobody"
git config --global user.email "
[email protected]"
```
Checkout the metasploit-framework installer builder and install omnibus' dependencies.
```shell
# checkout the builder repository
git clone https://github.com/rapid7/metasploit-omnibus.git
cd metasploit-omnibus
# install omnibus' dependencies
bundle install && bundle binstubs --all
```
Finally, build the installer itself:
```shell
# build the metasploit-framework package
bin/omnibus build metasploit-framework
```
when complete, there will be a new installable .deb file under the 'pkg' directory.
## Building on Windows
From Windows 10, install ruby, msys2, ruby-devkit, wixtoolset, git. Add the following command to the the preparation steps before executing the `build` command.
```
xz -d local/cache/*.xz
```
## Building on OS X
From OS X, first install XCode and the command line development tools. I use ruby, bundler, git, bison and flex from the Mac Homebrew project. The rest of the steps are identical to building on Ubuntu. A .pkg file will be under the pkg directory instead.
## Clean
You can clean up all temporary files generated during the build process with
the `clean` command:
```shell
$ bin/omnibus clean metasploit-framework
```
Adding the `--purge` purge option removes __ALL__ files generated during the
build including the project install directory (`/opt/metasploit-framework`) and
the package cache directory (`/var/cache/omnibus/pkg`) as well as __ALL__ files
in the local package cache directory (`./local/cache`):
```shell
$ bin/omnibus clean metasploit-framework --purge
```
Restore the git submodule that contains the local package cache:
```shell
$ git submodule update -i
```
## Publish
Omnibus has a built-in mechanism for releasing to a variety of "backends", such
as Amazon S3. You must set the proper credentials in your `omnibus.rb` config
file or specify them via the command line.
```shell
$ bin/omnibus publish path/to/*.deb --backend s3
```
## Help
Full help for the Omnibus command line interface can be accessed with the
`help` command:
```shell
$ bin/omnibus help
```
", Assign "at most 3 tags" to the expected json: {"id":"9561","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"