base on The HIP Environment and ROCm Kit - A lightweight open source build system for HIP and ROCm # TheRock [](https://github.com/pre-commit/pre-commit) [](https://github.com/ROCm/TheRock/actions/workflows/ci.yml?query=branch%3Amain) TheRock (The HIP Environment and ROCm Kit) is a lightweight open source build platform for HIP and ROCm. The project is currently in an **early preview state** but is under active development and welcomes contributors. Come try us out! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for more info. ## Features TheRock includes: - Nightly releases of ROCm and PyTorch - A CMake super-project for HIP and ROCm source builds - Support for building PyTorch with ROCm from source - [JAX support](https://github.com/ROCm/TheRock/issues/247) and other external project builds are in the works! - Tools for developing individual ROCm components - Comprehensive CI/CD pipelines for building, testing, and releasing supported components ### Support status For HIP and ROCm: | | Build from source | Prebuilt packages | Python packages | | ------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Linux | ✅ Supported | [](https://github.com/ROCm/TheRock/actions/workflows/release_portable_linux_packages.yml?query=branch%3Amain) | [](https://github.com/ROCm/TheRock/actions/workflows/release_portable_linux_packages.yml?query=branch%3Amain) | | Windows | ✅ Supported | [](https://github.com/ROCm/TheRock/actions/workflows/release_windows_packages.yml?query=branch%3Amain) | 🟡 In Progress ([#827](https://github.com/ROCm/TheRock/issues/827)) | For PyTorch with ROCm: | | PyTorch source build | PyTorch Python packages | PyTorch Docker images | | ------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Linux | ✅ Supported | [](https://github.com/ROCm/TheRock/actions/workflows/release_linux_pytorch_wheels.yml?query=branch%3Amain) | [](https://github.com/ROCm/TheRock/actions/workflows/publish_pytorch_dev_docker.yml?query=branch%3Amain) | | Windows | 🟡 In progress ([#589](https://github.com/ROCm/TheRock/issues/589)) | 🟡 In Progress ([#827](https://github.com/ROCm/TheRock/issues/827)) | N/A | ## Installing from releases See the [Releases Page](RELEASES.md) for instructions on how to install prebuilt ROCm and PyTorch packages. ## Building from source We keep the following instructions for recent, commonly used operating system versions. Most build failures are due to minor operating system differences in dependencies and project setup. Refer to the [Environment Setup Guide](docs/environment_setup_guide.md) for contributed instructions and configurations for alternatives. > [!TIP] > While building from source offers the greatest flexibility, > [installing from releases](#installing-from-releases) in supported > configurations is often faster and easier. ### Setup - Ubuntu (24.04) ```bash # Install Ubuntu dependencies sudo apt install gfortran git git-lfs ninja-build cmake g++ pkg-config xxd patchelf automake libtool python3-venv python3-dev libegl1-mesa-dev # Clone the repository git clone https://github.com/ROCm/TheRock.git cd TheRock # Init python virtual environment and install python dependencies python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt # Download submodules and apply patches python ./build_tools/fetch_sources.py ``` ### Setup - Windows 11 (VS 2022) > [!IMPORTANT] > See [windows_support.md](./docs/development/windows_support.md) for setup > instructions on Windows, in particular > the section for > [installing tools](./docs/development/windows_support.md#install-tools). ```bash # Install dependencies following the Windows support guide # Clone interop library from https://github.com/nod-ai/amdgpu-windows-interop # for CLR (the "HIP runtime") on Windows. The path used can also be configured # using the `THEROCK_AMDGPU_WINDOWS_INTEROP_DIR` CMake variable. git clone https://github.com/nod-ai/amdgpu-windows-interop.git # Clone the repository git clone https://github.com/ROCm/TheRock.git cd TheRock # Init python virtual environment and install python dependencies python -m venv .venv .venv\Scripts\Activate.bat pip install -r requirements.txt # Download submodules and apply patches python ./build_tools/fetch_sources.py ``` ### Build configuration The build can be customized through cmake feature flags. #### Required configuration flags - `-DTHEROCK_AMDGPU_FAMILIES=` or - `-DTHEROCK_AMDGPU_TARGETS=` > [!NOTE] > Not all family and targets are currently supported. > See [therock_amdgpu_targets.cmake](cmake/therock_amdgpu_targets.cmake) file > for available options. #### Optional configuration flags By default, the project builds everything available. The following group flags enable/disable selected subsets: | Group flag | Description | | -------------------------------- | ------------------------------------ | | `-DTHEROCK_ENABLE_ALL=OFF` | Disables all optional components | | `-DTHEROCK_ENABLE_CORE=OFF` | Disables all core components | | `-DTHEROCK_ENABLE_COMM_LIBS=OFF` | Disables all communication libraries | | `-DTHEROCK_ENABLE_MATH_LIBS=OFF` | Disables all math libraries | | `-DTHEROCK_ENABLE_ML_LIBS=OFF` | Disables all ML libraries | | `-DTHEROCK_ENABLE_PROFILER=OFF` | Disables profilers | Individual features can be controlled separately (typically in combination with `-DTHEROCK_ENABLE_ALL=OFF` or `-DTHEROCK_RESET_FEATURES=ON` to force a minimal build): | Component flag | Description | | ---------------------------------- | --------------------------------------------- | | `-DTHEROCK_ENABLE_COMPILER=ON` | Enables the GPU+host compiler toolchain | | `-DTHEROCK_ENABLE_HIPIFY=ON` | Enables the hipify tool | | `-DTHEROCK_ENABLE_CORE_RUNTIME=ON` | Enables the core runtime components and tools | | `-DTHEROCK_ENABLE_HIP_RUNTIME=ON` | Enables the HIP runtime components | | `-DTHEROCK_ENABLE_ROCPROFV3=ON` | Enables rocprofv3 | | `-DTHEROCK_ENABLE_RCCL=ON` | Enables RCCL | | `-DTHEROCK_ENABLE_PRIM=ON` | Enables the PRIM library | | `-DTHEROCK_ENABLE_BLAS=ON` | Enables the BLAS libraries | | `-DTHEROCK_ENABLE_RAND=ON` | Enables the RAND libraries | | `-DTHEROCK_ENABLE_SOLVER=ON` | Enables the SOLVER libraries | | `-DTHEROCK_ENABLE_SPARSE=ON` | Enables the SPARSE libraries | | `-DTHEROCK_ENABLE_MIOPEN=ON` | Enables MIOpen | > [!TIP] > Enabling any features will implicitly enable their *minimum* dependencies. Some > libraries (like MIOpen) have a number of *optional* dependencies, which must > be enabled manually if enabling/disabling individual features. > [!TIP] > A report of enabled/disabled features and flags will be printed on every > CMake configure. ### CMake build usage To build ROCm/HIP: ```bash cmake -B build -GNinja . -DTHEROCK_AMDGPU_FAMILIES=gfx110X-dgpu cmake --build build ``` #### CCache usage on Linux To build with the [ccache](https://ccache.dev/) compiler cache: - You must have a recent ccache (>= 4.11 at the time of writing) that supports proper caching with the `--offload-compress` option used for compressing AMDGPU device code. - `export CCACHE_SLOPPINESS=include_file_ctime` to support hard-linking - Proper setup of the `compiler_check` directive to do safe caching in the presence of compiler bootstrapping - Set the C/CXX compiler launcher options to cmake appropriately. Since these options are very fiddly and prone to change over time, we recommend using the `./build_tools/setup_ccache.py` script to create a `.ccache` directory in the repository root with hard coded configuration suitable for the project. Example: ```bash # Any shell used to build must eval setup_ccache.py to set environment # variables. eval "$(./build_tools/setup_ccache.py)" cmake -B build -GNinja -DTHEROCK_AMDGPU_FAMILIES=gfx110X-dgpu \ -DCMAKE_C_COMPILER_LAUNCHER=ccache \ -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ . cmake --build build ``` #### CCache usage on Windows We are still investigating the exact proper options for ccache on Windows and do not currently recommend that end users enable it. ### Running tests Project-wide testing can be controlled with the standard CMake `-DBUILD_TESTING=ON|OFF` flag. This gates both setup of build tests and compilation of installed testing artifacts. Tests of the integrity of the build are enabled by default and can be run with ctest: ``` ctest --test-dir build ``` Testing functionality on an actual GPU is in progress and will be documented separately. ## Development manuals - [Contribution Guidelines](CONTRIBUTING.md): Documentation for the process of contributing to this project including a quick pointer to its governance. - [Development Guide](docs/development/development_guide.md): Documentation on how to use TheRock as a daily driver for developing any of its contained ROCm components (i.e. vs interacting with each component build individually). - [Build System](docs/development/build_system.md): More detailed information about TheRock's build system relevant to people looking to extend TheRock, add components, etc. - [Environment Setup Guide](docs/environment_setup_guide.md): Comprehensive guide for setting up a build environment, known workarounds, and other operating specific information. - [Git Chores](docs/development/git_chores.md): Procedures for managing the codebase, specifically focused on version control, upstream/downstream, etc. - [Dependencies](docs/development/dependencies.md): Further specifications on ROCm-wide standards for depending on various components. - [Build Containers](docs/development/build_containers.md): Further information about containers used for building TheRock on CI. - [Build Artifacts](docs/development/artifacts.md): Documentation about the outputs of the build system. - [Releases Page](RELEASES.md): Documentation for how to leverage our build artifacts. - [Roadmap for Support](ROADMAP.md): Documentation for our prioritized roadmap to support AMD GPUs. ", Assign "at most 3 tags" to the expected json: {"id":"14113","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"