base on CUDA accelerated rasterization of gaussian splatting # gsplat [](https://github.com/nerfstudio-project/gsplat/actions/workflows/core_tests.yml) [](https://github.com/nerfstudio-project/gsplat/actions/workflows/doc.yml) [http://www.gsplat.studio/](http://www.gsplat.studio/) gsplat is an open-source library for CUDA accelerated rasterization of gaussians with python bindings. It is inspired by the SIGGRAPH paper [3D Gaussian Splatting for Real-Time Rendering of Radiance Fields](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/), but we’ve made gsplat even faster, more memory efficient, and with a growing list of new features! <div align="center"> <video src="https://github.com/nerfstudio-project/gsplat/assets/10151885/64c2e9ca-a9a6-4c7e-8d6f-47eeacd15159" width="100%" /> </div> ## Installation **Dependence**: Please install [Pytorch](https://pytorch.org/get-started/locally/) first. The easiest way is to install from PyPI. In this way it will build the CUDA code **on the first run** (JIT). ```bash pip install gsplat ``` Alternatively you can install gsplat from source. In this way it will build the CUDA code during installation. ```bash pip install git+https://github.com/nerfstudio-project/gsplat.git ``` We also provide [pre-compiled wheels](https://docs.gsplat.studio/whl) for both linux and windows on certain python-torch-CUDA combinations (please check first which versions are supported). Note this way you would have to manually install [gsplat's dependencies](https://github.com/nerfstudio-project/gsplat/blob/6022cf45a19ee307803aaf1f19d407befad2a033/setup.py#L115). For example, to install gsplat for pytorch 2.0 and cuda 11.8 you can run ``` pip install ninja numpy jaxtyping rich pip install gsplat --index-url https://docs.gsplat.studio/whl/pt20cu118 ``` To build gsplat from source on Windows, please check [this instruction](docs/INSTALL_WIN.md). ## Evaluation This repo comes with a standalone script that reproduces the official Gaussian Splatting with exactly the same performance on PSNR, SSIM, LPIPS, and converged number of Gaussians. Powered by gsplat’s efficient CUDA implementation, the training takes up to **4x less GPU memory** with up to **15% less time** to finish than the official implementation. Full report can be found [here](https://docs.gsplat.studio/main/tests/eval.html). ```bash pip install -r examples/requirements.txt # download mipnerf_360 benchmark data python examples/datasets/download_dataset.py # run batch evaluation bash examples/benchmarks/basic.sh ``` ## Examples We provide a set of examples to get you started! Below you can find the details about the examples (requires to install some exta dependencies via `pip install -r examples/requirements.txt`) - [Train a 3D Gaussian splatting model on a COLMAP capture.](https://docs.gsplat.studio/main/examples/colmap.html) - [Fit a 2D image with 3D Gaussians.](https://docs.gsplat.studio/main/examples/image.html) - [Render a large scene in real-time.](https://docs.gsplat.studio/main/examples/large_scale.html) ## Development and Contribution This repository was born from the curiosity of people on the Nerfstudio team trying to understand a new rendering technique. We welcome contributions of any kind and are open to feedback, bug-reports, and improvements to help expand the capabilities of this software. This project is developed by the following wonderful contributors (unordered): - [Angjoo Kanazawa](https://people.eecs.berkeley.edu/~kanazawa/) (UC Berkeley): Mentor of the project. - [Matthew Tancik](https://www.matthewtancik.com/about-me) (Luma AI): Mentor of the project. - [Vickie Ye](https://people.eecs.berkeley.edu/~vye/) (UC Berkeley): Project lead. v0.1 lead. - [Matias Turkulainen](https://maturk.github.io/) (Aalto University): Core developer. - [Ruilong Li](https://www.liruilong.cn/) (UC Berkeley): Core developer. v1.0 lead. - [Justin Kerr](https://kerrj.github.io/) (UC Berkeley): Core developer. - [Brent Yi](https://github.com/brentyi) (UC Berkeley): Core developer. - [Zhuoyang Pan](https://panzhy.com/) (ShanghaiTech University): Core developer. - [Jianbo Ye](http://www.jianboye.org/) (Amazon): Core developer. We also have a white paper with about the project with benchmarking and mathematical supplement with conventions and derivations, available [here](https://arxiv.org/abs/2409.06765). If you find this library useful in your projects or papers, please consider citing: ``` @article{ye2024gsplatopensourcelibrarygaussian, title={gsplat: An Open-Source Library for {Gaussian} Splatting}, author={Vickie Ye and Ruilong Li and Justin Kerr and Matias Turkulainen and Brent Yi and Zhuoyang Pan and Otto Seiskari and Jianbo Ye and Jeffrey Hu and Matthew Tancik and Angjoo Kanazawa}, year={2024}, eprint={2409.06765}, journal={arXiv preprint arXiv:2409.06765}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2409.06765}, } ``` We welcome contributions of any kind and are open to feedback, bug-reports, and improvements to help expand the capabilities of this software. Please check [docs/DEV.md](docs/DEV.md) for more info about development. ", Assign "at most 3 tags" to the expected json: {"id":"10690","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"