base on Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group) <div align="center"> <a href="http://www.openucx.org/"><img src="./docs/doxygen/UCX_Logo_930x933.png" width="200"></a> <br> <a href="https://twitter.com/intent/follow?screen_name=openucx"> <img src="https://img.shields.io/twitter/follow/openucx?style=social&logo=twitter" alt="follow on Twitter"></a> <a href="https://openucx.github.io/ucx/api/latest/html/"><img src="docs/doxygen/api.svg"></a> <a href='https://openucx.readthedocs.io/en/master/?badge=master'><img src='https://readthedocs.org/projects/openucx/badge/?version=master' alt='Documentation Status' /> <a href="https://github.com/openucx/ucx/releases/latest"><img src="docs/doxygen/release.svg"></a> </div> <hr> # Unified Communication X Unified Communication X (UCX) is an [award winning](https://losalamosreporter.com/2019/11/07/nine-los-alamos-national-laboratory-projects-win-rd-100-awards), optimized production proven-communication framework for modern, high-bandwidth and low-latency networks. UCX exposes a set of abstract communication primitives that utilize the best of available hardware resources and offloads. These include RDMA (InfiniBand and RoCE), TCP, GPUs, shared memory, and network atomic operations. Please visit our [documentation site](https://openucx.readthedocs.io/en/master) for more details. Please review our ["Membership Voluntary Consensus Standard"](https://ucfconsortium.org/policy/) and ["Export Compliant Contribution Submissions"](https://ucfconsortium.org/policy/) policies. <hr> <!-- TOC generated by https://github.com/ekalinin/github-markdown-toc --> * [Using UCX](#using-ucx) * [Known issues](#known-issues) * [Architecture](#architecture) * [Supported Transports](#supported-transports) * [Supported CPU Architectures](#supported-cpu-architectures) * [Licenses](#licenses) * [Our Community](#our-community) * [Contributor Agreement and Guidelines](#contributor-agreement-and-guidelines) * [Publications](#publications) <hr> ## Using UCX ### Release Builds Building UCX is typically a combination of running "configure" and "make". If using a release tarball execute the following commands to install the UCX system from within the directory at the top of the tree: ```sh $ ./contrib/configure-release --prefix=/where/to/install $ make -j8 $ make install ``` If directly cloning the git repository use: ```sh $ ./autogen.sh $ ./contrib/configure-release --prefix=/where/to/install $ make -j8 $ make install ``` NOTE: Compiling support for various networks or other specific hardware may require additional command line flags when running configure. ### Developer Builds ```bash $ ./autogen.sh $ ./contrib/configure-devel --prefix=$PWD/install-debug ``` *** NOTE: Developer builds of UCX typically include a large performance penalty at run-time because of extra debugging code. ### Build RPM package ```bash $ contrib/buildrpm.sh -s -b ``` ### Build DEB package ```bash $ dpkg-buildpackage -us -uc ``` ### Build Doxygen documentation ```bash $ make docs ``` ### OpenMPI and OpenSHMEM installation with UCX [Wiki page](http://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX) ### MPICH installation with UCX [Wiki page](http://github.com/openucx/ucx/wiki/MPICH-installation-with-UCX) ### UCX Performance Test Start server: ```sh $ ./src/tools/perf/ucx_perftest -c 0 ``` Connect client: ```sh $ ./src/tools/perf/ucx_perftest <server-hostname> -t tag_lat -c 1 ``` > NOTE the `-c` flag sets CPU affinity. If running both >commands on same host, make sure you set the affinity to different CPU cores. ### Running internal unit tests ```sh $ make -C test/gtest test ``` <hr> ## Known issues * UCX version 1.8.0 has a bug that may cause data corruption when TCP transport is used in conjunction with shared memory transport. It is advised to upgrade to UCX version 1.9.0 and above. UCX versions released before 1.8.0 don't have this bug. * UCX may hang with glibc versions 2.25-2.29 due to known bugs in the pthread_rwlock functions. When such hangs occur, one of the UCX threads gets stuck in pthread_rwlock_rdlock (which is called by ucs_rcache_get), even though no other thread holds the lock. A related issue is reported in [glibc Bug 23844](https://sourceware.org/bugzilla/show_bug.cgi?id=23844). If this issue occurs, it is advised to use glibc version provided with your OS distribution or build glibc from source using versions less than 2.25 or greater than 2.29. * Due to compatibility flaw when using UCX with rdma-core v22 setting UCX_DC_MLX5_RX_INLINE=0 is unsupported and will make DC transport unavailable. This issue is fixed in rdma-core v24 and backported to rdma-core-22.4-2.el7 rpm. See [ucx issue 5749](https://github.com/openucx/ucx/issues/5749) for more details. <hr> ## Architecture  | Component | Role | Description | | :---: | :---: | --- | | UCP | Protocol | Implements high-level abstractions such as tag-matching, streams, connection negotiation and establishment, multi-rail, and handling different memory types | | UCT | Transport | Implements low-level communication primitives such as active messages, remote memory access, and atomic operations | | UCS | Services | A collection of data structures, algorithms, and system utilities for common use | | UCM | Memory | Intercepts memory allocation and release events, used by the memory registration cache | <hr> ## Supported Transports * [Infiniband](https://www.infinibandta.org/) * [Omni-Path](https://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-driving-exascale-computing.html) * [RoCE](http://www.roceinitiative.org/) * [Cray Gemini and Aries](https://www.cray.com/) * [CUDA](https://developer.nvidia.com/cuda-zone) * [ROCm](https://rocm.github.io/) * Shared Memory * posix, sysv, [cma](https://dl.acm.org/citation.cfm?id=2616532), [knem](http://knem.gforge.inria.fr/), and [xpmem](https://github.com/openucx/xpmem) * TCP/IP **NOTE:** UCX >= 1.12.0 requires rdma-core >= 28.0 or MLNX_OFED >= 5.0 for [Infiniband](https://www.infinibandta.org/) and [RoCE](http://www.roceinitiative.org/) transports support. <hr> ## Supported CPU Architectures * [x86_64](https://en.wikipedia.org/wiki/X86-64) * [Power8/9](https://www.ibm.com/support/knowledgecenter/en/POWER9/p9hdx/POWER9welcome.htm) * [Arm v8](https://www.arm.com/products/silicon-ip-cpu) <hr> ## Licenses UCX is licensed as: * [BSD3](LICENSE) <hr> ## Our Community * [Project Website](http://www.openucx.org/) * [ReadTheDocs](https://openucx.readthedocs.io/en/master/) * [Github](http://www.github.com/openucx/ucx/) * [Software Releases](http://www.github.com/openucx/ucx/releases) * [Mailing List](https://elist.ornl.gov/mailman/listinfo/ucx-group) * [Twitter](https://twitter.com/openucx) <hr> ## Contributor Agreement and Guidelines In order to contribute to UCX, please sign up with an appropriate [Contributor Agreement](http://www.openucx.org/license/). All contributors have to comply with ["Membership Voluntary Consensus Standard"](https://ucfconsortium.org/policy/) and ["Export Compliant Contribution Submissions"](https://ucfconsortium.org/policy/) policies. Follow these [instructions](https://github.com/openucx/ucx/wiki/Guidance-for-contributors) when submitting contributions and changes. ## Publications To reference UCX in a publication, please use the following entry: ```bibtex @inproceedings{shamis2015ucx, title={UCX: an open source framework for HPC network APIs and beyond}, author={Shamis, Pavel and Venkata, Manjunath Gorentla and Lopez, M Graham and Baker, Matthew B and Hernandez, Oscar and Itigin, Yossi and Dubman, Mike and Shainer, Gilad and Graham, Richard L and Liss, Liran and others}, booktitle={2015 IEEE 23rd Annual Symposium on High-Performance Interconnects}, pages={40--43}, year={2015}, organization={IEEE} } ``` To reference the UCX website: ```bibtex @misc{openucx-website, title = {{The Unified Communication X Library}}, key = {{{The Unified Communication X Library}}, howpublished = {{\url{http://www.openucx.org}}} } ``` ", Assign "at most 3 tags" to the expected json: {"id":"7187","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"