base on Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by Clickhouse and OpenTelemetry. <p align="center"> <a href="https://hyperdx.io"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/images/logo_dark.png#gh-dark-mode-only"> <img alt="hyperdx logo" src="./.github/images/logo_light.png#gh-light-mode-only"> </picture> </a> </p> --- # HyperDX [HyperDX](https://hyperdx.io) helps engineers quickly figure out why production is broken by centralizing and correlating logs, metrics, traces, exceptions and session replays in one place. An open source and developer-friendly alternative to Datadog and New Relic. <p align="center"> <a href="https://www.hyperdx.io/docs">Documentation</a> • <a href="https://hyperdx.io/discord">Chat on Discord</a> • <a href="https://api.hyperdx.io/login/demo">Live Demo</a> • <a href="https://github.com/hyperdxio/hyperdx/issues/new">Bug Reports</a> • <a href="./CONTRIBUTING.md">Contributing</a> </p> - 🕵️ Correlate end to end, go from browser session replay to logs and traces in just a few clicks - 🔥 Blazing fast performance powered by Clickhouse - 🔍 Intuitive full-text search and property search syntax (ex. `level:err`) - ⏱️ Monitor health and performance from HTTP requests to DB queries (APM) - 🤖 Automatically cluster event patterns from billions of events - 📈 Dashboard high cardinality events without a complex query language - 🔔 Set up alerts on logs, metrics, or traces in just a few clicks - `{` Automatic JSON/structured log parsing - 🔭 OpenTelemetry native <br/> <img alt="Search logs and traces all in one place" src="./.github/images/search_splash.png" title="Search logs and traces all in one place"> ### Additional Screenshots <details> <summary><b>📈 Dashboards</b></summary> <img alt="Dashboard" src="./.github/images/dashboard.png"> </details> <details> <summary><b>🤖 Automatic Event Pattern Clustering</b></summary> <img alt="Event Pattern Clustering" src="./.github/images/pattern3.png"> </details> <details> <summary><b>🖥️ Session Replay & RUM</b></summary> <img alt="Event Pattern Clustering" src="./.github/images/session.png"> </details> ## Spinning Up HyperDX The HyperDX stack ingests, stores, and searches/graphs your telemetry data. After standing up the Docker Compose stack, you'll want to instrument your app to send data over to HyperDX. You can get started by deploying a complete stack via Docker Compose. After cloning this repository, simply start the stack with: ```bash docker compose up -d ``` Afterwards, you can visit http://localhost:8080 to access the HyperDX UI. > If your server is behind a firewall, you'll need to open/forward port 8080, > 8000 and 4318 on your firewall for the UI, API and OTel collector > respectively. > We recommend at least 4GB of RAM and 2 cores for testing. **Enabling Self-instrumentation/Demo Logs** To get a quick preview of HyperDX, you can enable self-instrumentation and demo logs by setting the `HYPERDX_API_KEY` to your ingestion key (go to [http://localhost:8080/team](http://localhost:8080/team) after creating your account) and then restart the stack. This will redirect internal telemetry from the frontend app, API, host metrics and demo logs to your new HyperDX instance. ex. ```sh HYPERDX_API_KEY=<YOUR_INGESTION_KEY> docker compose up -d ``` > If you need to use `sudo` for docker, make sure to forward the environment > variable with the `-E` flag: > `HYPERDX_API_KEY=<YOUR_KEY> sudo -E docker compose up -d` **Changing Hostname and Port** By default, HyperDX app/api will run on localhost with port `8080`/`8000`. You can change this by updating `HYPERDX_APP_**` and `HYPERDX_API_**` variables in the `.env` file. After making your changes, rebuild images with `make build-local`. **DB Migration** Before running the migration, you'll need to install the migration tools: 1. Install local dependencies with `make`. 2. Install [golang-migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate) CLI. You can initiate the DB migration process by executing `make dev-migrate-db`. This will run the migration scripts in `/packages/api/migrations` against the local DB. ### Hosted Cloud HyperDX is also available as a hosted cloud service at [hyperdx.io](https://hyperdx.io). You can sign up for a free account and start sending data in minutes. ## Instrumenting Your App To get logs, metrics, traces, session replay, etc into HyperDX, you'll need to instrument your app to collect and send telemetry data over to your HyperDX instance. We provide a set of SDKs and integration options to make it easier to get started with HyperDX, such as [Browser](https://www.hyperdx.io/docs/install/browser), [Node.js](https://www.hyperdx.io/docs/install/javascript), and [Python](https://www.hyperdx.io/docs/install/python) You can find the full list in [our docs](https://www.hyperdx.io/docs). **OpenTelemetry** Additionally, HyperDX is compatible with [OpenTelemetry](https://opentelemetry.io/), a vendor-neutral standard for instrumenting your application backed by CNCF. Supported languages/platforms include: - Kubernetes - Javascript - Python - Java - Go - Ruby - PHP - .NET - Elixir - Rust (Full list [here](https://opentelemetry.io/docs/instrumentation/)) Once HyperDX is running, you can point your OpenTelemetry SDK to the OpenTelemetry collector spun up at `http://localhost:4318`. ## Local Mode Interested in using HyperDX for local development and debugging? Check out HyperDX Local, a single container local-optimized version of HyperDX that allows you to pipe OpenTelemetry telemetry (logs, metrics, traces) to a local instance of HyperDX running on your own machine. Get started with HyperDX Local by running the following Docker command: ```bash docker run -p 8000:8000 -p 4318:4318 -p 4317:4317 -p 8080:8080 -p 8002:8002 hyperdx/hyperdx-local ``` [Read more about HyperDX Local](./LOCAL.md) ## Contributing We welcome all contributions! There's many ways to contribute to the project, including but not limited to: - Opening a PR ([Contribution Guide](./CONTRIBUTING.md)) - [Submitting feature requests or bugs](https://github.com/hyperdxio/hyperdx/issues/new) - Improving our product or contribution documentation - Voting on [open issues](https://github.com/hyperdxio/hyperdx/issues) or contributing use cases to a feature request ## Motivation Our mission is to help engineers ship reliable software. To enable that, we believe every engineer needs to be able to easily leverage production telemetry to quickly solve burning production issues. However, in our experience, the existing tools we've used tend to fall short in a few ways: 1. They're expensive, and the pricing has failed to scale with TBs of telemetry becoming the norm, leading to teams aggressively cutting the amount of data they can collect. 2. They're hard to use, requiring full-time SREs to set up, and domain experts to use confidently. 3. They requiring hopping from tool to tool (logs, session replay, APM, exceptions, etc.) to stitch together the clues yourself. We're still early on in our journey, but are building in the open to solve these key issues in observability. We hope you give HyperDX a try and let us know how we're doing! ## Open Source vs Hosted Cloud HyperDX is open core, with most of our features available here under an MIT license. We have a cloud-hosted version available at [hyperdx.io](https://hyperdx.io) with a few [additional features](https://www.hyperdx.io/docs/oss-vs-cloud) beyond what's offered in the open source version. Our cloud hosted version exists so that we can build a sustainable business and continue building HyperDX as an open source platform. We hope to have more comprehensive documentation on how we balance between cloud-only and open source features in the future. In the meantime, we're highly aligned with Gitlab's [stewardship model](https://handbook.gitlab.com/handbook/company/stewardship/). ## Frequently Asked Questions #### How to suppress all logs from HyperDX itself ? To suppress logs of a service, you can comment out the `HYPERDX_API_KEY` environment variable in the docker-compose.yml file. The alternative is to set the `HYPERDX_LOG_LEVEL` environment variable to 'error' to only log errors. ## Contact - [Open an Issue](https://github.com/hyperdxio/hyperdx/issues/new) - [Discord](https://discord.gg/FErRRKU78j) - [Email](mailto:[email protected]) ## License [MIT](/LICENSE) ", Assign "at most 3 tags" to the expected json: {"id":"2030","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"