base on :zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 500+ plugins. Alternative to Zapier, Rundeck, Camunda, Airflow... <p align="center"> <a href="https://www.kestra.io"> <img src="https://kestra.io/banner.png" alt="Kestra workflow orchestrator" /> </a> </p> <h1 align="center" style="border-bottom: none"> Event-Driven Declarative Orchestration Platform </h1> <div align="center"> <a href="https://github.com/kestra-io/kestra/releases"><img src="https://img.shields.io/github/tag-pre/kestra-io/kestra.svg?color=blueviolet" alt="Last Version" /></a> <a href="https://github.com/kestra-io/kestra/blob/develop/LICENSE"><img src="https://img.shields.io/github/license/kestra-io/kestra?color=blueviolet" alt="License" /></a> <a href="https://github.com/kestra-io/kestra/stargazers"><img src="https://img.shields.io/github/stars/kestra-io/kestra?color=blueviolet&logo=github" alt="Github star" /></a> <br> <a href="https://kestra.io"><img src="https://img.shields.io/badge/Website-kestra.io-192A4E?color=blueviolet" alt="Kestra infinitely scalable orchestration and scheduling platform"></a> <a href="https://kestra.io/slack"><img src="https://img.shields.io/badge/Slack-Join%20Community-blueviolet?logo=slack" alt="Slack"></a> </div> <br /> <p align="center"> <a href="https://x.com/kestra_io"><img height="25" src="https://kestra.io/twitter.svg" alt="X(formerly Twitter)" /></a> &nbsp; <a href="https://www.linkedin.com/company/kestra/"><img height="25" src="https://kestra.io/linkedin.svg" alt="linkedin" /></a> &nbsp; <a href="https://www.youtube.com/@kestra-io"><img height="25" src="https://kestra.io/youtube.svg" alt="youtube" /></a> &nbsp; </p> <p align="center"> <a href="https://go.kestra.io/video/product-overview" target="_blank"> <img src="https://kestra.io/startvideo.png" alt="Get started in 4 minutes with Kestra" width="640px" /> </a> </p> <p align="center" style="color:grey;"><i>Click on the image to learn how to get started with Kestra in 4 minutes.</i></p> ## 🌟 What is Kestra? Kestra is an open-source, event-driven orchestration platform that makes both **scheduled** and **event-driven** workflows easy. By bringing **Infrastructure as Code** best practices to data, process, and microservice orchestration, you can build reliable [workflows](https://kestra.io/docs/getting-started) directly from the UI in just a few lines of YAML. **Key Features:** - **Everything as Code and from the UI:** keep **workflows as code** with a **Git Version Control** integration, even when building them from the UI. - **Event-Driven & Scheduled Workflows:** automate both **scheduled** and **real-time** event-driven workflows via a simple `trigger` definition. - **Declarative YAML Interface:** define workflows using a simple configuration in the **built-in code editor**. - **Rich Plugin Ecosystem:** hundreds of plugins built in to extract data from any database, cloud storage, or API, and **run scripts in any language**. - **Intuitive UI & Code Editor:** build and visualize workflows directly from the UI with syntax highlighting, auto-completion and real-time syntax validation. - **Scalable:** designed to handle millions of workflows, with high availability and fault tolerance. - **Version Control Friendly:** write your workflows from the built-in code Editor and push them to your preferred Git branch directly from Kestra, enabling best practices with CI/CD pipelines and version control systems. - **Structure & Resilience**: tame chaos and bring resilience to your workflows with **namespaces**, **labels**, **subflows**, **retries**, **timeout**, **error handling**, **inputs**, **outputs** that generate artifacts in the UI, **variables**, **conditional branching**, **advanced scheduling**, **event triggers**, **backfills**, **dynamic tasks**, **sequential and parallel tasks**, and skip tasks or triggers when needed by setting the flag `disabled` to `true`. 🧑‍💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is **always managed declaratively in code**, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls). <p align="center"> <img src="https://kestra.io/adding-tasks.gif" alt="Adding new tasks in the UI"> </p> --- ## 🚀 Quick Start ### Try the Live Demo Try Kestra with our [**Live Demo**](https://demo.kestra.io/ui/login?auto). No installation required! ### Get Started Locally in 5 Minutes #### Launch Kestra in Docker Make sure that Docker is running. Then, start Kestra in a single command: ```bash docker run --pull=always --rm -it -p 8080:8080 --user=root \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /tmp:/tmp kestra/kestra:latest server local ``` If you're on Windows and use PowerShell: ```powershell docker run --pull=always --rm -it -p 8080:8080 --user=root ` -v "/var/run/docker.sock:/var/run/docker.sock" ` -v "C:/Temp:/tmp" kestra/kestra:latest server local ``` If you're on Windows and use Command Prompt (CMD): ```cmd docker run --pull=always --rm -it -p 8080:8080 --user=root ^ -v "/var/run/docker.sock:/var/run/docker.sock" ^ -v "C:/Temp:/tmp" kestra/kestra:latest server local ``` If you're on Windows and use WSL (Linux-based environment in Windows): ```bash docker run --pull=always --rm -it -p 8080:8080 --user=root \ -v "/var/run/docker.sock:/var/run/docker.sock" \ -v "C:/Temp:/tmp" kestra/kestra:latest server local ``` Check our [Installation Guide](https://kestra.io/docs/installation) for other deployment options (Docker Compose, Podman, Kubernetes, AWS, GCP, Azure, and more). Access the Kestra UI at [http://localhost:8080](http://localhost:8080) and start building your first flow! #### Your First Hello World Flow Create a new flow with the following content: ```yaml id: hello_world namespace: dev tasks: - id: say_hello type: io.kestra.plugin.core.log.Log message: "Hello, World!" ``` Run the flow and see the output in the UI! --- ## 🧩 Plugin Ecosystem Kestra's functionality is extended through a rich [ecosystem of plugins](https://kestra.io/plugins) that empower you to run tasks anywhere and code in any language, including Python, Node.js, R, Go, Shell, and more. Here's how Kestra plugins enhance your workflows: - **Run Anywhere:** - **Local or Remote Execution:** Execute tasks on your local machine, remote servers via SSH, or scale out to serverless containers using [Task Runners](https://kestra.io/docs/task-runners). - **Docker and Kubernetes Support:** Seamlessly run Docker containers within your workflows or launch Kubernetes jobs to handle compute-intensive workloads. - **Code in Any Language:** - **Scripting Support:** Write scripts in your preferred programming language. Kestra supports Python, Node.js, R, Go, Shell, and others, allowing you to integrate existing codebases and deployment patterns. - **Flexible Automation:** Execute shell commands, run SQL queries against various databases, and make HTTP requests to interact with APIs. - **Event-Driven and Real-Time Processing:** - **Real-Time Triggers:** React to events from external systems in real-time, such as file arrivals, new messages in message buses (Kafka, Redis, Pulsar, AMQP, MQTT, NATS, AWS SQS, Google Pub/Sub, Azure Event Hubs), and more. - **Custom Events:** Define custom events to trigger flows based on specific conditions or external signals, enabling highly responsive workflows. - **Cloud Integrations:** - **AWS, Google Cloud, Azure:** Integrate with a variety of cloud services to interact with storage solutions, messaging systems, compute resources, and more. - **Big Data Processing:** Run big data processing tasks using tools like Apache Spark or interact with analytics platforms like Google BigQuery. - **Monitoring and Notifications:** - **Stay Informed:** Send messages to Slack channels, email notifications, or trigger alerts in PagerDuty to keep your team updated on workflow statuses. Kestra's plugin ecosystem is continually expanding, allowing you to tailor the platform to your specific needs. Whether you're orchestrating complex data pipelines, automating scripts across multiple environments, or integrating with cloud services, there's likely a plugin to assist. And if not, you can always [build your own plugins](https://kestra.io/docs/plugin-developer-guide/) to extend Kestra's capabilities. 🧑‍💻 **Note:** This is just a glimpse of what Kestra plugins can do. Explore the full list on our [Plugins Page](https://kestra.io/plugins). --- ## 📚 Key Concepts - **Flows:** the core unit in Kestra, representing a workflow composed of tasks. - **Tasks:** individual units of work, such as running a script, moving data, or calling an API. - **Namespaces:** logical grouping of flows for organization and isolation. - **Triggers:** schedule or events that initiate the execution of flows. - **Inputs & Variables:** parameters and dynamic data passed into flows and tasks. --- ## 🎨 Build Workflows Visually Kestra provides an intuitive UI that allows you to interactively build and visualize your workflows: - **Drag-and-Drop Interface:** add and rearrange tasks from the Topology Editor. - **Real-Time Validation:** instant feedback on your workflow's syntax and structure to catch errors early. - **Auto-Completion:** smart suggestions as you type to write flow code quickly and without syntax errors. - **Live Topology View:** see your workflow as a Directed Acyclic Graph (DAG) that updates in real-time. --- ## 🔧 Extensible and Developer-Friendly ### Plugin Development Create custom plugins to extend Kestra's capabilities. Check out our [Plugin Developer Guide](https://kestra.io/docs/plugin-developer-guide/) to get started. ### Infrastructure as Code - **Version Control:** store your flows in Git repositories. - **CI/CD Integration:** automate deployment of flows using CI/CD pipelines. - **Terraform Provider:** manage Kestra resources with the [official Terraform provider](https://kestra.io/docs/terraform/). --- ## 🌐 Join the Community Stay connected and get support: - **Slack:** Join our [Slack community](https://kestra.io/slack) to ask questions and share ideas. - **LinkedIn:** Follow us on [LinkedIn](https://www.linkedin.com/company/kestra/) — next to Slack and GitHub, this is our main channel to share updates and product announcements. - **YouTube:** Subscribe to our [YouTube channel](https://www.youtube.com/@kestra-io) for educational video content. We publish new videos every week! - **X:** Follow us on [X](https://x.com/kestra_io) if you're still active there. --- ## 🤝 Contributing We welcome contributions of all kinds! - **Report Issues:** Found a bug or have a feature request? Open an [issue on GitHub](https://github.com/kestra-io/kestra/issues). - **Contribute Code:** Check out our [Contributor Guide](https://kestra.io/docs/getting-started/contributing) for initial guidelines, and explore our [good first issues](https://go.kestra.io/contributing) for beginner-friendly tasks to tackle first. - **Develop Plugins:** Build and share plugins using our [Plugin Developer Guide](https://kestra.io/docs/plugin-developer-guide/). - **Contribute to our Docs:** Contribute edits or updates to keep our [documentation](https://github.com/kestra-io/docs) top-notch. --- ## 📄 License Kestra is licensed under the Apache 2.0 License © [Kestra Technologies](https://kestra.io). --- ## ⭐️ Stay Updated Give our repository a star to stay informed about the latest features and updates! [](https://github.com/kestra-io/kestra) --- Thank you for considering Kestra for your workflow orchestration needs. We can't wait to see what you'll build! ", Assign "at most 3 tags" to the expected json: {"id":"2714","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"