base on FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis. # FastStream <b>Effortless event stream integration for your services</b> --- <p align="center"> <a href="https://trendshift.io/repositories/3043" target="_blank"><img src="https://trendshift.io/api/badge/repositories/3043" alt="airtai%2Ffaststream | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <br/> <br/> <a href="https://github.com/airtai/faststream/actions/workflows/pr_tests.yaml" target="_blank"> <img src="https://github.com/airtai/faststream/actions/workflows/pr_tests.yaml/badge.svg?branch=main" alt="Test Passing"/> </a> <a href="https://coverage-badge.samuelcolvin.workers.dev/redirect/airtai/faststream" target="_blank"> <img src="https://coverage-badge.samuelcolvin.workers.dev/airtai/faststream.svg" alt="Coverage"/> </a> <a href="https://www.pepy.tech/projects/faststream" target="_blank"> <img src="https://static.pepy.tech/personalized-badge/faststream?period=month&units=international_system&left_color=grey&right_color=green&left_text=downloads/month" alt="Downloads"/> </a> <a href="https://pypi.org/project/faststream" target="_blank"> <img src="https://img.shields.io/pypi/v/faststream?label=PyPI" alt="Package version"/> </a> <a href="https://pypi.org/project/faststream" target="_blank"> <img src="https://img.shields.io/pypi/pyversions/faststream.svg" alt="Supported Python versions"/> </a> <br/> <a href="https://github.com/airtai/faststream/actions/workflows/pr_codeql.yaml" target="_blank"> <img src="https://github.com/airtai/faststream/actions/workflows/pr_codeql.yaml/badge.svg" alt="CodeQL"/> </a> <a href="https://github.com/airtai/faststream/actions/workflows/pr_dependency-review.yaml" target="_blank"> <img src="https://github.com/airtai/faststream/actions/workflows/pr_dependency-review.yaml/badge.svg" alt="Dependency Review"/> </a> <a href="https://github.com/airtai/faststream/blob/main/LICENSE" target="_blank"> <img src="https://img.shields.io/github/license/airtai/faststream.png" alt="License"/> </a> <a href="https://github.com/airtai/faststream/blob/main/CODE_OF_CONDUCT.md" target="_blank"> <img src="https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg" alt="Code of Conduct"/> </a> <a href="https://discord.gg/qFm6aSqq59" target="_blank"> <img alt="Discord" src="https://img.shields.io/discord/1085457301214855171?logo=discord&label=EN"/> </a> <a href="https://t.me/python_faststream" target="_blank"> <img alt="Telegram" src="https://img.shields.io/badge/-telegram-black?color=blue&logo=telegram&label=RU"/> </a> <br/> <a href="https://gurubase.io/g/faststream" target="_blank"> <img src="https://img.shields.io/badge/Gurubase-Ask%20FastStream%20Guru-006BFF" alt="Gurubase"/> </a> </p> --- ## Features [**FastStream**](https://faststream.airt.ai/latest/) simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentation generation automatically. Making streaming microservices has never been easier. Designed with junior developers in mind, **FastStream** simplifies your work while keeping the door open for more advanced use cases. Here's a look at the core features that make **FastStream** a go-to framework for modern, data-centric microservices. - **Multiple Brokers**: **FastStream** provides a unified API to work across multiple message brokers ([**Kafka**](https://kafka.apache.org/), [**RabbitMQ**](https://www.rabbitmq.com/), [**NATS**](https://nats.io/), [**Redis**](https://redis.io/) support) - [**Pydantic Validation**](#writing-app-code): Leverage [**Pydantic's**](https://docs.pydantic.dev/) validation capabilities to serialize and validate incoming messages - [**Automatic Docs**](#project-documentation): Stay ahead with automatic [**AsyncAPI**](https://www.asyncapi.com/) documentation - **Intuitive**: Full-typed editor support makes your development experience smooth, catching errors before they reach runtime - [**Powerful Dependency Injection System**](#dependencies): Manage your service dependencies efficiently with **FastStream**'s built-in DI system - [**Testable**](#testing-the-service): Supports in-memory tests, making your CI/CD pipeline faster and more reliable - **Extensible**: Use extensions for lifespans, custom serialization and middleware - [**Integrations**](#any-framework): **FastStream** is fully compatible with any HTTP framework you want ([**FastAPI**](#fastapi-plugin) especially) That's **FastStream** in a nutshell—easy, efficient, and powerful. Whether you're just starting with streaming microservices or looking to scale, **FastStream** has got you covered. --- **Documentation**: <a href="https://faststream.airt.ai/latest/" target="_blank">https://faststream.airt.ai/latest/</a> --- ## History **FastStream** is a new package based on the ideas and experiences gained from [**FastKafka**](https://github.com/airtai/fastkafka) and [**Propan**](https://github.com/lancetnik/propan). By joining our forces, we picked up the best from both packages and created a unified way to write services capable of processing streamed data regardless of the underlying protocol. We'll continue to maintain both packages, but new development will be in this project. If you are starting a new service, this package is the recommended way to do it. --- ## Install **FastStream** works on **Linux**, **macOS**, **Windows** and most **Unix**-style operating systems. You can install it with `pip` as usual: ```sh pip install faststream[kafka] # or pip install faststream[rabbit] # or pip install faststream[nats] # or pip install faststream[redis] ``` By default **FastStream** uses **PydanticV2** written in **Rust**, but you can downgrade it manually, if your platform has no **Rust** support - **FastStream** will work correctly with **PydanticV1** as well. --- ## Writing app code **FastStream** brokers provide convenient function decorators `@broker.subscriber` and `@broker.publisher` to allow you to delegate the actual process of: - consuming and producing data to Event queues, and - decoding and encoding JSON-encoded messages These decorators make it easy to specify the processing logic for your consumers and producers, allowing you to focus on the core business logic of your application without worrying about the underlying integration. Also, **FastStream** uses [**Pydantic**](https://docs.pydantic.dev/) to parse input JSON-encoded data into Python objects, making it easy to work with structured data in your applications, so you can serialize your input messages just using type annotations. Here is an example Python app using **FastStream** that consumes data from an incoming data stream and outputs the data to another one: ```python from faststream import FastStream from faststream.kafka import KafkaBroker # from faststream.rabbit import RabbitBroker # from faststream.nats import NatsBroker # from faststream.redis import RedisBroker broker = KafkaBroker("localhost:9092") # broker = RabbitBroker("amqp://guest:guest@localhost:5672/") # broker = NatsBroker("nats://localhost:4222/") # broker = RedisBroker("redis://localhost:6379/") app = FastStream(broker) @broker.subscriber("in") @broker.publisher("out") async def handle_msg(user: str, user_id: int) -> str: return f"User: {user_id} - {user} registered" ``` Also, **Pydantic**’s [`BaseModel`](https://docs.pydantic.dev/usage/models/) class allows you to define messages using a declarative syntax, making it easy to specify the fields and types of your messages. ```python from pydantic import BaseModel, Field, PositiveInt from faststream import FastStream from faststream.kafka import KafkaBroker broker = KafkaBroker("localhost:9092") app = FastStream(broker) class User(BaseModel): user: str = Field(..., examples=["John"]) user_id: PositiveInt = Field(..., examples=["1"]) @broker.subscriber("in") @broker.publisher("out") async def handle_msg(data: User) -> str: return f"User: {data.user} - {data.user_id} registered" ``` --- ## Testing the service The service can be tested using the `TestBroker` context managers, which, by default, puts the Broker into "testing mode". The Tester will redirect your `subscriber` and `publisher` decorated functions to the InMemory brokers, allowing you to quickly test your app without the need for a running broker and all its dependencies. Using pytest, the test for our service would look like this: ```python # Code above omitted 👆 import pytest import pydantic from faststream.kafka import TestKafkaBroker @pytest.mark.asyncio async def test_correct(): async with TestKafkaBroker(broker) as br: await br.publish({ "user": "John", "user_id": 1, }, "in") @pytest.mark.asyncio async def test_invalid(): async with TestKafkaBroker(broker) as br: with pytest.raises(pydantic.ValidationError): await br.publish("wrong message", "in") ``` ## Running the application The application can be started using built-in **FastStream** CLI command. Before running the service, install **FastStream CLI** using the following command: ```shell pip install "faststream[cli]" ``` To run the service, use the **FastStream CLI** command and pass the module (in this case, the file where the app implementation is located) and the app symbol to the command. ``` shell faststream run basic:app ``` After running the command, you should see the following output: ``` shell INFO - FastStream app starting... INFO - input_data | - `HandleMsg` waiting for messages INFO - FastStream app started successfully! To exit press CTRL+C ``` Also, **FastStream** provides you with a great hot reload feature to improve your Development Experience ``` shell faststream run basic:app --reload ``` And multiprocessing horizontal scaling feature as well: ``` shell faststream run basic:app --workers 3 ``` You can learn more about **CLI** features [here](https://faststream.airt.ai/latest/getting-started/cli/) --- ## Project Documentation **FastStream** automatically generates documentation for your project according to the [**AsyncAPI**](https://www.asyncapi.com/) specification. You can work with both generated artifacts and place a web view of your documentation on resources available to related teams. The availability of such documentation significantly simplifies the integration of services: you can immediately see what channels and message formats the application works with. And most importantly, it won't cost anything - **FastStream** has already created the docs for you!  --- ## Dependencies **FastStream** (thanks to [**FastDepends**](https://lancetnik.github.io/FastDepends/)) has a dependency management system similar to `pytest fixtures` and `FastAPI Depends` at the same time. Function arguments declare which dependencies you want are needed, and a special decorator delivers them from the global Context object. ```python from faststream import Depends, Logger async def base_dep(user_id: int) -> bool: return True @broker.subscriber("in-test") async def base_handler(user: str, logger: Logger, dep: bool = Depends(base_dep)): assert dep is True logger.info(user) ``` --- ## HTTP Frameworks integrations ### Any Framework You can use **FastStream** `MQBrokers` without a `FastStream` application. Just *start* and *stop* them according to your application's lifespan. ```python from aiohttp import web from faststream.kafka import KafkaBroker broker = KafkaBroker("localhost:9092") @broker.subscriber("test") async def base_handler(body): print(body) async def start_broker(app): await broker.start() async def stop_broker(app): await broker.close() async def hello(request): return web.Response(text="Hello, world") app = web.Application() app.add_routes([web.get("/", hello)]) app.on_startup.append(start_broker) app.on_cleanup.append(stop_broker) if __name__ == "__main__": web.run_app(app) ``` ### **FastAPI** Plugin Also, **FastStream** can be used as part of **FastAPI**. Just import a **StreamRouter** you need and declare the message handler with the same `@router.subscriber(...)` and `@router.publisher(...)` decorators. ```python from fastapi import FastAPI from pydantic import BaseModel from faststream.kafka.fastapi import KafkaRouter router = KafkaRouter("localhost:9092") class Incoming(BaseModel): m: dict @router.subscriber("test") @router.publisher("response") async def hello(m: Incoming): return {"response": "Hello, world!"} app = FastAPI() app.include_router(router) ``` More integration features can be found [here](https://faststream.airt.ai/latest/getting-started/integrations/fastapi/) --- ## Stay in touch Please show your support and stay in touch by: - giving our [GitHub repository](https://github.com/airtai/faststream/) a star, and - joining our [EN Discord server](https://discord.gg/qFm6aSqq59) - joining our [RU Telegram group](https://t.me/python_faststream) Your support helps us to stay in touch with you and encourages us to continue developing and improving the framework. Thank you for your support! --- ## Contributors Thanks to all of these amazing people who made the project better! <a href="https://github.com/airtai/faststream/graphs/contributors"> <img src="https://contrib.rocks/image?repo=airtai/faststream"/> </a> ", Assign "at most 3 tags" to the expected json: {"id":"3043","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"