base on Building blocks for rapid development of GenAI applications <div align="center"> <h1>🐰 Ragbits</h1> *Building blocks for rapid development of GenAI applications* [Homepage](https://deepsense.ai/rd-hub/ragbits/) | [Documentation](https://ragbits.deepsense.ai) | [Contact](https://deepsense.ai/contact/) <a href="https://trendshift.io/repositories/13966" target="_blank"><img src="https://trendshift.io/api/badge/repositories/13966" alt="deepsense-ai%2Fragbits | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> [](https://pypi.org/project/ragbits) [](https://pypi.org/project/ragbits) [](https://pypi.org/project/ragbits) </div> --- ## Features ### 🔨 Build Reliable & Scalable GenAI Apps - **Swap LLMs anytime** – Switch between [100+ LLMs via LiteLLM](https://ragbits.deepsense.ai/how-to/llms/use_llms/) or run [local models](https://ragbits.deepsense.ai/how-to/llms/use_local_llms/). - **Type-safe LLM calls** – Use Python generics to [enforce strict type safety](https://ragbits.deepsense.ai/how-to/prompts/use_prompting/#how-to-configure-prompts-output-data-type) in model interactions. - **Bring your own vector store** – Connect to [Qdrant](https://ragbits.deepsense.ai/api_reference/core/vector-stores/#ragbits.core.vector_stores.qdrant.QdrantVectorStore), [PgVector](https://ragbits.deepsense.ai/api_reference/core/vector-stores/#ragbits.core.vector_stores.pgvector.PgVectorStore), and more with built-in support. - **Developer tools included** – [Manage vector stores](https://ragbits.deepsense.ai/cli/main/#ragbits-vector-store), query pipelines, and [test prompts from your terminal](https://ragbits.deepsense.ai/quickstart/quickstart1_prompts/#testing-the-prompt-from-the-cli). - **Modular installation** – Install only what you need, reducing dependencies and improving performance. ### 📚 Fast & Flexible RAG Processing - **Ingest 20+ formats** – Process PDFs, HTML, spreadsheets, presentations, and more. Process data using [Docling](https://github.com/docling-project/docling), [Unstructured](https://github.com/Unstructured-IO/unstructured) or create a custom parser. - **Handle complex data** – Extract tables, images, and structured content with built-in VLMs support. - **Connect to any data source** – Use prebuilt connectors for S3, GCS, Azure, or implement your own. - **Scale ingestion** – Process large datasets quickly with [Ray-based parallel processing](https://ragbits.deepsense.ai/how-to/document_search/distributed_ingestion/#how-to-ingest-documents-in-a-distributed-fashion). ### 🚀 Deploy & Monitor with Confidence - **Real-time observability** – Track performance with [OpenTelemetry](https://ragbits.deepsense.ai/how-to/project/use_tracing/#opentelemetry-trace-handler) and [CLI insights](https://ragbits.deepsense.ai/how-to/project/use_tracing/#cli-trace-handler). - **Built-in testing** – Validate prompts [with promptfoo](https://ragbits.deepsense.ai/how-to/prompts/promptfoo/) before deployment. - **Auto-optimization** – Continuously evaluate and refine model performance. - **Chat UI** – Deploy [chatbot interface](https://ragbits.deepsense.ai/how-to/chatbots/api/) with API, persistance and user feedback. ## Installation To get started quickly, you can install with: ```sh pip install ragbits ``` This is a starter bundle of packages, containing: - [`ragbits-core`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-core) - fundamental tools for working with prompts, LLMs and vector databases. - [`ragbits-agents`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-agents) - abstractions for building agentic systems. - [`ragbits-document-search`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-document-search) - retrieval and ingestion piplines for knowledge bases. - [`ragbits-evaluate`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-evaluate) - unified evaluation framework for Ragbits components. - [`ragbits-chat`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-chat) - full-stack infrastructure for building conversational AI applications. - [`ragbits-cli`](https://github.com/deepsense-ai/ragbits/tree/main/packages/ragbits-cli) - `ragbits` shell command for interacting with Ragbits components. Alternatively, you can use individual components of the stack by installing their respective packages. ## Quickstart ### Basics To define a prompt and run LLM: ```python import asyncio from pydantic import BaseModel from ragbits.core.llms import LiteLLM from ragbits.core.prompt import Prompt class QuestionAnswerPromptInput(BaseModel): question: str class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, str]): system_prompt = """ You are a question answering agent. Answer the question to the best of your ability. """ user_prompt = """ Question: {{ question }} """ llm = LiteLLM(model_name="gpt-4.1-nano") async def main() -> None: prompt = QuestionAnswerPrompt(QuestionAnswerPromptInput(question="What are high memory and low memory on linux?")) response = await llm.generate(prompt) print(response) if __name__ == "__main__": asyncio.run(main()) ``` ### Document Search To build and query a simple vector store index: ```python import asyncio from ragbits.core.embeddings import LiteLLMEmbedder from ragbits.core.vector_stores import InMemoryVectorStore from ragbits.document_search import DocumentSearch embedder = LiteLLMEmbedder(model_name="text-embedding-3-small") vector_store = InMemoryVectorStore(embedder=embedder) document_search = DocumentSearch(vector_store=vector_store) async def run() -> None: await document_search.ingest("web://https://arxiv.org/pdf/1706.03762") result = await document_search.search("What are the key findings presented in this paper?") print(result) if __name__ == "__main__": asyncio.run(run()) ``` ### Retrieval-Augmented Generation To build a simple RAG pipeline: ```python import asyncio from collections.abc import Iterable from pydantic import BaseModel from ragbits.core.embeddings import LiteLLMEmbedder from ragbits.core.llms import LiteLLM from ragbits.core.prompt import Prompt from ragbits.core.vector_stores import InMemoryVectorStore from ragbits.document_search import DocumentSearch from ragbits.document_search.documents.element import Element class QuestionAnswerPromptInput(BaseModel): question: str context: Iterable[Element] class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, str]): system_prompt = """ You are a question answering agent. Answer the question that will be provided using context. If in the given context there is not enough information refuse to answer. """ user_prompt = """ Question: {{ question }} Context: {% for chunk in context %}{{ chunk.text_representation }}{%- endfor %} """ llm = LiteLLM(model_name="gpt-4.1-nano") embedder = LiteLLMEmbedder(model_name="text-embedding-3-small") vector_store = InMemoryVectorStore(embedder=embedder) document_search = DocumentSearch(vector_store=vector_store) async def run() -> None: question = "What are the key findings presented in this paper?" await document_search.ingest("web://https://arxiv.org/pdf/1706.03762") chunks = await document_search.search(question) prompt = QuestionAnswerPrompt(QuestionAnswerPromptInput(question=question, context=chunks)) response = await llm.generate(prompt) print(response) if __name__ == "__main__": asyncio.run(run()) ``` ### Chat UI To expose your RAG application through Ragbits UI: ```python from collections.abc import AsyncGenerator, Iterable from pydantic import BaseModel from ragbits.chat.api import RagbitsAPI from ragbits.chat.interface import ChatInterface from ragbits.chat.interface.types import ChatContext, ChatResponse from ragbits.core.embeddings import LiteLLMEmbedder from ragbits.core.llms import LiteLLM from ragbits.core.prompt import Prompt, ChatFormat from ragbits.core.vector_stores import InMemoryVectorStore from ragbits.document_search import DocumentSearch from ragbits.document_search.documents.element import Element class QuestionAnswerPromptInput(BaseModel): question: str context: Iterable[Element] class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, str]): system_prompt = """ You are a question answering agent. Answer the question that will be provided using context. If in the given context there is not enough information refuse to answer. """ user_prompt = """ Question: {{ question }} Context: {% for chunk in context %}{{ chunk.text_representation }}{%- endfor %} """ class MyChat(ChatInterface): async def setup(self) -> None: self.llm = LiteLLM(model_name="gpt-4.1-nano") self.embedder = LiteLLMEmbedder(model_name="text-embedding-3-small") self.vector_store = InMemoryVectorStore(embedder=self.embedder) self.document_search = DocumentSearch(vector_store=self.vector_store) await self.document_search.ingest("web://https://arxiv.org/pdf/1706.03762") async def chat( self, message: str, history: ChatFormat | None = None, context: ChatContext | None = None, ) -> AsyncGenerator[ChatResponse, None]: chunks = await self.document_search.search(message) prompt = QuestionAnswerPrompt(QuestionAnswerPromptInput(question=message, context=chunks)) async for text in self.llm.generate_streaming(prompt): yield self.create_text_response(text) if __name__ == "__main__": api = RagbitsAPI(MyChat) api.run() ``` ## Rapid development Create Ragbits projects from templates: ```sh uvx create-ragbits-app ``` Explore `create-ragbits-app` repo [here](https://github.com/deepsense-ai/create-ragbits-app). If you have a new idea for a template, feel free to contribute! ## Documentation - [Quickstart](https://ragbits.deepsense.ai/quickstart/quickstart1_prompts/) - Get started with Ragbits in a few minutes - [How-to](https://ragbits.deepsense.ai/how-to/prompts/use_prompting) - Learn how to use Ragbits in your projects - [CLI](https://ragbits.deepsense.ai/cli/main) - Learn how to run Ragbits in your terminal - [API reference](https://ragbits.deepsense.ai/api_reference/core/prompt) - Explore the underlying Ragbits API ## Contributing We welcome contributions! Please read [CONTRIBUTING.md](https://github.com/deepsense-ai/ragbits/tree/main/CONTRIBUTING.md) for more information. ## License Ragbits is licensed under the [MIT License](https://github.com/deepsense-ai/ragbits/tree/main/LICENSE). ", Assign "at most 3 tags" to the expected json: {"id":"13966","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"