base on A toolkit to create optimal Production-readyRetrieval Augmented Generation(RAG) setup for your data   # [](https://www.python.org/) [](https://github.com/KruxAI/ragbuilder/releases/) [](https://github.com/KruxAI/ragbuilder/blob/master/LICENSE) [](https://github.com/KruxAI/ragbuilder/commit/)  RagBuilder is a toolkit that helps you create optimal Production-ready Retrieval-Augmented-Generation (RAG) setup for your data automatically. By performing hyperparameter tuning on various RAG parameters (Eg: chunking strategy: semantic, character etc., chunk size: 1000, 2000 etc.), RagBuilder evaluates these configurations against a test dataset to identify the best-performing setup for your data. Additionally, RagBuilder includes several state-of-the-art, pre-defined RAG templates that have shown strong performance across diverse datasets. So just bring your data, and RagBuilder will generate a production-grade RAG setup in just minutes. ## Features - **Hyperparameter Tuning**: Efficiently optimize your RAG configurations using Bayesian optimization - **Pre-defined RAG Templates**: Use state-of-the-art templates that have demonstrated strong performance Eg: Graph retriever, Contextual chunker etc.) - **Evaluation Dataset Options**: Generate synthetic test dataset or provide your own - **Component Access**: Direct access to vectorstore, retriever, and generator components - **API Deployment**: Easily deploy as an API service - **Project Persistence**: Save and load optimized RAG pipelines ## Installation ```bash # Create a new venv uv venv ragbuilder # Activate the new venv source ragbuilder/bin/activate # Install uv pip install ragbuilder ``` See other installation options here ([link](https://docs.ragbuilder.io/quickstart/#installation)) ## Quick Start ```python from ragbuilder import RAGBuilder # Initialize and optimize with defaults builder = RAGBuilder.from_source_with_defaults(input_source='https://lilianweng.github.io/posts/2023-06-23-agent/') results = builder.optimize() # Run a query through the complete pipeline response = results.invoke("What is HNSW?") # View optimization summary print(results.summary()) ``` ### Setting Default Models You can specify default LLM and embedding models that will be used throughout the pipeline: `````python from langchain_openai import AzureChatOpenAI, AzureOpenAIEmbeddings # Initialize with custom defaults builder = RAGBuilder.from_source_with_defaults( input_source='data.pdf', default_llm=AzureChatOpenAI(model="gpt-4o", temperature=0.0), default_embeddings=AzureOpenAIEmbeddings(model="text-embedding-3-large"), n_trials=20 # Set number of optimization trials ) # Or when creating a RAGBuilder instance with fine grained custom configuration builder = RAGBuilder( data_ingest_config=data_ingest_config, # Custom Data Ingestion parameters default_llm=AzureChatOpenAI(model="gpt-4o", temperature=0.0), default_embeddings=AzureOpenAIEmbeddings(model="text-embedding-3-large") ) ````` ## Configuration Guide ### Basic Configuration For most use cases, the default configuration provides good results: ```python builder = RAGBuilder.from_source_with_defaults( input_source='path/to/your/data', test_dataset='path/to/test/data' # Optional ) ``` ## Advanced Configuration For fine-grained control over your RAG pipeline, you can customize every aspect: ````python from ragbuilder.config import ( DataIngestOptionsConfig, RetrievalOptionsConfig, GenerationOptionsConfig ) # Configure data ingestion data_ingest_config = DataIngestOptionsConfig( input_source="data.pdf", document_loaders=[ {"type": "pymupdf"}, {"type": "unstructured"} ], chunking_strategies=[{ "type": "RecursiveCharacterTextSplitter", "chunker_kwargs": {"separators": ["\n\n", "\n", " ", ""]} }], chunk_size={"min": 500, "max": 2000, "stepsize": 500}, embedding_models=[{ "type": "openai", "model_kwargs": {"model": "text-embedding-3-large"} }] ) # Initialize with custom configs builder = RAGBuilder( data_ingest_config=data_ingest_config, default_llm=AzureChatOpenAI(model="gpt-4o", temperature=0.0), default_embeddings=AzureOpenAIEmbeddings(model="text-embedding-3-large") ) # Run individual module level optimization builder.optimize_data_ingest() # Configure retrieval options retrieval_config = RetrievalOptionsConfig( retrievers=[ { "type": "vector_similarity", "retriever_k": [20], "weight": 0.5 }, { "type": "bm25", "retriever_k": [20], "weight": 0.5 } ], rerankers=[{ "type": "BAAI/bge-reranker-base" }], top_k=[3, 5] ) # Run retrieval optimization with custom config builder.optimize_retrieval(retrieval_config) # Configure Generation related options gen_config = GenerationOptionsConfig( llms = [ LLMConfig(type="azure_openai", model_kwargs={'model':'gpt-4o-mini', 'temperature':0.2}), LLMConfig(type="azure_openai", model_kwargs={'model':'gpt-4o', 'temperature':0.2}), ], optimization={ "n_trials": 10, "n_jobs": 1, "study_name": "lillog_agents_study", "optimization_direction": "maximize" }, evaluation_config={"type": "ragas"}, ) # Run generation optimization with custom config builder.optimize_generation(gen_config) results = builder.optimization_results response = adv_results.invoke("What is HNSW?") ```` ## Component Options Reference ### Document Loaders - `unstructured`: General-purpose loader - `pymupdf`: Optimized for PDFs - `pypdf`: Alternative PDF loader - `web`: Web page loader - Custom loaders via `custom_class` ### Chunking Strategies - `RecursiveCharacterTextSplitter`: Recursive character text splitter - `CharacterTextSplitter`: Character text splitter - `MarkdownHeaderTextSplitter`: Markdown-header based splitter - `HTMLHeaderTextSplitter`: HTML-header based splitter - `SemanticChunker`: Semantic chunker - `TokenTextSplitter`: Token-based splitter - Custom splitters via `custom_class` ### Retrievers - `vector_similarity`: Vector similarity search - `vector_mmr`: Vector MMR search - `bm25`: Keyword-based search using BM25 - `multi_query`: Multi-query retrievers - `parent_doc_full`: Parent document full-doc retrieval - `parent_doc_large`: Parent document large-chunks retrieval - `graph`: Graph-based retrieval (requires Neo4j) - Custom retrievers via `custom_class` ### Rerankers - `BAAI/bge-reranker-base`: BGE base reranker - `mixedbread-ai/mxbai-rerank-base-v1`: mxbai reranker base v1 - `mixedbread-ai/mxbai-rerank-large-v1`: mxbai reranker large v1 - `cohere`: Cohere's reranking model - `jina`: Jina reranker - `flashrank`: Flaskrank reranker - `rankllm`: RankLLM reranker - `colbert`: Colbert reranker - Custom rerankers via `custom_class` ## Environment Variables Create a `.env` file in your project directory: ````env # Required OPENAI_API_KEY=your_key_here # Optional - For additional features MISTRAL_API_KEY=your_key_here COHERE_API_KEY=your_key_here AZURE_OPENAI_API_KEY=your_key_here AZURE_OPENAI_ENDPOINT=your_endpoint_here # For Graph-based RAG NEO4J_URI=bolt://localhost:7687 NEO4J_USERNAME=neo4j NEO4J_PASSWORD=your_password ```` ## Advanced Topics ### Custom Evaluation Metrics ```python from ragbuilder import EvaluationConfig config = EvaluationConfig( type="custom", custom_class="your_module.CustomEvaluator", evaluator_kwargs={ "metrics": ["precision", "recall", "f1_score"] } ) ``` ### Optimization Configuration Fine-tune the optimization parameters: ```python from ragbuilder import OptimizationConfig config = OptimizationConfig( n_trials=20, n_jobs=1, study_name="my_optimization", optimization_direction="maximize" ) ``` ## API Deployment RAGBuilder can be deployed as an API service: ````python # Initialize and optimize builder = RAGBuilder.from_source_with_defaults('data.pdf') results = builder.optimize() # Deploy as API builder.serve(host="0.0.0.0", port=8000) ```` Access via: - `POST /query` - Run queries through the RAG pipeline ## Project Management Save and load optimized RAG pipelines: ````python # Save project builder.save('rag_project/') # Load existing project builder = RAGBuilder.load('rag_project/') # Access components vectorstore = builder.data_ingest.get_vectorstore() retriever = builder.retrieval.get_retriever() generator = builder.generation.get_generator() ```` ## Best Practices 1. **Start Simple** - Begin with `from_source_with_defaults()` - Add complexity only when needed 2. **Test Data Quality** - Provide representative test queries - Use domain-specific evaluation metrics 3. **Resource Management** - Monitor memory usage with large datasets - Use chunking for large documents 4. **Production Deployment** - Save optimized projects for reuse - Monitor API performance metrics - Implement rate limiting for API endpoints ## Usage Analytics We collect anonymous usage metrics to improve RAGBuilder: - Number of optimization runs - Success/failure rates - No personal or business data is collected To opt-out set `ENABLE_ANALYTICS=False` in `.env`: ## Contributing We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. ", Assign "at most 3 tags" to the expected json: {"id":"11926","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"