base on RESTai is an AIaaS (AI as a Service) open-source platform. Built on top of LlamaIndex & Langchain. Supports any public LLM supported by LlamaIndex and any local LLM suported by Ollama/vLLM/etc. Precise embeddings usage and tuning. Built-in image generation (Dall-E, SD, Flux) and dynamic loading generators. <!-- markdownlint-disable MD033 --> <h1 align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/restai-logo.png" alt="RestAI Logo"/> </br>RESTai </h1> <p align="center"> <strong>AIaaS (AI as a Service) for everyone. Create AI projects and consume them using a simple REST API.</strong> </p> <h2 align="center"> Demo: <a href="https://ai.ince.pt">https://ai.ince.pt</a> Username: <code>demo</code> Password: <code>demo</code> </h2> <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/home.png" alt="RESTai Home"/> </div> <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/out.gif" alt="RESTai Video"/> </div> ## Features - **Projects**: There are multiple project types, each with its own features. ([rag](https://github.com/apocas/restai?tab=readme-ov-file#rag), [ragsql](https://github.com/apocas/restai?tab=readme-ov-file#ragsql), [inference](https://github.com/apocas/restai?tab=readme-ov-file#inference), [vision](https://github.com/apocas/restai?tab=readme-ov-file#vision), [router](https://github.com/apocas/restai?tab=readme-ov-file#router), [agent](https://github.com/apocas/restai?tab=readme-ov-file#agent)) - **Users**: A user represents a user of the system. It's used for authentication and authorization (basic auth). Each user may have access to multiple projects. - **LLMs**: Supports any public LLM supported by LlamaIndex. Which includes any local LLM supported by Ollama, LiteLLM, etc. - **VRAM**: Automatic VRAM management. Local image generators are run in a separate process, which allows for better resource management. - **API**: The API is a first-class citizen of RestAI. All endpoints are documented using [Swagger](https://apocas.github.io/restai/). - **Frontend**: There is a frontend available at [restai-frontend](https://github.com/apocas/restai-frontend) - **Image Generation**: Supports local and remote image generators. Local image generators are run in a separate process. New generators are [easily added](https://github.com/apocas/restai?tab=readme-ov-file#image-generators) and loaded dynamically. ## Project Types ### RAG <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/rag.png" width="750" style="margin: 10px;" alt="RESTai RAG"/> </div> - **Embeddings**: You may use any embeddings model supported by llamaindex. Check embeddings [definition](modules/embeddings.py). - **Vectorstore**: There are two vectorstores supported: `Chroma` and `Redis` - **Retrieval**: It features an embeddings search and score evaluator, which allows you to evaluate the quality of your embeddings and simulate the RAG process before the LLM. Reranking is also supported, ColBERT and LLM based. - **Loaders**: You may use any loader supported by llamaindex. - **Sandboxed mode**: RAG projects have "sandboxed" mode, which means that a locked default answer will be given when there aren't embeddings for the provided question. This is useful for chatbots, where you want to provide a default answer when the LLM doesn't know how to answer the question, reduncing hallucination. - **Evaluation**: You may evaluate your RAG agent using [deepeval](https://github.com/confident-ai/deepeval). Using the `eval` property in the RAG endpoint. ### RAGSQL <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/ragsql.jpg" width="750" style="margin: 10px;"/> </div> - **Connection**: Supply a MySQL or PostgreSQL connection string and it will automatically crawl the DB schema, using table and column names it’s able to figure out how to translate the question to sql and then write a response. ### Agent - ReAct Agents, specify which tools to use in the project and the agent will figure out how to use them to achieve the objective. - New tools are easily added. Just create a new tool in the `tools` folder and it will be automatically picked up by RESTai. Check the `app/llms/tools` folder for examples using the builtin tools. - **Tools**: Supply all the tools names you want the Agent to use in this project. (separated by commas) - **Terminal**: Core tool that allows the agent to execute commands via SSH. (using [containerssh.io](https://containerssh.io) or similar is recommended) <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/agent1.png" width="40%" style="margin: 10px;"/> <img src="https://github.com/apocas/restai/blob/master/readme/assets/agent2.png" width="40%" style="margin: 10px;"/> </div> ### Inference <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/inference.png" width="750" style="margin: 10px;"/> </div> ### Vision - **img2text**: RESTai supports virtually any vision model. #### LLaVA <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/llava.png" width="25%" style="margin: 10px;"/> </div> ### Image Generators - New generators are easily added. Just create a new tool in the `generators` folder and it will be automatically picked up by RESTai. Check the `app/image/workers` folder for examples using the builtin generators. - **text2img**: RESTai supports txt2image like Stable Diffusion, Flux, DallE... - **img2img**: RESTai supports img2img like BMBG2, InstantID, ... #### Flux1 <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/flux1.png" width="50%" style="margin: 10px;"/> </div> #### Stable Diffusion & [InstantID](https://github.com/InstantID/InstantID) <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/vision_sd.png" width="25%" style="margin: 10px;"/> <img src="https://github.com/apocas/restai/blob/master/readme/assets/avatar.png" width="25%" style="margin: 10px;"/> <img src="https://github.com/apocas/restai/blob/master/readme/assets/rmbg2.png" width="25%" style="margin: 10px;"/> </div> ### Router - Routes a message to the most suitable project. It's useful when you have multiple projects and you want to route the question to the most suitable one. <div align="center"> <img src="https://github.com/apocas/restai/blob/master/readme/assets/router.png" width="750" style="margin: 10px;"/> </div> - **Routes**: Very similar to Zero Shot React strategy, but each route is a project. The router will route the question to the project that has the highest score. It's useful when you have multiple projects and you want to route the question to the most suitable one. ## LLMs - You may use any LLM supported by Ollama and/or LlamaIndex. ## Installation - RESTai uses [Poetry](https://python-poetry.org/) to manage dependencies. Install it with `pip install poetry`. ## Development - `make install` - `make dev` (starts RESTai in development mode) ## Production - `make install` - `make start` ## Docker - Edit the .env file accordingly - `docker compose --env-file .env up --build` You can specify profiles `docker compose --profile redis --profile mysql ....` to include additional components like the redis cache backend or a DB server, here are the supported profiles: - `--profile redis` Starts and sets redis as the cache backend - `--profile mysql` Starts and enables Mysql as the database server - `--profile postgres` Starts and enables Postgres as the database server The variables MYSQL_HOST and POSTGRES_HOST should match the names of the respective services "mysql" and "postgres" and not localhost or 127.0.0.1 when using the containers To delete everything or a specific container don't forget to pass the necessary profiles to the compose command, EX: - Removing everything `docker compose --profile mysql --profile postgres down --rmi all` - Removing singular database volume `docker compose --profile mysql down --volumes` *Note: the local_cache volume will also get removed since it's in the main service and not in any profile* ## API - **Endpoints**: All the API endpoints are documented and available at: [Endpoints](https://apocas.github.io/restai/api.html) - **Swagger**: Swagger/OpenAPI documentation: [Swagger](https://apocas.github.io/restai/swagger/) ## Frontend - Source code at [https://github.com/apocas/restai-frontend](https://github.com/apocas/restai-frontend). - `make install` automatically installs the frontend. ## Tests - Tests are implemented using `pytest`. Run them with `make test`. ## License Pedro Dias - [@pedromdias](https://twitter.com/pedromdias) Licensed under the Apache license, version 2.0 (the "license"); You may not use this file except in compliance with the license. You may obtain a copy of the license at: http://www.apache.org/licenses/LICENSE-2.0.html Unless required by applicable law or agreed to in writing, software distributed under the license is distributed on an "as is" basis, without warranties or conditions of any kind, either express or implied. See the license for the specific language governing permissions and limitations under the license.", Assign "at most 3 tags" to the expected json: {"id":"9031","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"