base on # Braintrust AI Proxy The Braintrust AI proxy offers a unified way to access the world's leading AI models through a single API, including models from [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://docs.anthropic.com/claude/reference/getting-started-with-the-api), [LLaMa 2](https://ai.meta.com/llama/), [Mistral](https://mistral.ai/), and more. The benefits of using the proxy include: - **Code Simplification**: Use a consistent API across different AI providers. - **Cost Reduction**: The proxy automatically caches results, reusing them when possible. - **Enhanced Observability**: Log requests automatically for better tracking and debugging. \[Coming soon\] See the full list of supported models [here](https://www.braintrust.dev/docs/guides/proxy#supported-models). To read more about why we launched the AI proxy, check out our [announcement blog post](https://braintrust.dev/blog/ai-proxy). This repository contains the code for the proxy — both the underlying implementation and wrappers that allow you to deploy it on [Vercel](https://vercel.com), [Cloudflare](https://developers.cloudflare.com/workers/), [AWS Lambda](https://aws.amazon.com/lambda/), or an [Express](https://expressjs.com/) server. ## Just let me try it! You can communicate with the proxy via the standard OpenAI drivers/API, and simply set the base url to `https://api.braintrust.dev/v1/proxy`. Try running the following script in your favorite language, twice. ### TypeScript ```javascript copy import { OpenAI } from "openai"; const client = new OpenAI({ baseURL: "https://api.braintrust.dev/v1/proxy", apiKey: process.env.OPENAI_API_KEY, // Can use Braintrust, Anthropic, etc. keys here too }); async function main() { const start = performance.now(); const response = await client.chat.completions.create({ model: "gpt-3.5-turbo", // // Can use claude-2, llama-2-13b-chat here too messages: [{ role: "user", content: "What is a proxy?" }], seed: 1, // A seed activates the proxy's cache }); console.log(response.choices[0].message.content); console.log(`Took ${(performance.now() - start) / 1000}s`); } main(); ``` ### Python ```python copy from openai import OpenAI import os import time client = OpenAI( base_url="https://api.braintrust.dev/v1/proxy", api_key=os.environ["OPENAI_API_KEY"], # Can use Braintrust, Anthropic, etc. keys here too ) start = time.time() response = client.chat.completions.create( model="gpt-3.5-turbo", # Can use claude-2, llama-2-13b-chat here too messages=[{"role": "user", "content": "What is a proxy?"}], seed=1, # A seed activates the proxy's cache ) print(response.choices[0].message.content) print(f"Took {time.time()-start}s") ``` ### cURL ```bash copy time curl -i https://api.braintrust.dev/v1/proxy/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "What is a proxy?" } ], "seed": 1 }' \ -H "Authorization: Bearer $OPENAI_API_KEY" # Can use Braintrust, Anthropic, etc. keys here too ``` ## Deploying You can find the full documentation for using the proxy [here](https://www.braintrust.dev/docs/guides/proxy). The proxy is hosted for you, with end-to-end encryption, at `https://api.braintrust.dev/v1/proxy`. However, you can also deploy it yourself and customize its behavior. To see docs for how to deploy on various platforms, see the READMEs in the corresponding folders: - [Vercel](./apis/vercel) - [Cloudflare](./apis/cloudflare) - [AWS Lambda](./apis/node) - [Express](./apis/node) ## Developing To build the proxy, install [pnpm](https://pnpm.io/installation) and run: ```bash pnpm install pnpm build ``` ", Assign "at most 3 tags" to the expected json: {"id":"5414","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"