base on Chat with your PDFs with AI <a href="https://www.pdftochat.com/"> <img alt="PDFToChat – Chat with your PDFs in seconds." src="./public/og-image.png"> <h1 align="center">PDFToChat</h1> </a> <p align="center"> Chat with your PDFs in seconds. Powered by Together AI and Pinecone. </p> <p align="center"> <a href="#tech-stack"><strong>Tech Stack</strong></a> · <a href="#deploy-your-own"><strong>Deploy Your Own</strong></a> · <a href="#common-errors"><strong>Common Errors</strong></a> · <a href="#credits"><strong>Credits</strong></a> · <a href="#future-tasks"><strong>Future Tasks</strong></a> </p> <br/> ## Tech Stack - Next.js [App Router](https://nextjs.org/docs/app) for the framework - Mixtral through [Together AI](https://dub.sh/together-ai) inference for the LLM - M2 Bert 80M through [Together AI](https://dub.sh/together-ai) for embeddings - [LangChain.js](https://js.langchain.com/docs/get_started/introduction/) for the RAG code - [MongoDB Atlas](https://www.mongodb.com/atlas/database) for the vector database - [Bytescale](https://www.bytescale.com/) for the PDF storage - [Vercel](https://vercel.com/) for hosting and for the postgres DB - [Clerk](https://clerk.dev/) for user authentication - [Tailwind CSS](https://tailwindcss.com/) for styling ## Deploy Your Own You can deploy this template to Vercel or any other host. Note that you'll need to: - Set up [Together.ai](https://dub.sh/together-ai/) - Set up a [MongoDB Atlas](https://www.mongodb.com/atlas/database) Atlas database with 768 dimensions - See instructions below for MongoDB - Set up [Bytescale](https://www.bytescale.com/) - Set up [Clerk](https://clerk.dev/) - Set up [Vercel](https://vercel.com/) - (Optional) Set up [LangSmith](https://smith.langchain.com/) for tracing. See the .example.env for a list of all the required environment variables. You will also need to prepare your database schema by running `npx prisma db push`. ### MongoDB Atlas To set up a [MongoDB Atlas](https://www.mongodb.com/atlas/database) database as the backing vectorstore, you will need to perform the following steps: 1. Sign up on their website, then create a database cluster. Find it under the `Database` sidebar tab. 2. Create a **collection** by switching to `Collections` the tab and creating a blank collection. 3. Create an **index** by switching to the `Atlas Search` tab and clicking `Create Search Index`. 4. Make sure you select `Atlas Vector Search - JSON Editor`, select the appropriate database and collection, and paste the following into the textbox: ```json { "fields": [ { "numDimensions": 768, "path": "embedding", "similarity": "euclidean", "type": "vector" }, { "path": "docstore_document_id", "type": "filter" } ] } ``` Note that the `numDimensions` is 768 dimensions to match the embeddings model we're using, and that we have another index on `docstore_document_id`. This allows us to filter later. You may call the index whatever you wish, just make a note of it! 5. Finally, retrieve and set the following environment variables: ```ini NEXT_PUBLIC_VECTORSTORE=mongodb # Set MongoDB Atlas as your vectorstore MONGODB_ATLAS_URI= # Connection string for your database. MONGODB_ATLAS_DB_NAME= # The name of your database. MONGODB_ATLAS_COLLECTION_NAME= # The name of your collection. MONGODB_ATLAS_INDEX_NAME= # The name of the index you just created. ``` ## Common errors - Check that you've created an `.env` file that contains your valid (and working) API keys, environment and index name. - Check that you've set the vector dimensions to `768` and that `index` matched your specified field in the `.env variable`. - Check that you've added a credit card on Together AI if you're hitting rate limiting issues due to the free tier ## Credits - [Youssef](https://twitter.com/YoussefUiUx) for the design of the app - [Mayo](https://twitter.com/mayowaoshin) for the original RAG repo and inspiration - [Jacob](https://twitter.com/Hacubu) for the LangChain help - Together AI, Bytescale, Pinecone, and Clerk for sponsoring ## Future tasks These are some future tasks that I have planned. Contributions are welcome! - [ ] Add a trash icon for folks to delete PDFs from the dashboard and implement delete functionality - [ ] Try different embedding models like UAE-large-v1 to see if it improves accuracy - [ ] Explore best practices for auto scrolling based on other chat apps like chatGPT - [ ] Do some prompt engineering for Mixtral to make replies as good as possible - [ ] Protect API routes by making sure users are signed in before executing chats - [ ] Run an initial benchmark on how accurate chunking / retrieval are - [ ] Research best practices for chunking and retrieval and play around with them – ideally run benchmarks - [ ] Try out Langsmith for more observability into how the RAG app runs - [ ] Add demo video to the homepage to demonstrate functionality more easily - [ ] Upgrade to Next.js 14 and fix any issues with that - [ ] Implement sources like perplexity to be clickable with more info - [ ] Add analytics to track the number of chats & errors - [ ] Make some changes to the default tailwind `prose` to decrease padding - [ ] Add an initial message with sample questions or just add them as bubbles on the page - [ ] Add an option to get answers as markdown or in regular paragraphs - [ ] Implement something like SWR to automatically revalidate data - [ ] Save chats for each user to get back to later in the postgres DB - [ ] Bring up a message to direct folks to compress PDFs if they're beyond 10MB - [ ] Use a self-designed custom uploader - [ ] Use a session tracking tool to better understand how folks are using the site - [ ] Add better error handling overall with appropriate toasts when actions fail - [ ] Add support for images in PDFs with something like [Nougat](https://replicate.com/meta/nougat) ", Assign "at most 3 tags" to the expected json: {"id":"7227","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"