base on Record voice notes & transcribe, summarize, and get tasks <a href="https://usenotesgpt.com/"> <img alt="NotesGPT – AI-powered voice note taking in seconds." src="/public/images/og-image.png"> <h1 align="center">notesGPT</h1> </a> <p align="center"> Generate action items from your notes in seconds. Powered by Convex, Together.ai, and Whisper. </p> <p align="center"> <a href="#tech-stack"><strong>Tech Stack</strong></a> · <a href="#deploy-your-own"><strong>Deploy Your Own</strong></a> · <a href="#future-tasks"><strong>Future Tasks</strong></a> </p> <br/> ## Tech Stack - [Convex](https://convex.dev/) for the database and cloud functions - Next.js [App Router](https://nextjs.org/docs/app) for the framework - [Together Inference](https://dub.sh/together-ai) for the LLM (Mixtral) - [Together Embeddings](https://dub.sh/together-ai) for the embeddings for search - [Convex File Storage](https://docs.convex.dev/file-storage) for storing voice notes - [Convex Vector search](https://docs.convex.dev/vector-search) for vector search - [Replicate](https://replicate.com/) for Whisper transcriptions - [Clerk](https://clerk.dev/) for user authentication - [Tailwind CSS](https://tailwindcss.com/) for styling ## Deploy Your Own You can deploy this template by setting up the following services and adding their environment variables: 1. Run `npm install` to install dependencies. 2. Run `npm run dev`. It will prompt you to log into [Convex](https://convex.dev) and create a project. 3. It will then ask you to supply the `CLERK_ISSUER_URL`. To do this: 1. Make a [Clerk](https://clerk.dev) account. 2. Copy both the `CLERK_SECRET_KEY` and `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` [API keys](https://dashboard.clerk.com/last-active?path=api-keys) into `.env.local`. 3. Do steps 1-3 [here](https://docs.convex.dev/auth/clerk) and copy the Issuer URL. It should look something like `https://some-animal-123.clerk.accounts.dev`. 4. Add `CLERK_ISSUER_URL` to your [Convex Environment Variables](https://dashboard.convex.dev/deployment/settings/environment-variables?var=CLERK_ISSUER_URL) (deep link also available in your terminal). Paste the Issuer URL as the value and click "Save". 4. Now your frontend and backend should be running and you should be able to log in but not record. 5. Make a [Together](https://dub.sh/together-ai) account to get your [API key](https://api.together.xyz/settings/api-keys). 6. Make a [Replicate](https://replicate.com) account to get your [API key](https://replicate.com/account/api-tokens). 7. Save your environment variables in Convex [as `REPLICATE_API_KEY` and `TOGETHER_API_KEY`](https://dashboard.convex.dev/deployment/settings/environment-variables?var=REPLICATE_API_KEY&var=TOGETHER_API_KEY). ## Future tasks - [ ] Keep recording for future playback and display it on the page somewhere - [ ] Animate the purple microphone to be in sync with your voice - [ ] Store completed action items for the future instead of fully deleting them - [ ] Make text/images in the landing page smaller to account for multiple screen sizes. - [ ] Make the search experience a little smoother overall by searching automatically on entering text - [ ] Be able to have this as a PWA if there's an easy step to do that - [ ] Prompt engineer the summary a little more to be more useful than what's currently displaying - [ ] Add a Notion integration to be able to get the transcript + summary + action items on there - [ ] UI updates to make it look a little nicer based on Youssef's redesign - [ ] Be able to edit action items after the fact and set a due date for them - [ ] Account for layout shift on the dashboard page when refreshing – show a loading state on content? - [ ] Make action items animate out + make checkbox rounded + add a little check icon on hover - [ ] Migrate to incredibly fast whisper ", Assign "at most 3 tags" to the expected json: {"id":"7831","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"