base on A simple toy demo of a local voice assistant with whisper and large language model. # A Simple Voice Assistant Script English | [简体中文](README-CN.md) This is a simple Python script project that allows dialogue with a local large language model through voice. The voice recognition part of this project is from the [Apple MLX example repo](https://github.com/ml-explore/mlx-examples/tree/main/whisper), and the textual responses are generated using the Yi model from [01.AI](https://www.lingyiwanwu.com). For more details, see the [Acknowledgments](## Acknowledgments) section. ### File Structure ```bash ├───main.py ├───models ├───prompts ├───recordings ├───tools │ └───list_microphones.py ├───whisper ``` This project is a single-script project, with main.py containing all program logic. The `models/` folder stores model files. `prompts/` contains prompt words. `recordings/` holds temporary recordings. `tools/list_microphones.py` is a simple script to view the microphone list, used in `main.py` to specify the microphone number. `whisper/` is from the [Apple MLX example repo](https://github.com/ml-explore/mlx-examples/tree/main/whisper), used for recognizing user's voice input. ## Installation Guide This project is based on the Python programming language, and the Python version used for program operation is 3.11.5. It is recommended to configure the Python environment using [Anaconda](https://www.anaconda.com). The following setup process has been tested and passed on macOS systems. Windows and Linux can use speech_recognition and pyttsx3 to replace the whisper and say commands mentioned below. The following are console/terminal/shell commands. ### Environment Configuration ``` conda create -n VoiceAI python=3.11 conda activate VoiceAI pip install -r requirements.txt CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python # Install audio processing tools brew install portaudio pip install pyaudio ``` ### Model Files The model files are stored in the `models/` folder and specified in the script via the `MODEL_PATH` variable. It is recommended to download the gguf format models from TheBloke and XeIaso, where the 6B model has a smaller memory footprint: - [TheBloke/Yi-34B-Chat-GGUF](https://huggingface.co/TheBloke/Yi-34B-Chat-GGUF/blob/main/yi-34b-chat.Q8_0.gguf) - [XeIaso/Yi-6B-Chat-GGUF](https://huggingface.co/XeIaso/yi-chat-6B-GGUF/blob/main/yi-chat-6b.Q8_0.gguf) The voice recognition model is by default stored in `models/whisper-large-v3/`, specified in the script via `WHISP_PATH`. The [version](https://huggingface.co/mlx-community/whisper-large-v3-mlx) converted by mlx-community can be directly downloaded. #### Model Files The voice recognition part of this project is based on OpenAI's whisper model, its implementation comes from the [Apple MLX example repo](https://github.com/ml-explore/mlx-examples/tree/main/whisper). The version used in this project is from January 2024, #80d1867. In the future, users can fetch new versions as needed. The responses in this project are generated by the large language model Yi from [01.AI](https://www.lingyiwanwu.com), where Yi-34B-Chat is more powerful. The [8-bit quantized version made by TheBloke](https://huggingface.co/TheBloke/Yi-34B-Chat-GGUF) has a memory footprint of 39.04 GB and is recommended for use if hardware conditions permit. This model runs locally based on the [LangChain](https://www.langchain.com) framework and [llama.cpp by Georgi Gerganov](https://github.com/ggerganov/llama.cpp). Thank you to all selfless programmers for their contributions to the open-source community! ", Assign "at most 3 tags" to the expected json: {"id":"5821","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"