base on Make Mac apps accessible for AI agents <picture> <img alt="Shows a black Browser Use Logo in light color mode and a white one in dark color mode." src="./static/macos-use.png" width="full"> </picture> <br/> [](https://github.com/browser-use/macOS-use/stargazers) [](https://link.browser-use.com/discord) [](https://x.com/OfirOzeri) [](https://x.com/gregpr07) [](https://x.com/mamagnus00) <div align="center"> <h2>Tell your MacBook what to do, and it's done—across ANY app.</h2> Created by <a href="https://x.com/OfirOzeri">Ofir Ozeri </a>♥️ migrated in collaboration with <a href="https://x.com/mamagnus00">Magnus</a> and <a href="https://x.com/gregpr07">Gregor</a><br> </div> <br> macOS-use enables AI agents to interact with your Macbook [see it in action!](#demos) # Quick start ⚠️ Important: Review the [Warning](#warning) section before proceeding. <br> ### With pip: ```bash pip install mlx-use ``` ### From github Clone first <br> ```bash git clone https://github.com/browser-use/macOS-use.git && cd macOS-use ``` Don't forget API key <br>Supported providers: [OAI](https://platform.openai.com/docs/quickstart), [Anthropic](https://docs.anthropic.com/en/api/admin-api/apikeys/get-api-key) or [Gemini](https://ai.google.dev/gemini-api/docs/api-key) (deepseek R1 coming soon!) <br> At the moment, macOS-use works best with OAI or Anthropic API, although Gemini is free. While Gemini works great too, it is not as reliable. <br> ```bash cp .env.example .env ``` ```bash open ./.env ``` We recommend using macOS-use with uv environment <br> ```bash brew install uv && uv venv && source .venv/bin/activate ``` Install locally and you're good to go! try the first exmaple! <br> ```bash uv pip install --editable . && python examples/try.py ``` Try prompting it with ```bash open the calculator app ``` # Demos <h3> Click the GIF for the full video! </h3> [prompt](https://github.com/browser-use/macOS-use/blob/main/examples/calculate.py): Calculate how much is 5 X 4 and return the result, then call done. ```bash python examples/calculate.py ``` <br> [](https://x.com/OfirOzeri/status/1883110905665433681) <br/> [prompt](https://github.com/browser-use/macOS-use/blob/main/examples/login_to_auth0.py): Go to auth0.com, sign in with google auth, choose ofiroz91 gmail account, login to the website and call done when you finish. ```bash python examples/login_to_auth0.py ``` <br> [](https://x.com/OfirOzeri/status/1883455599423434966) <br/> [prompt](https://github.com/browser-use/macOS-use/blob/main/examples/check_time_online.py): Can you check what hour is Shabbat in israel today? call done when you finish. ```bash python examples/check_time_online.py ``` <br> [](https://x.com/OfirOzeri/status/1883109604416278552) <br> # Our Vision: TLDR: Tell every Apple device what to do, and see it done. on EVERY APP. <br><br> This project aimes to build the AI agent for the MLX by Apple framework that would allow the agent to perform any action on any Apple device. Our final goal is a open source that anyone can clone, powered by the [mlx](https://github.com/ml-explore/mlx) and [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) to run local private infrence at zero cost. ## Roadmap goals: 1. Support MacBooks at SOTA reliability - [ ] Refine the Agent prompting. - [ ] Release the first working version to pypi. - [ ] Improve self-correction. - [x] Adding ability to check which apps the machine has installed. - [x] Add feature to allow the agent to check existing apps if failing, e.g. calendar app actual name is iCal. - [ ] Add action for the agent to ask input from the user. - [ ] Test Test Test! and let us know what and how to improve! - [ ] Make task cheaper and more efficient. 2. Support local inference with small fine tuned model. - [ ] Add support for inference with local models using mlx and mlx-vlm. - [ ] Fine tune a small model that every device can run inference with. - [ ] SOTA reliability. 3. Support iPhone/iPad <br> # WARNING This project is still under development and user discretion is advised! macOS-use can and will use your do [login](#demos), use private credentials, [auth services](https://github.com/browser-use/macOS-use/blob/main/examples/login_to_auth0.py) or stored passwords to complete its task, launch and interact WITH EVERY APP and UI component in your MacBook and restrictions to the model are still under active development! It is not recommended to operate it unsupervised YET macOS-use WILL NOT STOP at captcha or any other forms of bot identifications, so once again, user discretion is advised. ## Disclaimer: As this is an early stage release, You might experience varying success rates depending on task prompt, we're actively working on improvements. <br> talk me on [X/Twitter](https://x.com/OfirOzeri) or contact me on [Discord](https://link.browser-use.com/discord), your input is crucial and highly valuable!<br> # Contributing We are a new project and would love contributors! Feel free to PR, open issues for bugs or feature requests. # Thanks I would like to extend my heartfelt thanks to [](https://x.com/gregpr07) and [](https://x.com/mamagnus00) for their incredible work in developing Browser Use. Their dedication and expertise have been invaluable, especially in helping with the migration process and I couldn't have done it without them! ", Assign "at most 3 tags" to the expected json: {"id":"14260","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"