base on A vector search SQLite extension that runs anywhere! # `sqlite-vec` [](https://discord.gg/Ve7WeCJFXk) An extremely small, "fast enough" vector search SQLite extension that runs anywhere! A successor to [`sqlite-vss`](https://github.com/asg017/sqlite-vss) <!-- deno-fmt-ignore-start --> > [!IMPORTANT] > _`sqlite-vec` is a pre-v1, so expect breaking changes!_ <!-- deno-fmt-ignore-end --> - Store and query float, int8, and binary vectors in `vec0` virtual tables - Written in pure C, no dependencies, runs anywhere SQLite runs (Linux/MacOS/Windows, in the browser with WASM, Raspberry Pis, etc.) - Pre-filter vectors with `rowid IN (...)` subqueries <p align="center"> <a href="https://hacks.mozilla.org/2024/06/sponsoring-sqlite-vec-to-enable-more-powerful-local-ai-applications/"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/logos/mozilla.dark.svg"> <source media="(prefers-color-scheme: light)" srcset="./.github/logos/mozilla.svg"> <img alt="Mozilla Builders logo" width=400> </picture> </a> </p> <p align="center"> <i> <code>sqlite-vec</code> is a <a href="https://hacks.mozilla.org/2024/06/sponsoring-sqlite-vec-to-enable-more-powerful-local-ai-applications/">Mozilla Builders project</a>, with additional sponsorship from <a href="https://fly.io/"><img width=14px src="./.github/logos/flyio.small.ico"/> Fly.io </a>, <a href="https://tur.so/sqlite-vec"><img width=14px src="./.github/logos/turso.small.ico"/> Turso</a>, and <a href="https://sqlitecloud.io/"><img width=14px src="./.github/logos/sqlitecloud.small.svg"/> SQLite Cloud</a>. See <a href="#sponsors">the Sponsors section</a> for more details. </i> </p> ## Installing See [Installing `sqlite-vec`](https://alexgarcia.xyz/sqlite-vec/installation.html) for more details. | Language | Install | More Info | | | -------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Python | `pip install sqlite-vec` | [`sqlite-vec` with Python](https://alexgarcia.xyz/sqlite-vec/python.html) | [](https://pypi.org/project/sqlite-vec/) | | Node.js | `npm install sqlite-vec` | [`sqlite-vec` with Node.js](https://alexgarcia.xyz/sqlite-vec/js.html) | [](https://www.npmjs.com/package/sqlite-vec) | | Ruby | `gem install sqlite-vec` | [`sqlite-vec` with Ruby](https://alexgarcia.xyz/sqlite-vec/ruby.html) |  | | Go | `go get -u github.com/asg017/sqlite-vec/bindings/go` | [`sqlite-vec` with Go](https://alexgarcia.xyz/sqlite-vec/go.html) | [](https://pkg.go.dev/github.com/asg017/asg017/sqlite-vec-go-bindings/cgo) | | Rust | `cargo add sqlite-vec` | [`sqlite-vec` with Rust](https://alexgarcia.xyz/sqlite-vec/rust.html) | [](https://crates.io/crates/sqlite-vec) | | Datasette | `datasette install datasette-sqlite-vec` | [`sqlite-vec` with Datasette](https://alexgarcia.xyz/sqlite-vec/datasette.html) | [](https://datasette.io/plugins/datasette-sqlite-vec) | | rqlite | `rqlited -extensions-path=sqlite-vec.tar.gz` | [`sqlite-vec` with rqlite](https://alexgarcia.xyz/sqlite-vec/rqlite.html) | [](https://rqlite.io/docs/guides/extensions/) | | `sqlite-utils` | `sqlite-utils install sqlite-utils-sqlite-vec` | [`sqlite-vec` with sqlite-utils](https://alexgarcia.xyz/sqlite-vec/sqlite-utils.html) | [](https://datasette.io/plugins/datasette-sqlite-vec) | | Github Release | | |  | ## Sample usage ```sql .load ./vec0 create virtual table vec_examples using vec0( sample_embedding float[8] ); -- vectors can be provided as JSON or in a compact binary format insert into vec_examples(rowid, sample_embedding) values (1, '[-0.200, 0.250, 0.341, -0.211, 0.645, 0.935, -0.316, -0.924]'), (2, '[0.443, -0.501, 0.355, -0.771, 0.707, -0.708, -0.185, 0.362]'), (3, '[0.716, -0.927, 0.134, 0.052, -0.669, 0.793, -0.634, -0.162]'), (4, '[-0.710, 0.330, 0.656, 0.041, -0.990, 0.726, 0.385, -0.958]'); -- KNN style query select rowid, distance from vec_examples where sample_embedding match '[0.890, 0.544, 0.825, 0.961, 0.358, 0.0196, 0.521, 0.175]' order by distance limit 2; /* ┌───────┬──────────────────┐ │ rowid │ distance │ ├───────┼──────────────────┤ │ 2 │ 2.38687372207642 │ │ 1 │ 2.38978505134583 │ └───────┴──────────────────┘ */ ``` ## Sponsors Development of `sqlite-vec` is supported by multiple generous sponsors! Mozilla is the main sponsor through the new Builders project. <p align="center"> <a href="https://hacks.mozilla.org/2024/06/sponsoring-sqlite-vec-to-enable-more-powerful-local-ai-applications/"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/logos/mozilla.dark.svg"> <source media="(prefers-color-scheme: light)" srcset="./.github/logos/mozilla.svg"> <img alt="Mozilla Builders logo" width=400> </picture> </a> </p> `sqlite-vec` is also sponsored by the following companies: <a href="https://fly.io/"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/logos/flyio.dark.svg"> <source media="(prefers-color-scheme: light)" srcset="./.github/logos/flyio.svg"> <img alt="Fly.io logo" src="./.github/logos/flyio.svg" width="32%"> </picture> </a> <a href="https://tur.so/sqlite-vec"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/logos/turso.svg"> <source media="(prefers-color-scheme: light)" srcset="./.github/logos/turso.svg"> <img alt="Turso logo" src="./.github/logos/turso.svg" width="32%"> </picture> </a> <a href="https://sqlitecloud.io/"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./.github/logos/sqlitecloud.dark.svg"> <source media="(prefers-color-scheme: light)" srcset="./.github/logos/sqlitecloud.svg"> <img alt="SQLite Cloud logo" src="./.github/logos/flyio.svg" width="32%"> </picture> </a> As well as multiple individual supporters on [Github sponsors](https://github.com/sponsors/asg017/)! If your company interested in sponsoring `sqlite-vec` development, send me an email to get more info: https://alexgarcia.xyz ## See Also - [**`sqlite-ecosystem`**](https://github.com/asg017/sqlite-ecosystem), Maybe more 3rd party SQLite extensions I've developed - [**`sqlite-rembed`**](https://github.com/asg017/sqlite-rembed), Generate text embeddings from remote APIs like OpenAI/Nomic/Ollama, meant for testing and SQL scripts - [**`sqlite-lembed`**](https://github.com/asg017/sqlite-lembed), Generate text embeddings locally from embedding models in the `.gguf` format ", Assign "at most 3 tags" to the expected json: {"id":"9907","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"