mirror of
https://github.com/asg017/sqlite-vec.git
synced 2026-04-25 00:36:56 +02:00
A vector search SQLite extension that runs anywhere!
* initial pass at PARTITION KEY support. * Initial pass, allow auxiliary columns on vec0 virtual tables * update TODO * Initial pass at metadata filtering * unit tests * gha this PR branch * fixup tests * doc internal * fix tests, KNN/rowids in * define SQLITE_INDEX_CONSTRAINT_OFFSET * whoops * update tests, syrupy, use uv * un ignore pyproject.toml * dot * tests/ * type error? * win: .exe, update error name * try fix macos python, paren around expr? * win bash? * dbg :( * explicit error * op * dbg win * win ./tests/.venv/Scripts/python.exe * block UPDATEs on partition key values for now * test this branch * accidentally removved "partition key type mistmatch" block during merge * typo ugh * bruv * start aux snapshots * drop aux shadow table on destroy * enforce column types * block WHERE constraints on auxiliary columns in KNN queries * support delete * support UPDATE on auxiliary columns * test this PR * dont inline that * test-metadata.py * memzero text buffer * stress test * more snpashot tests * rm double/int32, just float/int64 * finish type checking * long text support * DELETE support * UPDATE support * fix snapshot names * drop not-used in eqp * small fixes * boolean comparison handling * ensure error is raised when long string constraint * new version string for beta builds * typo whoops * ann-filtering-benchmark directory * test-case * updates * fix aux column error when using non-default rowid values, needs test * refactor some text knn filtering * rowids blob read only on text metadata filters * refactor * add failing test causes for non eq text knn * text knn NE * test cases diff * GT * text knn GT/GE fixes * text knn LT/LE * clean * vtab_in handling * unblock aux failures for now * guard sqlite3_vtab_in * else in guard? * fixes and tests * add broken shadow table test * rename _metadata_chunksNN shadown table to _metadatachunksNN, for proper shadowName detection * _metadata_text_NN shadow tables to _metadatatextNN * SQLITE_VEC_VERSION_MAJOR SQLITE_VEC_VERSION_MINOR and SQLITE_VEC_VERSION_PATCH in sqlite-vec.h * _info shadow table * forgot to update aux snapshot? * fix aux tests |
||
|---|---|---|
| .github | ||
| benchmarks | ||
| bindings | ||
| examples | ||
| scripts | ||
| site | ||
| tests | ||
| .gitignore | ||
| ARCHITECTURE.md | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| Makefile | ||
| README.md | ||
| reference.yaml | ||
| SECURITY.md | ||
| sqlite-dist.toml | ||
| sqlite-vec.c | ||
| sqlite-vec.h.tmpl | ||
| test.sql | ||
| tmp-static.py | ||
| TODO | ||
| VERSION | ||
sqlite-vec
An extremely small, "fast enough" vector search SQLite extension that runs
anywhere! A successor to sqlite-vss
Important
sqlite-vecis a pre-v1, so expect breaking changes!
- Store and query float, int8, and binary vectors in
vec0virtual tables - Written in pure C, no dependencies, runs anywhere SQLite runs (Linux/MacOS/Windows, in the browser with WASM, Raspberry Pis, etc.)
- Pre-filter vectors with
rowid IN (...)subqueries
sqlite-vec is a
Mozilla Builders project,
with additional sponsorship from
Fly.io ,
Turso, and
SQLite Cloud.
See the Sponsors section for more details.
Installing
See Installing sqlite-vec
for more details.
| Language | Install | More Info | |
|---|---|---|---|
| Python | pip install sqlite-vec |
sqlite-vec with Python |
|
| Node.js | npm install sqlite-vec |
sqlite-vec with Node.js |
|
| Ruby | gem install sqlite-vec |
sqlite-vec with Ruby |
|
| Go | go get -u github.com/asg017/sqlite-vec/bindings/go |
sqlite-vec with Go |
|
| Rust | cargo add sqlite-vec |
sqlite-vec with Rust |
|
| Datasette | datasette install datasette-sqlite-vec |
sqlite-vec with Datasette |
|
| rqlite | rqlited -extensions-path=sqlite-vec.tar.gz |
sqlite-vec with rqlite |
|
sqlite-utils |
sqlite-utils install sqlite-utils-sqlite-vec |
sqlite-vec with sqlite-utils |
|
| Github Release |
Sample usage
.load ./vec0
create virtual table vec_examples using vec0(
sample_embedding float[8]
);
-- vectors can be provided as JSON or in a compact binary format
insert into vec_examples(rowid, sample_embedding)
values
(1, '[-0.200, 0.250, 0.341, -0.211, 0.645, 0.935, -0.316, -0.924]'),
(2, '[0.443, -0.501, 0.355, -0.771, 0.707, -0.708, -0.185, 0.362]'),
(3, '[0.716, -0.927, 0.134, 0.052, -0.669, 0.793, -0.634, -0.162]'),
(4, '[-0.710, 0.330, 0.656, 0.041, -0.990, 0.726, 0.385, -0.958]');
-- KNN style query
select
rowid,
distance
from vec_examples
where sample_embedding match '[0.890, 0.544, 0.825, 0.961, 0.358, 0.0196, 0.521, 0.175]'
order by distance
limit 2;
/*
┌───────┬──────────────────┐
│ rowid │ distance │
├───────┼──────────────────┤
│ 2 │ 2.38687372207642 │
│ 1 │ 2.38978505134583 │
└───────┴──────────────────┘
*/
Sponsors
Development of sqlite-vec is supported by multiple generous sponsors! Mozilla
is the main sponsor through the new Builders project.
sqlite-vec is also sponsored by the following companies:
As well as multiple individual supporters on Github sponsors!
If your company interested in sponsoring sqlite-vec development, send me an
email to get more info: https://alexgarcia.xyz
See Also
sqlite-ecosystem, Maybe more 3rd party SQLite extensions I've developedsqlite-rembed, Generate text embeddings from remote APIs like OpenAI/Nomic/Ollama, meant for testing and SQL scriptssqlite-lembed, Generate text embeddings locally from embedding models in the.ggufformat