Commit graph

38 commits

Author SHA1 Message Date
Alex Garcia
0de765f457
Add ANN search support for vec0 virtual table (#273)
Add approximate nearest neighbor infrastructure to vec0: shared distance
dispatch (vec0_distance_full), flat index type with parser, NEON-optimized
cosine/Hamming for float32/int8, amalgamation script, and benchmark suite
(benchmarks-ann/) with ground-truth generation and profiling tools. Remove
unused vec_npy_each/vec_static_blobs code, fix missing stdint.h include.
2026-03-31 01:03:32 -07:00
Alex Garcia
681f9028af --managed-python, fix flakey tests 2026-02-13 07:08:48 -08:00
Alex Garcia
8bc206fa0b test: drop vec_xyz if it exists 2026-02-13 07:05:39 -08:00
Alex Garcia
44e4438ed5 fix segfault on invalid vec_each() input, fixes #163 2025-01-10 14:44:37 -08:00
Alex Garcia
352f953fc0
Metadata filtering (#124)
* initial pass at PARTITION KEY support.

* Initial pass, allow auxiliary columns on vec0 virtual tables

* update TODO

* Initial pass at metadata filtering

* unit tests

* gha this PR branch

* fixup tests

* doc internal

* fix tests, KNN/rowids in

* define SQLITE_INDEX_CONSTRAINT_OFFSET

* whoops

* update tests, syrupy, use uv

* un ignore pyproject.toml

* dot

* tests/

* type error?

* win: .exe, update error name

* try fix macos python, paren around expr?

* win bash?

* dbg :(

* explicit error

* op

* dbg win

* win ./tests/.venv/Scripts/python.exe

* block UPDATEs on partition key values for now

* test this branch

* accidentally removved "partition key type mistmatch" block during merge

* typo ugh

* bruv

* start aux snapshots

* drop aux shadow table on destroy

* enforce column types

* block WHERE constraints on auxiliary columns in KNN queries

* support delete

* support UPDATE on auxiliary columns

* test this PR

* dont inline that

* test-metadata.py

* memzero text buffer

* stress test

* more snpashot tests

* rm double/int32, just float/int64

* finish type checking

* long text support

* DELETE support

* UPDATE support

* fix snapshot names

* drop not-used in eqp

* small fixes

* boolean comparison handling

* ensure error is raised when long string constraint

* new version string for beta builds

* typo whoops

* ann-filtering-benchmark directory

* test-case

* updates

* fix aux column error when using non-default rowid values, needs test

* refactor some text knn filtering

* rowids blob read only on text metadata filters

* refactor

* add failing test causes for non eq text knn

* text knn NE

* test cases diff

* GT

* text knn GT/GE fixes

* text knn LT/LE

* clean

* vtab_in handling

* unblock aux failures for now

* guard sqlite3_vtab_in

* else in guard?

* fixes and tests

* add broken shadow table test

* rename _metadata_chunksNN shadown table to _metadatachunksNN, for proper shadowName detection

* _metadata_text_NN shadow tables to _metadatatextNN

* SQLITE_VEC_VERSION_MAJOR SQLITE_VEC_VERSION_MINOR and SQLITE_VEC_VERSION_PATCH in sqlite-vec.h

* _info shadow table

* forgot to update aux snapshot?

* fix aux tests
2024-11-20 00:59:34 -08:00
Alex Garcia
6658624172
PARTITION KEY support (#122)
* initial pass at PARTITION KEY support.

* unit tests

* gha this PR branch

* fixup tests

* doc internal

* fix tests, KNN/rowids in

* define SQLITE_INDEX_CONSTRAINT_OFFSET

* whoops

* update tests, syrupy, use uv

* un ignore pyproject.toml

* dot

* tests/

* type error?

* win: .exe, update error name

* try fix macos python, paren around expr?

* win bash?

* dbg :(

* explicit error

* op

* dbg win

* win ./tests/.venv/Scripts/python.exe

* block UPDATEs on partition key values for now
2024-11-20 00:02:04 -08:00
Alex Garcia
763aad5d6a Remove vec_npy_each from default entrypoint and move to sqlite3_vec_numpy_init entrypoint 2024-09-25 23:07:17 -07:00
Alex Garcia
7ea402931e fmt and SQLITE_VEC_OMIT_FS fixes 2024-08-10 23:33:28 -07:00
Alex Garcia
2ed95aacc5 ensure UPDATEs and DELETEs work on vec0 tables with text primary keys, refs #77 2024-08-09 12:16:56 -07:00
Alex Garcia
e379c205c8 limit checks 2024-08-01 02:45:51 -07:00
Alex Garcia
a0bc9404ce static updates 2024-07-31 12:56:09 -07:00
Alex Garcia
0f5bc2f254 fmt 2024-07-23 23:57:42 -07:00
Alex Garcia
21d442903e test vec0 vacuums 2024-07-23 22:36:42 -07:00
Alex Garcia
633db6e9cc add l1 distance to vec0 tables 2024-07-23 14:04:17 -07:00
Alex Garcia
79491542e5 Merge branch 'main' of github.com:asg017/sqlite-vec into main 2024-07-23 12:27:37 -07:00
Daniel Levi-Minzi
25b85afc89
l1 distance (#39)
* initial work on l1

* l1 int8 neon implementation

* tweak l1 int8 and add test

* broken overflow still

* some progress on l1

* change to i32 instead of i64

* remove comment

* ignore poetry stuff

* unrolled l1 int8 and format

* remove comments
2024-07-23 09:04:15 -07:00
Alex Garcia
7fc8248f28 ensure statements opened by vec0 are finalize before commits. 2024-07-23 08:59:34 -07:00
Alex Garcia
ff6cf96e2a vec_type(), API references 2024-07-22 21:24:44 -07:00
Alex Garcia
f217cbf2bd knn cleanups and tests 2024-07-05 12:07:45 -07:00
Alex Garcia
3442131613 test_vec_quantize_binary small test for windows 2024-06-28 16:27:41 -07:00
Alex Garcia
37b4c2e9dc is it q?? trying to fix windows i64 stuff 2024-06-28 16:06:04 -07:00
Alex Garcia
2eafd843d7 no inline, windows i64 fix? 2024-06-28 16:00:58 -07:00
Alex Garcia
a5525c9a5d vec0 point and knn error handling 2024-06-28 15:29:13 -07:00
Alex Garcia
2fdd760dd1 fmt 2024-06-28 10:51:59 -07:00
Alex Garcia
b923c596df a ton more error handing, vec0 insert/delete/update, npy fixes 2024-06-28 10:51:49 -07:00
Alex Garcia
9dc772e9f9 format, pragma_table_list -> sqlite_master 2024-06-25 08:54:51 -07:00
Alex Garcia
feea3bfe43 remove vec_expo, impl drop vec0 2024-06-25 08:52:48 -07:00
Alex Garcia
d5363ccde7 re-add threadsafe, tests for drop column support 2024-06-24 23:34:40 -07:00
Alex Garcia
3a8ab9b489 several vec0 and np fixes 2024-06-24 23:26:11 -07:00
Alex Garcia
4196f16b27 error handling on vec0 INSERT operations, introduce evidence-of 2024-06-23 17:46:42 -07:00
Alex Garcia
b62f6f19a8 doc updates 2024-06-22 16:46:33 -07:00
Daniel Levi-Minzi
e9bf355a70
int8 vec from text (#17)
* parse int8 vec from json format text

* add small test cases for text based loading
2024-06-08 11:50:10 -07:00
Alex Garcia
29507aa45d fmt + better typedef common types 2024-04-20 17:02:19 -07:00
Alex Garcia
64745ab4a2 gha: more fixes 2024-04-20 16:09:19 -07:00
Alex Garcia
0f08b94dbd gha: windows dont typedef, fix linux eqp 2024-04-20 16:06:49 -07:00
Alex Garcia
df4d629891 gha: more linxu + windows fix attempts 2024-04-20 16:04:08 -07:00
Alex Garcia
1b3731b2bb gha: trying to fix linux + windows testing 2024-04-20 15:58:44 -07:00
Alex Garcia
4c8ad629e0 Initial commit 2024-04-20 13:38:58 -07:00