apunkt/sqlite-vec
1
0
Fork 0
mirror of https://github.com/asg017/sqlite-vec.git synced 2026-04-25 00:36:56 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Add DiskANN index for vec0 virtual table

Browse source 
Add DiskANN graph-based index: builds a Vamana graph with configurable R
(max degree) and L (search list size, separate for insert/query), supports
int8 quantization with rescore, lazy reverse-edge replacement, pre-quantized
query optimization, and insert buffer reuse. Includes shadow table management,
delete support, KNN integration, compile flag (SQLITE_VEC_ENABLE_DISKANN),
release-demo workflow, fuzz targets, and tests. Fixes rescore int8
quantization bug.
This commit is contained in:
Alex Garcia 2026-03-29 19:46:53 -07:00
parent e2c38f387c
commit 575371d751
23 changed files with 6550 additions and 135 deletions
Download patch file Download diff file Expand all files Collapse all files

18
sqlite-vec-rescore.c
View file

@ -156,21 +156,11 @@ static void rescore_quantize_float_to_bit(const float *src, uint8_t *dst,
static void rescore_quantize_float_to_int8(const float *src, int8_t *dst,
size_t dimensions) {
float vmin = src[0], vmax = src[0];
for (size_t i = 1; i < dimensions; i++) {
if (src[i] < vmin) vmin = src[i];
if (src[i] > vmax) vmax = src[i];
}
float range = vmax - vmin;
if (range == 0.0f) {
memset(dst, 0, dimensions);
return;
}
float scale = 255.0f / range;
float step = 2.0f / 255.0f;
for (size_t i = 0; i < dimensions; i++) {
float v = (src[i] - vmin) * scale - 128.0f;
if (v < -128.0f) v = -128.0f;
if (v > 127.0f) v = 127.0f;
float v = (src[i] - (-1.0f)) / step - 128.0f;
if (!(v <= 127.0f)) v = 127.0f;
if (!(v >= -128.0f)) v = -128.0f;
dst[i] = (int8_t)v;
}
}