Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
# Auto-detect clang with libFuzzer support.
|
|
|
|
|
# Priority: Homebrew LLVM (macOS ARM) → Homebrew LLVM (macOS Intel) →
|
|
|
|
|
# versioned clang (Linux) → system clang
|
|
|
|
|
FUZZ_CC ?= $(shell \
|
|
|
|
|
if [ -x /opt/homebrew/opt/llvm/bin/clang ]; then \
|
|
|
|
|
echo "/opt/homebrew/opt/llvm/bin/clang"; \
|
|
|
|
|
elif [ -x /usr/local/opt/llvm/bin/clang ]; then \
|
|
|
|
|
echo "/usr/local/opt/llvm/bin/clang"; \
|
|
|
|
|
elif command -v clang-18 >/dev/null 2>&1; then \
|
|
|
|
|
echo "clang-18"; \
|
|
|
|
|
elif command -v clang-17 >/dev/null 2>&1; then \
|
|
|
|
|
echo "clang-17"; \
|
|
|
|
|
elif command -v clang >/dev/null 2>&1; then \
|
|
|
|
|
echo "clang"; \
|
|
|
|
|
else \
|
|
|
|
|
echo "FUZZ_CC_NOT_FOUND"; \
|
|
|
|
|
fi)
|
2024-07-25 11:16:06 -07:00
|
|
|
|
Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
# AddressSanitizer + UndefinedBehaviorSanitizer + libFuzzer.
|
|
|
|
|
# Override FUZZ_SANITIZERS to change (e.g., drop ubsan on Windows).
|
|
|
|
|
FUZZ_SANITIZERS ?= -fsanitize=address,undefined,fuzzer
|
|
|
|
|
|
|
|
|
|
# On macOS, Homebrew LLVM may need -Wl,-ld_classic to work with the system linker.
|
|
|
|
|
FUZZ_LDFLAGS ?= $(shell \
|
|
|
|
|
if [ "$$(uname -s)" = "Darwin" ]; then \
|
|
|
|
|
echo "-Wl,-ld_classic"; \
|
|
|
|
|
fi)
|
|
|
|
|
|
|
|
|
|
FUZZ_CFLAGS = $(FUZZ_SANITIZERS) -I ../../ -I ../../vendor -DSQLITE_CORE -g $(FUZZ_LDFLAGS)
|
|
|
|
|
FUZZ_SRCS = ../../vendor/sqlite3.c ../../sqlite-vec.c
|
|
|
|
|
|
|
|
|
|
TARGET_DIR = ./targets
|
2024-07-25 11:16:06 -07:00
|
|
|
|
|
|
|
|
$(TARGET_DIR):
|
|
|
|
|
mkdir -p $@
|
|
|
|
|
|
Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
# Existing targets (filename uses -, Makefile target uses _)
|
|
|
|
|
$(TARGET_DIR)/vec0_create: vec0-create.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/exec: exec.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/json: json.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/numpy: numpy.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
# New targets
|
|
|
|
|
$(TARGET_DIR)/shadow_corrupt: shadow-corrupt.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/vec0_operations: vec0-operations.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/scalar_functions: scalar-functions.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/vec0_create_full: vec0-create-full.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/metadata_columns: metadata-columns.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/vec_each: vec-each.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
2026-03-02 20:45:50 -08:00
|
|
|
$(TARGET_DIR)/vec_mismatch: vec-mismatch.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
Complete vec0 DELETE: zero data, reclaim empty chunks, fix metadata rc bug (#268)
When a row is deleted from a vec0 virtual table, the rowid slot in
_chunks.rowids and vector data in _vector_chunksNN.vectors are now
zeroed out (previously left as stale data, tracked in #54). When all
rows in a chunk are deleted (validity bitmap all zeros), the chunk and
its associated vector/metadata shadow table rows are reclaimed.
- Add vec0Update_Delete_ClearRowid to zero the rowid blob slot
- Add vec0Update_Delete_ClearVectors to zero all vector blob slots
- Add vec0Update_Delete_DeleteChunkIfEmpty to detect and delete
fully-empty chunks from _chunks, _vector_chunksNN, _metadatachunksNN
- Fix missing rc check in ClearMetadata loop (bug: errors were silently
ignored)
- Fix vec0_new_chunk to explicitly set _rowid_ on shadow table INSERTs
(SHADOW_TABLE_ROWID_QUIRK: "rowid PRIMARY KEY" without INTEGER type
is not a true rowid alias, causing blob_open failures after chunk
delete+recreate cycles)
- Add 13 new tests covering rowid/vector zeroing, chunk reclamation,
metadata/auxiliary/partition/text-PK/int8/bit variants, and
page_count shrinkage verification
- Add vec0-delete-completeness fuzz target
- Update snapshots for new delete zeroing behavior
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 00:02:36 -07:00
|
|
|
$(TARGET_DIR)/vec0_delete_completeness: vec0-delete-completeness.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
2026-03-29 19:45:54 -07:00
|
|
|
$(TARGET_DIR)/rescore_operations: rescore-operations.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_create: rescore-create.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_quantize: rescore-quantize.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE -DSQLITE_VEC_TEST $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_shadow_corrupt: rescore-shadow-corrupt.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_knn_deep: rescore-knn-deep.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_quantize_edge: rescore-quantize-edge.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE -DSQLITE_VEC_TEST $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/rescore_interleave: rescore-interleave.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) -DSQLITE_VEC_ENABLE_RESCORE $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
2026-03-29 19:46:23 -07:00
|
|
|
$(TARGET_DIR)/ivf_create: ivf-create.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_operations: ivf-operations.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_quantize: ivf-quantize.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_kmeans: ivf-kmeans.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_shadow_corrupt: ivf-shadow-corrupt.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_knn_deep: ivf-knn-deep.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_cell_overflow: ivf-cell-overflow.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
|
|
|
|
$(TARGET_DIR)/ivf_rescore: ivf-rescore.c $(FUZZ_SRCS) | $(TARGET_DIR)
|
|
|
|
|
$(FUZZ_CC) $(FUZZ_CFLAGS) $(FUZZ_SRCS) $< -o $@
|
|
|
|
|
|
Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
FUZZ_TARGETS = vec0_create exec json numpy \
|
|
|
|
|
shadow_corrupt vec0_operations scalar_functions \
|
Complete vec0 DELETE: zero data, reclaim empty chunks, fix metadata rc bug (#268)
When a row is deleted from a vec0 virtual table, the rowid slot in
_chunks.rowids and vector data in _vector_chunksNN.vectors are now
zeroed out (previously left as stale data, tracked in #54). When all
rows in a chunk are deleted (validity bitmap all zeros), the chunk and
its associated vector/metadata shadow table rows are reclaimed.
- Add vec0Update_Delete_ClearRowid to zero the rowid blob slot
- Add vec0Update_Delete_ClearVectors to zero all vector blob slots
- Add vec0Update_Delete_DeleteChunkIfEmpty to detect and delete
fully-empty chunks from _chunks, _vector_chunksNN, _metadatachunksNN
- Fix missing rc check in ClearMetadata loop (bug: errors were silently
ignored)
- Fix vec0_new_chunk to explicitly set _rowid_ on shadow table INSERTs
(SHADOW_TABLE_ROWID_QUIRK: "rowid PRIMARY KEY" without INTEGER type
is not a true rowid alias, causing blob_open failures after chunk
delete+recreate cycles)
- Add 13 new tests covering rowid/vector zeroing, chunk reclamation,
metadata/auxiliary/partition/text-PK/int8/bit variants, and
page_count shrinkage verification
- Add vec0-delete-completeness fuzz target
- Update snapshots for new delete zeroing behavior
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 00:02:36 -07:00
|
|
|
vec0_create_full metadata_columns vec_each vec_mismatch \
|
2026-03-29 19:45:54 -07:00
|
|
|
vec0_delete_completeness \
|
|
|
|
|
rescore_operations rescore_create rescore_quantize \
|
|
|
|
|
rescore_shadow_corrupt rescore_knn_deep \
|
2026-03-29 19:46:23 -07:00
|
|
|
rescore_quantize_edge rescore_interleave \
|
|
|
|
|
ivf_create ivf_operations \
|
|
|
|
|
ivf_quantize ivf_kmeans ivf_shadow_corrupt \
|
|
|
|
|
ivf_knn_deep ivf_cell_overflow ivf_rescore
|
Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
|
|
|
|
|
all: $(addprefix $(TARGET_DIR)/,$(FUZZ_TARGETS))
|
2024-07-25 11:16:06 -07:00
|
|
|
|
|
|
|
|
clean:
|
|
|
|
|
rm -rf $(TARGET_DIR)/*
|
Add comprehensive fuzz testing infrastructure with 6 new targets
- Fix numpy.c: tautology bug (|| → &&), infinite loop, and missing
sqlite3_vec_numpy_init call
- Replace tests/fuzz/Makefile: auto-detect clang, add UBSAN, macOS
ld_classic workaround, generic build rules for all 10 targets
- Add 6 new fuzz targets: shadow-corrupt (corrupted shadow tables),
vec0-operations (INSERT/DELETE/query sequences), scalar-functions
(all 18 SQL scalar functions), vec0-create-full (CREATE + lifecycle),
metadata-columns (metadata/auxiliary columns), vec-each (vec_each TVF)
- Add seed corpora for shadow-corrupt, vec0-operations, exec, and json
- Add fuzz-build/fuzz-quick/fuzz-long targets to root Makefile
All 10 targets verified building and running on macOS ARM (Apple Silicon).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:33:05 -08:00
|
|
|
|
|
|
|
|
.PHONY: all clean
|