webclaw/crates/noxa-rag
Jacob Magar 20e880eea5 feat(noxa-68r.2,noxa-68r.3,noxa-68r.4): chunker, TEI provider, Qdrant store
noxa-68r.2 — chunker.rs:
- MarkdownSplitter with ChunkConfig::new(lower..upper).with_sizer(tokenizer)
- chunk_char_indices() API (not chunks_with_offsets — doesn't exist in text-splitter 0.29)
- Manual sliding-window overlap via overlap_prefix()
- min_words filter + max_chunks_per_page cap
- plain_text fallback, empty content -> Vec::new()

noxa-68r.3 — embed/tei.rs:
- TeiProvider::new() (hardcoded 1024 dims) + new_with_probe() (dynamic)
- Batch size 96 (RTX 4070 tuned); halve to 48 on 413
- Exponential backoff on 429/503 (100/200/400ms, max 3 retries)
- EmbedRequest always sends inputs as array for consistent Vec<Vec<f32>> response

noxa-68r.4 — store/qdrant.rs:
- QdrantStore with REST via reqwest feature (gRPC port 6334)
- create_collection: Cosine/HNSW m=16 ef_construct=200, payload indexes
- upsert() batched 256 with .wait(true)
- delete_by_url() with URL normalization
- search() extracts payload from ScoredPoint
2026-04-12 07:17:57 -04:00
..
src feat(noxa-68r.2,noxa-68r.3,noxa-68r.4): chunker, TEI provider, Qdrant store 2026-04-12 07:17:57 -04:00
Cargo.toml feat(noxa-68r.1): crate scaffold, traits, and core types 2026-04-12 07:13:19 -04:00