Rohan Verma
69bdcf5946
Merge pull request #1491 from AnishSarkar22/feat/unified-model-connections
...
feat: Fix model attribution for prefix-stripped token usage callbacks
2026-06-14 17:50:48 -07:00
Rohan Verma
ea23c46b3a
Merge pull request #1489 from CREDO23/features/documents-injestion-layered-cached
...
[Performance] Document ingestion : Layered caching and Incremental chunk re-indexing
2026-06-14 17:47:17 -07:00
Anish Sarkar
f2214b3db0
Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections
2026-06-15 02:15:00 +05:30
Anish Sarkar
792f6eba92
fix(model-selector): update chat model display logic and replace icon for manage models button
2026-06-14 21:59:06 +05:30
Anish Sarkar
e38c569e89
fix(DocumentsSidebar): simplify connector button label logic for clarity
2026-06-14 21:40:59 +05:30
Anish Sarkar
5937ce2292
feat(model-selector): implement chat model filtering and improve selection handling
2026-06-14 21:31:05 +05:30
CREDO23
32a6e54ce6
Merge remote-tracking branch 'upstream/dev' into features/documents-injestion-layered-cached
2026-06-14 11:30:33 +02:00
Anish Sarkar
381d17d9b3
feat(chat-ui): integrate chat model selection handling in Composer and related components
2026-06-14 12:40:49 +05:30
Anish Sarkar
d9a4f14f99
feat(token-tracking): enhance model metadata reconciliation by adding bare model name handling
2026-06-14 12:18:22 +05:30
Rohan Verma
7654e909cf
Merge pull request #1488 from AnishSarkar22/feat/unified-model-connections
...
feat: unify provider-based model connection management
2026-06-13 16:26:23 -07:00
Anish Sarkar
7926814070
refactor(model-connections): remove unused fields and update verification logic
2026-06-14 02:46:19 +05:30
Anish Sarkar
3ba92dca13
refactor(layout): update container class for documentation
2026-06-13 23:00:53 +05:30
Anish Sarkar
9eefcd23b7
chore: ran linting
2026-06-13 22:18:19 +05:30
Anish Sarkar
1c72f4174b
refactor(model-connections): update placeholder text for API key fields
2026-06-13 22:17:48 +05:30
Anish Sarkar
c7409c8995
chore: ran linting
2026-06-13 21:59:35 +05:30
Anish Sarkar
ceace003aa
feat(local-models): add documentation for connecting local model providers
2026-06-13 21:52:45 +05:30
Anish Sarkar
97f004e7e1
refactor(onboarding, model-connections): enhance onboarding logic and streamline model connection handling by integrating chat model checks and improving state management
2026-06-13 20:48:24 +05:30
Anish Sarkar
ab5423d2d2
Merge remote-tracking branch 'upstream/dev' into feat/unified-model-connections
2026-06-13 19:04:49 +05:30
Anish Sarkar
02b5e758ed
refactor(search-space-settings, model-selector): replace CircleUser icon with Settings in search space settings and comment out max_input_tokens display in model selector
2026-06-13 18:44:06 +05:30
Anish Sarkar
212c8af682
refactor(model-selector, connection-settings): streamline model name handling and enhance connection settings dialog with improved state management for enabled models
2026-06-13 18:18:03 +05:30
Anish Sarkar
01cb4f281e
refactor(free-chat): remove FreeModelSelector from FreeChatPage and integrate it into FreeComposer for improved layout consistency
2026-06-13 16:31:03 +05:30
Anish Sarkar
76843f42f1
refactor(anonymous-models): remove description field from anonymous model responses and update related UI components
2026-06-13 16:30:26 +05:30
Anish Sarkar
576c56628a
chore(config): update global LLM configuration example with improved setup instructions, parameter naming, and enhanced comments for clarity
2026-06-13 14:57:14 +05:30
Anish Sarkar
e104193ddf
refactor(provider-configuration): standardize provider parameter naming across various modules and improve quota error handling in tests
2026-06-13 14:23:32 +05:30
Anish Sarkar
4a6a282a46
feat(runtime-cooldown): implement Redis-based shared cooldown management for model selection
2026-06-13 13:53:01 +05:30
Anish Sarkar
6d7732436d
feat(model-selector, model-connections-settings): integrate AUTO provider icon and streamline auto mode option rendering
2026-06-13 13:21:21 +05:30
Anish Sarkar
bd4a04f2e7
feat(database-migrations): add migration to remove legacy model config tables and remove stale model connection code
2026-06-13 12:45:43 +05:30
Anish Sarkar
50668775f8
feat(model-selector): enhance model selection with connection scope and free model indication
2026-06-13 12:14:17 +05:30
Anish Sarkar
02070201fb
feat(model-connections): add hint support for API Base URL field and improve dialog accessibility
2026-06-13 09:29:38 +05:30
Anish Sarkar
7493ba9324
refactor(icons): replace Clock3 icon with AlarmClock for automations
2026-06-13 03:34:13 +05:30
Anish Sarkar
78dbdc454a
feat(composer): update ComposerAction to include searchSpaceId and integrate ChatHeader component
2026-06-13 03:28:54 +05:30
Anish Sarkar
8fe9c21e76
feat(token-tracking): add model metadata registration and enhance token usage tracking
2026-06-13 03:08:35 +05:30
Anish Sarkar
5e86885a03
feat(model-connections): integrate model provider connections panel and connection card components
2026-06-13 02:40:22 +05:30
Anish Sarkar
15d9983669
feat(model-connections): enhance model selection facts and auto pinning logic
2026-06-13 02:19:27 +05:30
Anish Sarkar
45d27ba879
feat(model-connections): enhance auto mode with auto pinning
2026-06-13 01:39:26 +05:30
Anish Sarkar
7a1bb2acd6
feat(model-connections): refactor model roles UI
2026-06-13 00:46:53 +05:30
Anish Sarkar
e77b0c5d4e
feat(icons): add Azure, Claude, and LM Studio icons; update Bedrock icon and provider metadata
2026-06-13 00:25:53 +05:30
Anish Sarkar
9f6210ad08
feat(model-connections): add test preview functionality for model connections
2026-06-13 00:12:04 +05:30
CREDO23
dcebfc4756
Merge remote-tracking branch 'upstream/dev' into features/documents-injestion-layered-cached
2026-06-12 19:35:34 +02:00
Anish Sarkar
55f004e1da
feat(model-connections): improve model discovery error handling and enhance UI components
2026-06-12 22:50:50 +05:30
Anish Sarkar
407f2a9612
feat(model-connections): enhance model connection functionality with preview and selection features
2026-06-12 22:41:21 +05:30
CREDO23
311570b4f0
test(indexing): cover the edit path and make integration caches hermetic
...
Real-DB tests assert unchanged chunk rows survive edits, only new text is
embedded, removed rows are deleted with positions compacted, and the kill
switch restores full-replace. An autouse fixture disables the ETL/embedding
caches so a developer's .env can't leak cache hits into unrelated tests.
2026-06-12 18:53:21 +02:00
CREDO23
052e9ef4d1
refactor(chunks): order chunk reads by (document_id, position)
...
Presentation and citation ordering moves off Chunk.id/created_at to the
explicit position column (id kept as tiebreaker). Vector and ts_rank
ranking order_by clauses are untouched.
2026-06-12 18:53:21 +02:00
CREDO23
5a71769dba
fix(chunks): set position on remaining chunk insert paths
...
document_converters, the github size-fallback chunker, revert_service
restores, and the kb-persistence middleware now write explicit positions
(the middleware read path also orders by position).
2026-06-12 18:53:08 +02:00
CREDO23
7d55aaf2c1
feat(indexing): reconcile chunks incrementally on re-index
...
index() now loads existing rows and applies a content diff instead of
delete-all/reinsert-all: unchanged chunks keep their rows and embeddings
(zero HNSW/GIN churn), moved chunks get a position-only UPDATE, and only
new texts are embedded, batched with the summary embedding. First index
keeps the cache-aware build_chunk_embeddings path.
2026-06-12 18:53:08 +02:00
CREDO23
fd495e1b2f
feat(observability): add chunk reconcile metric and kill-switch flag
...
surfsense.indexing.reconcile.chunks counts reused/embedded/deleted chunks per
re-index. CHUNK_RECONCILE_ENABLED (default on) falls back to delete-all +
full re-embed if the diff path ever misbehaves.
2026-06-12 18:52:57 +02:00
CREDO23
8d413ea5c2
refactor(indexing): expose chunk_markdown and embed_batch helpers
...
Split _compute so the incremental edit path can reuse the exact same chunker
selection and embedding entry points (and their test patch targets) without
going through the doc-level cache.
2026-06-12 18:52:57 +02:00
CREDO23
f82dedf712
feat(indexing): add pure chunk reconciler for content-addressed diffs
...
Greedy multiset match on chunk text decides which rows keep their embeddings,
which texts need embedding, and which rows are deleted. No DB, no embeddings;
fully unit-tested (reuse, head insert, middle edit, deletion, duplicates,
reorder, full rewrite).
2026-06-12 18:52:46 +02:00
CREDO23
c6e71c851c
feat(chunks): add explicit position column with backfill migration
...
Chunk ids stop reflecting document order once incremental re-indexing keeps
unchanged rows across edits. Backfill preserves the historical id ordering
so behavior is identical on day one.
2026-06-12 18:52:45 +02:00
Anish Sarkar
356f0e56c5
feat(model-connections): add provider-specific connection forms and shared components
2026-06-12 22:04:44 +05:30
