Phase 1 (rename DB) commit 1g: rename the memberships table (no inbound
FKs reference it) and update the matching raw-SQL cleanup in the obsidian
plugin test. Relationship attribute names stay until Phase 2.
Phase 1 (rename DB) commit 1f: rename the invites table and flip the one
inbound FK (memberships.invited_by_invite_id). The invites row's own
role_id -> search_space_roles FK is left for the roles rename commit.
Phase 1 (rename DB) commit 1d: attribute->physical column shim on the
notifications satellite model, plus the inbox-list index column-ref and
its name aligned to workspace (ix_notifications_user_workspace_created).
Phase 1 (rename DB) commit 1b: same attribute->physical column shim as
1a, applied to the document_files satellite model. ORM attribute name
unchanged; FK target string left for the searchspaces rename commit.
Phase 1 (rename DB) commit 1a: shim every search_space_id /
owner_search_space_id column to the renamed physical column
(workspace_id / owner_workspace_id) via Column("workspace_id", ...),
keeping the ORM attribute name unchanged so existing callers are
untouched. Flip all Table-level references that resolve by column key
(inner __table_args__ strings, the ck_connections_scope_owner CHECK
text, and the _INDEX_DEFINITIONS runtime DDL) plus the searchspace-named
constraint/index names to workspace_id. Update the three raw-SQL test
fixtures that referenced the physical column.
Note: SQLAlchemy defaults a column's key to its name, so passing an
explicit "workspace_id" name moves the Table.c key to workspace_id;
Table-level string refs must therefore change in this phase (the
opposite of the original plan's finding 1). The ORM attribute, the
SearchSpace class, relationships, table names, and FK target strings are
intentionally left for later commits/Phase 2.
Verified: unit 2375 passed/1 skip, integration 346 passed (baseline
parity); create_all builds every table with workspace_id.
Migration 168 used bare add_column/drop_column, so `alembic upgrade head`
from an empty DB failed at 168 ("column revoked_at already exists"): the
0_initial_schema migration bootstraps via Base.metadata.create_all (the live
ORM shape), which already includes the hardened refresh_tokens columns, and
168's unguarded ALTERs then collided.
Guard every step (ADD COLUMN IF NOT EXISTS, DROP COLUMN IF EXISTS, existence-
checked backfill) so 168 no-ops on already-hardened (create_all) databases
while still fully transforming legacy is_revoked-shape databases. Matches the
guarded style already used by migrations 92/149/169. Net schema effect is
unchanged. Restores a clean from-scratch `alembic upgrade head` to head.
- Added logging for JSON parsing failures in structured output to aid in debugging.
- Updated the TranscriptTurn model to ignore unknown keys, improving flexibility for future model variations.
- Updated main-agent middleware to clarify that both filesystem reads/writes and knowledge-base retrieval are handled by the `knowledge_base` subagent.
- Introduced `_forward_mention_pins` function to carry `@`-mention pins into subagent state.
- Revised system prompts to reflect the new retrieval method and ensure proper citation handling.
- Removed the `search_knowledge_base` tool and its related tests, consolidating functionality under the `task` tool.
- Enhanced documentation to guide usage of the new retrieval approach and citation practices.
The document saving logic used a hardcoded character slice ([:4000]) for
the document summary content fed to the embedder. For UTF-8 documents
(e.g., Arabic, Chinese, Japanese), characters above U+007F take multiple
bytes but count as 1 character in a slice, potentially producing text
that exceeds the token limit of the embedding model.
Replaced the arbitrary slice with runcate_for_embedding(), which safely
bounds the text using the embedding model's actual tokenizer.
When a codeBlock is nested inside an indented structure like a
bulletListItem, the \_render_block\ function prepends the block's
indentation \prefix\ to every line.
However, for codeBlock elements, only the fence markers (the opening
and closing \\\) should carry the block indentation. Interior code
lines must not have the prefix prepended, because markdown parsers
treat leading spaces inside a code fence as part of the code content.
This fix removes the prefix from interior code lines to prevent
code snippets stored in notes from gaining spurious whitespace.
When the Unstructured API returns a Table element without text_as_html
in its metadata (e.g. local install or free-tier API), the lambda was
raising KeyError: 'text_as_html', crashing the entire document
indexing pipeline for any file containing tables.
Guard the key access with .get() and fall back to the plain extracted
text content (x) so the pipeline continues and the table content is
still indexed, just without HTML formatting.
The RAG/citation/context redesign in ADR 0001 is implemented and validated
(KB + web on the unified [n] citation spine, pull-based retrieval, eager path
retired). Drop the ADR and the one stale docstring reference to it.
The pull-based KB design (on-demand search_knowledge_base tool + pre-injected
workspace tree) fully replaced the old eager retrieval path. Remove its last
remnants:
- Delete KnowledgePriorityMiddleware (knowledge_search.py) and its tests.
- Drop the kb_priority state field + reducer default; trim
KbContextProjectionMiddleware to project only workspace_tree_text.
- Remove the now-dead feature flags enable_kb_priority_preinjection and
enable_kb_planner_runnable across backend (flags, route schema, tests,
env examples) and frontend (settings toggle, zod schema).
- Scrub <priority_documents> and stale KnowledgePriorityMiddleware references
from prompts, docstrings, and the ADR.
No functional change: nothing wrote kb_priority and neither flag gated live
behavior after the cutover. Full backend suite green (pre-existing unrelated
failures aside).
The legacy system_prompt_composer fragments and its default_system_instructions
wrapper were no longer referenced by any live code path (the main-agent prompt
builder owns composition now). Delete the whole orphaned tree and its test.
The model frequently writes citations glued to the preceding word
(docs[17]); the (?<!\w) lookbehind (added to dodge arr[1] array indexing)
silently skipped these, leaving raw [n] that fails to render and reads
like array access. Drop the lookbehind so glued citations resolve; genuine
code/array syntax stays protected by the existing code-region carve-out and
unresolved ordinals still drop harmlessly.
Rewrite the main-agent citation contract to a single [n] channel and sync
the orphaned system_prompt_composer surface to match; drop stale
[citation:chunk_id] / <chunk_index> references from dynamic_context and
provider hints. Reuse the shared hybrid search in the deliverables report
(citations omitted for now) and delete the orphaned report KB helper.
Remove the dead eager KnowledgePriorityMiddleware wiring (knowledge_priority
+ stack) and its legacy browse test. Update ADR 0001 to reflect the cutover.
web_search now registers each result as a WEB_RESULT (locator {url}) and
renders a <web_results> block of <document view="excerpt"> [n] passages,
returning Command(update={messages, citation_registry}) like
search_knowledge_base. Collapse the duplicate research-subagent web_search
into the shared tool and teach the prompts to cite web hits with [n].
The main agent's search_knowledge_base tool runs the hybrid spine, renders
a <retrieved_context> of numbered [n] passages, and persists the registry.
KB subagent prompts teach citing [n] from <document view="full"> reads
(evidence.chunk_ids -> evidence.citations). Delete the now-unused
search->read highlighting hand-off: the kb_matched_chunk_ids state field,
its reducer default, the tool's _matched_chunk_ids writer, and the dead
KnowledgePriorityMiddleware writes.
Add the checkpointed CitationRegistry (load/merge helpers + state field)
and a lightweight CitationStateMiddleware so subagents can register into
the same conversation registry. Resolve [n] -> [citation:<payload>] at
stream finalize from the registry, polymorphically by source type.
Add a shared document_render package that renders sources as
<document view="excerpt|full"> blocks with server-assigned [n] passage
labels (KB locator {document_id, chunk_id}, web locator {url}). Wire the
KB read backend (kb_postgres) and read_file to the new renderer and drop
the legacy per-document XML renderer (document_xml, retrieved_context) and
the old chunk_index / matched="true" / <chunk id> read format.
