mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-07-26 23:51:14 +02:00
2.2 KiB
2.2 KiB
Phase 5 — Migration
After Phases 1–4. One-time, per-workspace, behind the flag. Umbrella:
00-umbrella-plan.md.
Objective
Move each existing workspace's KB from Postgres-as-truth to git-as-truth by exporting current documents/folders into an initial git repo, then flipping KB_GIT_ENABLED for that workspace once search parity is verified.
Locked model
- Seed commit per workspace. Read current
folders+documents(source_markdown/content) and write the real tree into the Phase-1 repo as one seed commit (author=migration). - Preserve identity. Keep the
unique_identifier_hash↔ path mapping so connector re-syncs and existing references stay stable. - Verify before flip. After seeding, run
reindex(workspace)(Phase 4) and confirm the rebuilt chunk set reproduces pre-migration search behavior on a fixed query set. - Rollback window. Keep Postgres content intact until the flagged workspace is verified; flag flip is the point of no return per workspace.
Work items
app/kb_git/migrate.py—migrate_workspace(workspace_id): build the tree fromfolders/documentsusing the same path/filename rules as the live write path (path_resolverhelpers; git path = virtual path minus/documents; see00c-shared-contract.mdC1), writeDocument.source_markdown/contentas each file, preserveunique_identifier_hash, one seed commit.- Parity check: seed →
reindex→ compare chunk counts and top-k search results against a captured pre-migration baseline. - Per-workspace flag flip with a guard (refuse to flip if parity check fails).
- Dry-run mode (build repo, report diffs, do not flip).
Tests
- Round-trip: a seeded workspace's
reindexchunk set matches the pre-migration set for the same content. unique_identifier_hashmapping preserved (connector docs still resolve).- Dry-run flips nothing; failed parity check blocks the flip.
Out of scope
- Binary re-import (blobs stay in the blob store).
- Frontend cutover (separate umbrella).
Open questions
- Big-bang all-workspaces vs. staged per-workspace rollout order (recommend staged).
- Baseline capture method for the parity check (fixed query fixtures vs. sampled).