2026-06-01 22:00:28 -05:00
|
|
|
# TrustGraph Effect-Native Rewrite Opportunity Audit
|
|
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
This is the current backlog snapshot for the playbook in
|
|
|
|
|
`ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md`. The branch is `ts-port-effect-v4`.
|
|
|
|
|
The unrelated local file `.idea/effect.intellij.xml` must stay uncommitted.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
## Inputs
|
|
|
|
|
|
|
|
|
|
Verified source roots:
|
|
|
|
|
|
|
|
|
|
- TrustGraph TS port: `/home/elpresidank/YeeBois/dev/trustgraph/ts`
|
|
|
|
|
- Effect v4 subtree: `/home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4`
|
2026-06-02 00:22:04 -05:00
|
|
|
- Installed Effect beta used by this workspace: `ts/node_modules/effect`
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 02:15:22 -05:00
|
|
|
Current signal counts from `ts/packages` after the 2026-06-02 Client
|
|
|
|
|
WebSocket adapter slice:
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
| Signal | Count |
|
|
|
|
|
| --- | ---: |
|
2026-06-02 02:09:45 -05:00
|
|
|
| `Effect.runPromise` | 203 |
|
2026-06-02 01:59:51 -05:00
|
|
|
| `Map<` | 88 |
|
2026-06-02 02:15:22 -05:00
|
|
|
| `WebSocket` | 72 |
|
2026-06-02 01:59:51 -05:00
|
|
|
| `new Map` | 62 |
|
2026-06-02 00:54:47 -05:00
|
|
|
| `toPromiseRequestor` | 0 |
|
2026-06-01 22:00:28 -05:00
|
|
|
| `makeAsyncProcessor` | 19 |
|
2026-06-02 00:22:04 -05:00
|
|
|
| `receive(` | 18 |
|
2026-06-02 01:47:15 -05:00
|
|
|
| `while (` | 9 |
|
2026-06-02 02:15:22 -05:00
|
|
|
| `new Error` | 12 |
|
2026-06-02 00:22:04 -05:00
|
|
|
| `new Promise` | 10 |
|
2026-06-02 01:07:12 -05:00
|
|
|
| `JSON.parse` | 7 |
|
2026-06-02 01:59:51 -05:00
|
|
|
| `localStorage` | 8 |
|
2026-06-02 00:22:04 -05:00
|
|
|
| `JSON.stringify` | 6 |
|
|
|
|
|
| `setTimeout` | 4 |
|
|
|
|
|
| `process.env` | 3 |
|
|
|
|
|
|
|
|
|
|
Notes:
|
|
|
|
|
|
|
|
|
|
- The remaining `process.env` hits are in `packages/workbench/playwright.config.ts`.
|
|
|
|
|
- In production `packages/base`, `packages/cli`, and `packages/mcp` sources,
|
|
|
|
|
the strict scans for `new Error`, `new Promise`, `setTimeout`,
|
|
|
|
|
`JSON.parse`, `JSON.stringify`, and direct `process.env` reads are clean.
|
|
|
|
|
- `Effect.runPromise` is expected at external Promise compatibility
|
|
|
|
|
boundaries, but each match should still be audited for avoidable internal
|
|
|
|
|
runtime ownership.
|
2026-06-02 01:59:51 -05:00
|
|
|
- The `Map<` and `new Map` counts increased in this snapshot because the
|
|
|
|
|
Librarian slice introduced explicit ref-backed state types and clone helpers
|
|
|
|
|
while removing the service object's direct mutable maps/handles.
|
2026-06-02 02:09:45 -05:00
|
|
|
- The `Effect.runPromise` and `WebSocket` counts dropped in this snapshot
|
|
|
|
|
because `EffectRpcClient` now owns its RPC/socket layer with
|
|
|
|
|
`ManagedRuntime` and uses Effect's WebSocket constructor layer.
|
2026-06-02 02:15:22 -05:00
|
|
|
- The raw `WebSocket` count increased in this snapshot because the adapter
|
|
|
|
|
slice added focused tests and typed adapter names; production
|
|
|
|
|
`websocket-adapter.ts` is now clean of `try`/`catch`, normal `Error`, and
|
|
|
|
|
the previous constructor assertions.
|
|
|
|
|
- The `new Error` count dropped because `websocket-adapter.ts` now throws
|
|
|
|
|
`S.TaggedErrorClass` adapter errors.
|
2026-06-02 01:47:15 -05:00
|
|
|
- `Record<string, any>` and `throwLibrarianServiceError` are now clean in
|
|
|
|
|
`ts/packages`.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-01 22:11:03 -05:00
|
|
|
## Loop Passes
|
|
|
|
|
|
|
|
|
|
### 2026-06-02: Base Request/Response Facade
|
|
|
|
|
|
|
|
|
|
- Status: migrated and verified.
|
|
|
|
|
- Completed:
|
2026-06-02 00:22:04 -05:00
|
|
|
- Request/response startup now owns a scoped Effect runtime handle and maps
|
|
|
|
|
failures to TrustGraph tagged messaging errors.
|
|
|
|
|
- Runtime shutdown is idempotent and uses scoped fibers.
|
|
|
|
|
- Tests cover Promise compatibility, tagged timeout errors, and tagged
|
|
|
|
|
lifecycle errors.
|
2026-06-01 22:11:03 -05:00
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/base test`
|
|
|
|
|
- `bun run --cwd ts/packages/base build`
|
|
|
|
|
- `bun run --cwd ts check:tsgo`
|
|
|
|
|
- `bun run --cwd ts build`
|
|
|
|
|
- `bun run --cwd ts test`
|
2026-06-01 22:17:50 -05:00
|
|
|
|
|
|
|
|
### 2026-06-02: Gateway Dispatcher Requestor Cache
|
|
|
|
|
|
|
|
|
|
- Status: migrated and package-verified.
|
|
|
|
|
- Completed:
|
2026-06-02 00:22:04 -05:00
|
|
|
- Gateway dispatcher caches scoped `EffectRequestResponse` handles instead
|
|
|
|
|
of `Promise<RequestResponse>` values.
|
|
|
|
|
- Lazy requestor creation is serialized with `SynchronizedRef.modifyEffect`.
|
|
|
|
|
- Streaming final-marker detection is centralized.
|
|
|
|
|
- Dispatcher cleanup uses Effect scope/error handling instead of manual
|
|
|
|
|
`try`/`catch`.
|
2026-06-01 22:17:50 -05:00
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts check:tsgo`
|
2026-06-01 22:11:03 -05:00
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
### 2026-06-02: Strict Base, CLI, MCP, And tsgo Slice
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 00:40:44 -05:00
|
|
|
- Status: migrated, root-verified, committed, and pushed.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Completed:
|
|
|
|
|
- Base messaging, NATS backend, producer, consumer, subscriber,
|
|
|
|
|
request/response, runtime factories, processor programs, flow specs, and
|
|
|
|
|
LLM service now use Effect-native boundaries, schema codecs, scoped
|
|
|
|
|
cleanup, and `S.TaggedErrorClass.make(...)` errors.
|
|
|
|
|
- CLI commands now run Effect programs at the command boundary, wrap socket
|
|
|
|
|
lifecycle with `Effect.acquireUseRelease`, encode JSON through Effect
|
|
|
|
|
Schema, and write output without `console.log`.
|
|
|
|
|
- MCP Effect server now loads env/config through `Config`, wraps gateway
|
|
|
|
|
calls with `Effect.tryPromise`, constructs schema classes with `.make`, and
|
|
|
|
|
uses tagged errors.
|
|
|
|
|
- MCP stdio compatibility server keeps `createMcpServer` and `run`, but uses
|
|
|
|
|
Effect callbacks/tryPromise/schema encoding internally. `run()` uses
|
|
|
|
|
`ManagedRuntime`; `runMain()` uses `NodeRuntime.runMain`.
|
|
|
|
|
- Flow stateful service launch sites now pass an explicit `Context.Context`
|
|
|
|
|
into the base processor runtime instead of hiding requirements behind
|
|
|
|
|
assertions.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 00:40:44 -05:00
|
|
|
### 2026-06-02: ConfigService Ref-Backed State Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/config/service.ts` now models runtime state as a
|
|
|
|
|
`SynchronizedRef<ConfigServiceState>` instead of adding mutable
|
|
|
|
|
`store`, `version`, consumer, and producer fields onto the processor
|
|
|
|
|
object.
|
|
|
|
|
- Config operations have Effect-returning handlers with Promise facades only
|
|
|
|
|
on the exported compatibility methods.
|
|
|
|
|
- Request narrowing now uses `effect/Predicate` rather than request-record
|
|
|
|
|
type assertions.
|
|
|
|
|
- Persistence remains schema-backed and now reads/writes snapshots from the
|
|
|
|
|
ref-backed state.
|
|
|
|
|
- The consume loop now uses `Effect.whileLoop`; the remaining
|
|
|
|
|
`consumer.receive(2000)` call is a pubsub boundary for this service.
|
|
|
|
|
- Service startup now exposes `runMain()` through `NodeRuntime.runMain`.
|
|
|
|
|
The legacy `run()` Promise facade uses `ManagedRuntime`, and
|
|
|
|
|
`ts/scripts/run-config.ts` delegates directly to `runMain()` instead of
|
|
|
|
|
owning its own catch/process-exit wrapper.
|
|
|
|
|
- Config-service tests cover tagged invalid mutation errors, workspace
|
|
|
|
|
persistence, legacy load, concurrent ref-backed mutations, and push
|
|
|
|
|
publishing from the stored producer handle.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 00:54:47 -05:00
|
|
|
### 2026-06-02: RAG And Agent Requestor Bridge Slice
|
|
|
|
|
|
2026-06-02 01:07:12 -05:00
|
|
|
- Status: migrated, root-verified, committed, and pushed.
|
2026-06-02 00:54:47 -05:00
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/retrieval/graph-rag.ts` and
|
|
|
|
|
`ts/packages/flow/src/retrieval/document-rag.ts` now accept
|
|
|
|
|
`EffectRequestResponse` clients directly. The engines no longer adapt
|
|
|
|
|
Effect requestors back to Promise requestors and then wrap those calls in
|
|
|
|
|
`Effect.tryPromise`.
|
|
|
|
|
- `ts/packages/flow/src/retrieval/graph-rag-service.ts` and
|
|
|
|
|
`ts/packages/flow/src/retrieval/document-rag-service.ts` now pass native
|
|
|
|
|
flow requestors directly into the engines.
|
|
|
|
|
- `ts/packages/flow/src/agent/react/tools.ts` now accepts
|
|
|
|
|
`EffectRequestResponse` clients directly for graph RAG, document RAG,
|
|
|
|
|
triples, and MCP tool calls. Tool input narrowing uses Schema and
|
|
|
|
|
`effect/Predicate` rather than local request/response type assertions.
|
|
|
|
|
- `ts/packages/flow/src/agent/react/service.ts` wires default and configured
|
|
|
|
|
tools with native Effect requestors instead of `toPromiseRequestor`.
|
|
|
|
|
- Graph RAG, document RAG, and agent service startup now expose `runMain()`
|
|
|
|
|
through `NodeRuntime.runMain`; their legacy `run()` Promise facades use
|
|
|
|
|
`ManagedRuntime`.
|
|
|
|
|
- `ts/scripts/run-graph-rag.ts`, `ts/scripts/run-document-rag.ts`, and
|
|
|
|
|
`ts/scripts/run-agent.ts` now delegate to `runMain()`.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
|
2026-06-02 01:07:12 -05:00
|
|
|
### 2026-06-02: KnowledgeCore Ref-Backed State Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/cores/service.ts` now exposes a typed
|
|
|
|
|
`KnowledgeCoreService` instead of `AsyncProcessorRuntime & Record<string,
|
|
|
|
|
any>`.
|
|
|
|
|
- Runtime state now lives in
|
|
|
|
|
`SynchronizedRef<KnowledgeCoreServiceState>` with `kgCores`, `deCores`,
|
|
|
|
|
the request consumer, and response producer.
|
|
|
|
|
- Knowledge operations now have Effect-returning handlers with Promise
|
|
|
|
|
facades only on exported compatibility methods.
|
|
|
|
|
- Persistence now decodes legacy and current snapshot shapes with Effect
|
|
|
|
|
Schema and encodes JSON through Schema rather than raw
|
|
|
|
|
`JSON.parse`/`JSON.stringify` plus assertions.
|
|
|
|
|
- The consume loop now uses `Effect.whileLoop`; the remaining
|
|
|
|
|
`consumer.receive(2000)` call is a pubsub boundary for this service.
|
|
|
|
|
- The service exposes `runMain()` through `NodeRuntime.runMain`; legacy
|
|
|
|
|
`run()` uses `ManagedRuntime`, and `ts/scripts/run-knowledge.ts` delegates
|
|
|
|
|
to `runMain()`.
|
|
|
|
|
- `ts/packages/base/src/schema/messages.ts` now models legacy hyphenated
|
|
|
|
|
knowledge request/response aliases so the service can preserve the wire
|
|
|
|
|
shape without response type assertions.
|
|
|
|
|
- New knowledge-core tests cover ref-backed mutation, graph embedding alias
|
|
|
|
|
responses, concurrent state updates, and legacy persistence loading.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/base build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
|
2026-06-02 01:13:40 -05:00
|
|
|
### 2026-06-02: Flow Manager And Librarian Runtime Normalization
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/flow-manager/service.ts` and
|
|
|
|
|
`ts/packages/flow/src/librarian/service.ts` now expose `runMain()` through
|
|
|
|
|
`NodeRuntime.runMain`.
|
|
|
|
|
- Their legacy `run()` Promise facades now use `ManagedRuntime` instead of
|
|
|
|
|
directly owning `Effect.runPromise`.
|
|
|
|
|
- `ts/scripts/run-flow-manager.ts` and `ts/scripts/run-librarian.ts` now
|
|
|
|
|
delegate to `runMain()` instead of wrapping startup with local
|
|
|
|
|
`.catch(console.error/process.exit)` handlers.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 01:26:40 -05:00
|
|
|
### 2026-06-02: FlowManager Ref-Backed State Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/flow-manager/service.ts` now exposes a typed
|
|
|
|
|
`FlowManagerService` instead of `AsyncProcessorRuntime & Record<string,
|
|
|
|
|
any>`.
|
|
|
|
|
- Runtime state now lives in
|
|
|
|
|
`SynchronizedRef<FlowManagerServiceState>` with `flows`, `blueprints`, the
|
|
|
|
|
request consumer, response producer, and config request client.
|
|
|
|
|
- Flow operations now have Effect-returning handlers with Promise facades
|
|
|
|
|
only on exported compatibility methods.
|
|
|
|
|
- Blueprint config loading now narrows runtime values before constructing
|
|
|
|
|
`Blueprint` records, replacing the prior `parsed as Blueprint` shortcut.
|
|
|
|
|
- `start-flow` and `stop-flow` mutate the flow map through
|
|
|
|
|
`SynchronizedRef.modifyEffect`, making duplicate checks and map updates
|
|
|
|
|
atomic.
|
|
|
|
|
- The consume loop now uses `Effect.whileLoop`; the remaining
|
|
|
|
|
`consumer.receive(2000)` call is a pubsub boundary for this service.
|
|
|
|
|
- New flow-manager tests cover tagged errors, ref-backed flow mutation,
|
|
|
|
|
config push/delete requests, blueprint narrowing, duplicate concurrent
|
|
|
|
|
starts, and message-level flow-error responses.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts`
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 01:34:46 -05:00
|
|
|
### 2026-06-02: Librarian Schema And Assertion Cleanup Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/base/src/schema/messages.ts` now models librarian upload and
|
|
|
|
|
stream request/response fields directly, instead of requiring service-side
|
|
|
|
|
`as LibrarianResponse` casts for the existing wire protocol.
|
|
|
|
|
- `ts/packages/flow/src/librarian/service.ts` now decodes persisted
|
|
|
|
|
librarian state through a concrete `S.fromJsonString` schema instead of a
|
|
|
|
|
generic JSON decode plus `as A`.
|
|
|
|
|
- Document metadata `metadata` triples now narrow through Schema decoding
|
|
|
|
|
with `Option` before being included in normalized metadata.
|
|
|
|
|
- Upload, stream, and complete-upload request/response constructors now rely
|
|
|
|
|
on the schema-modeled fields instead of local type assertions.
|
|
|
|
|
- New librarian tests cover modeled upload fields, concrete persisted-state
|
|
|
|
|
loading, and schema-backed metadata triple normalization.
|
|
|
|
|
- Remaining:
|
2026-06-02 01:59:51 -05:00
|
|
|
- Resolved by the typed runtime loop and ref-backed state slices below.
|
2026-06-02 01:34:46 -05:00
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/base build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 01:39:04 -05:00
|
|
|
### 2026-06-02: Librarian Tagged Operation Helper Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- Removed the librarian `throwLibrarianServiceError` helper.
|
|
|
|
|
- `get-document-metadata`, `list-children`, `upload-chunk`,
|
|
|
|
|
`get-upload-status`, and `abort-upload` now dispatch through local
|
|
|
|
|
Effect-returning helpers that fail with `LibrarianServiceError`.
|
|
|
|
|
- Compatibility methods for those operations now return Promise facades
|
|
|
|
|
backed by `Effect.runPromise`.
|
|
|
|
|
- The librarian tests now await the Promise compatibility facade for upload
|
|
|
|
|
status.
|
|
|
|
|
- Remaining:
|
2026-06-02 01:59:51 -05:00
|
|
|
- Resolved by the typed runtime loop and ref-backed state slices below.
|
2026-06-02 01:39:04 -05:00
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts`
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 01:47:15 -05:00
|
|
|
### 2026-06-02: Librarian Typed Runtime Loop Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/librarian/service.ts` now exposes a typed
|
|
|
|
|
`LibrarianService` interface instead of `AsyncProcessorRuntime &
|
|
|
|
|
Record<string, any>`.
|
|
|
|
|
- Service construction now uses `makeAsyncProcessor<LibrarianServiceError>`
|
|
|
|
|
with `runEffect`; the old method-bag `run` override and
|
|
|
|
|
`as LibrarianService` cast are gone.
|
|
|
|
|
- The librarian startup poller now uses `Effect.whileLoop`.
|
|
|
|
|
- The local operation helpers retrieve the initialized service through an
|
|
|
|
|
Effect gate rather than closing over an unsafe partially built value.
|
|
|
|
|
- Remaining:
|
2026-06-02 01:59:51 -05:00
|
|
|
- Resolved by the ref-backed state slice below.
|
2026-06-02 01:47:15 -05:00
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 01:59:51 -05:00
|
|
|
### 2026-06-02: Librarian Ref-Backed State Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/flow/src/librarian/service.ts` now stores documents,
|
|
|
|
|
processing records, upload sessions, collection manager, and pubsub
|
|
|
|
|
handles in `SynchronizedRef<LibrarianServiceState>`.
|
|
|
|
|
- Document, processing, upload, collection, persistence, load, and stop paths
|
|
|
|
|
now read snapshots or mutate cloned maps/managers through the ref instead
|
|
|
|
|
of writing fields on the service object.
|
|
|
|
|
- Upload chunk updates clone nested `UploadSession.chunks` before replacing
|
|
|
|
|
the upload map entry, avoiding mutable nested state hidden behind the ref.
|
|
|
|
|
- Librarian response producers and consumers are read/nullified through
|
|
|
|
|
ref-backed handles.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/flow build`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `bun run --cwd ts/packages/flow test`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 02:09:45 -05:00
|
|
|
### 2026-06-02: Client RPC Managed Runtime Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/client/src/socket/effect-rpc-client.ts` now builds one
|
|
|
|
|
`ManagedRuntime` from the RPC client layer instead of manually creating a
|
|
|
|
|
`Scope`, building the layer, and calling `Effect.runPromise` for every
|
|
|
|
|
operation.
|
|
|
|
|
- RPC dispatch and stream dispatch continue to expose the existing
|
|
|
|
|
Promise-returning `EffectRpcClient` facade, but they run through the managed
|
|
|
|
|
runtime and close with `runtime.dispose()`.
|
|
|
|
|
- The Effect RPC socket path now consumes `Socket.layerWebSocketConstructorGlobal`
|
|
|
|
|
instead of a duplicate local WebSocket constructor layer.
|
|
|
|
|
- Dispatch payload construction now uses `DispatchPayload.make(...)` so
|
|
|
|
|
schema classes are not instantiated with `new`.
|
|
|
|
|
- Client socket logging and timestamp creation now use Effect `Logger` and
|
|
|
|
|
`Clock` instead of direct console and `Date.now()` calls in the touched
|
|
|
|
|
surface.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/client build`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `bun run --cwd ts/packages/client test`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 02:15:22 -05:00
|
|
|
### 2026-06-02: Client WebSocket Adapter Error Slice
|
|
|
|
|
|
|
|
|
|
- Status: migrated and root-verified.
|
|
|
|
|
- Completed:
|
|
|
|
|
- `ts/packages/client/src/socket/websocket-adapter.ts` now models host
|
|
|
|
|
fallback failures with `WebSocketAdapterError` via
|
|
|
|
|
`S.TaggedErrorClass`.
|
|
|
|
|
- Synchronous `getWebSocketConstructor()` and `getRandomValues()` facades
|
|
|
|
|
keep their public signatures while using `Result.try` instead of local
|
|
|
|
|
`try`/`catch` blocks.
|
|
|
|
|
- Runtime predicates now narrow WebSocket constructor modules and crypto
|
|
|
|
|
modules without the previous constructor/result type assertions.
|
|
|
|
|
- New adapter tests cover global WebSocket selection, optional `ws`
|
|
|
|
|
fallback, global crypto, typed crypto failure, and typed adapter errors.
|
|
|
|
|
- Verification:
|
|
|
|
|
- `bun run --cwd ts/packages/client build`
|
|
|
|
|
- `bun run --cwd ts/packages/client test -- src/__tests__/websocket-adapter.test.ts`
|
|
|
|
|
- `bun run --cwd ts/packages/client test`
|
|
|
|
|
- `cd ts && bun run check`
|
|
|
|
|
- `cd ts && bun run build`
|
|
|
|
|
- `cd ts && bun run test`
|
|
|
|
|
- `git diff --check`
|
|
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
## Subagent Findings To Preserve
|
|
|
|
|
|
|
|
|
|
- MCP/workbench:
|
|
|
|
|
- Make the Effect MCP server the canonical implementation. The old stdio
|
|
|
|
|
server should remain only as compatibility while parity is needed.
|
|
|
|
|
- Workbench BaseApi atoms can move toward `AtomRpc` or `AtomHttpApi` after
|
|
|
|
|
the client API is less Promise-first.
|
|
|
|
|
- MCP env is now Config-backed; continue that policy for future MCP settings.
|
|
|
|
|
- Flow stateful services:
|
2026-06-02 01:59:51 -05:00
|
|
|
- Config service, KnowledgeCore service, FlowManager, and Librarian
|
|
|
|
|
ref-backed state slices are complete. Follow-up service work should focus
|
|
|
|
|
on scoped layers, schedules where polling semantics allow, and managed
|
|
|
|
|
persistence providers rather than direct mutable service fields.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Persistence IO should move toward `FileSystem` or `KeyValueStore` where
|
|
|
|
|
the installed beta has the needed provider surface.
|
|
|
|
|
- Base messaging/processors:
|
|
|
|
|
- Subscriber queues/maps, processor/flow Promise compatibility, and dynamic
|
|
|
|
|
flow state should continue moving toward `Queue`, `Deferred`,
|
|
|
|
|
`SynchronizedRef`, `Schedule`, and scoped layers.
|
|
|
|
|
- Existing constructor shims and typed registries in base processors still
|
|
|
|
|
use type assertions; they need a typed factory/registry redesign rather
|
|
|
|
|
than more assertions.
|
|
|
|
|
- Gateway/client:
|
2026-06-02 02:09:45 -05:00
|
|
|
- `EffectRpcClient` now owns its socket/RPC layer with `ManagedRuntime`.
|
|
|
|
|
Remaining client cleanup should focus on `trustgraph-socket.ts`
|
2026-06-02 02:15:22 -05:00
|
|
|
higher-level normal `Error` throws/JSON parsing and the client newable
|
|
|
|
|
factory assertions.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Knowledge streams still duplicate legacy end-of-stream handling.
|
2026-06-02 02:15:22 -05:00
|
|
|
- WebSocket adapter host fallbacks now use `Result.try` and tagged adapter
|
|
|
|
|
errors while preserving sync exports.
|
2026-06-02 00:22:04 -05:00
|
|
|
- RAG/providers/storage:
|
2026-06-02 00:54:47 -05:00
|
|
|
- RAG and agent requestor bridges are complete: `toPromiseRequestor` has no
|
|
|
|
|
remaining `ts/packages` matches.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Provider SDKs and storage clients should become managed resources where
|
|
|
|
|
they have meaningful lifecycle.
|
|
|
|
|
- FalkorDB/Qdrant/Ollama/OpenAI-compatible surfaces still need config,
|
|
|
|
|
schema, and scope audits.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
## Ranked Findings
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
### P1: Finish Client RPC Boundary Modernization
|
|
|
|
|
|
|
|
|
|
- TrustGraph evidence:
|
2026-06-02 00:22:04 -05:00
|
|
|
- `ts/packages/client/src/socket/effect-rpc-client.ts`
|
|
|
|
|
- `ts/packages/client/src/socket/trustgraph-socket.ts`
|
|
|
|
|
- Effect primitives:
|
|
|
|
|
- `effect/unstable/socket` `Socket.makeWebSocket`, `fromWebSocket`,
|
2026-06-01 22:00:28 -05:00
|
|
|
`toChannel`, `layerWebSocket`.
|
2026-06-02 00:22:04 -05:00
|
|
|
- `effect/unstable/rpc/RpcClient.layerProtocolSocket`.
|
|
|
|
|
- `effect/unstable/rpc/RpcSerialization.layerNdjson` or `layerNdJsonRpc`.
|
|
|
|
|
- `ManagedRuntime` for compatibility facades when a Promise API must remain.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Rewrite shape:
|
2026-06-02 02:09:45 -05:00
|
|
|
- `EffectRpcClient` is now an internal managed runtime with Promise
|
|
|
|
|
compatibility facades.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Expose Promise-returning methods through a thin adapter.
|
2026-06-02 02:09:45 -05:00
|
|
|
- Finish replacing remaining normal client `Error` constructors with tagged
|
|
|
|
|
errors before they cross into shared Effect code.
|
2026-06-02 02:15:22 -05:00
|
|
|
- Replace remaining client newable factory assertions with a typed factory
|
|
|
|
|
shape that preserves current constructor/function compatibility.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Tests:
|
|
|
|
|
- `cd ts && bun run --cwd packages/client test`
|
2026-06-02 00:22:04 -05:00
|
|
|
|
|
|
|
|
### P1: Base Processor Registry And Constructor Shims
|
|
|
|
|
|
|
|
|
|
- TrustGraph evidence:
|
|
|
|
|
- `ts/packages/base/src/processor/async-processor.ts`
|
|
|
|
|
- `ts/packages/base/src/processor/flow.ts`
|
|
|
|
|
- `ts/packages/base/src/processor/flow-processor.ts`
|
|
|
|
|
- Effect primitives:
|
|
|
|
|
- Schema-backed registries, `Context`, `Layer`, `Effect.fn`, `Option`,
|
|
|
|
|
`Predicate`.
|
|
|
|
|
- Rewrite shape:
|
|
|
|
|
- Replace constructor `as unknown as` shims with typed factory exports.
|
|
|
|
|
- Replace resource lookup casts with schema-backed typed registry helpers.
|
|
|
|
|
- Do not add assertions to quiet Effect channel inference problems.
|
|
|
|
|
- Tests:
|
|
|
|
|
- `cd ts && bun run --cwd packages/base test`
|
|
|
|
|
- Root `cd ts && bun run check` because this surface easily pollutes Effect
|
|
|
|
|
error and requirement channels.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
### P1: Make SDK, Storage, And Provider Layers Managed Resources
|
|
|
|
|
|
|
|
|
|
- TrustGraph evidence:
|
|
|
|
|
- `ts/packages/flow/src/storage/triples/falkordb.ts`
|
|
|
|
|
- `ts/packages/flow/src/storage/embeddings/qdrant-graph.ts`
|
|
|
|
|
- `ts/packages/flow/src/storage/embeddings/qdrant-doc.ts`
|
|
|
|
|
- `ts/packages/flow/src/model/text-completion/*.ts`
|
2026-06-02 00:22:04 -05:00
|
|
|
- `ts/packages/flow/src/embeddings/ollama.ts`
|
|
|
|
|
- Effect primitives:
|
2026-06-01 22:00:28 -05:00
|
|
|
- `Effect.acquireRelease`, `Layer.scoped`, `Config`, `ConfigProvider`,
|
2026-06-02 00:22:04 -05:00
|
|
|
`Metric`, `Logger`, Effect AI provider layers.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Rewrite shape:
|
2026-06-02 00:22:04 -05:00
|
|
|
- Move env/config reading into `Config` loaders and provider-specific layers.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Scope SDK clients that need explicit close/disconnect.
|
2026-06-02 00:22:04 -05:00
|
|
|
- Remove `Effect.void as Effect.Effect<undefined>` stream assertions by
|
|
|
|
|
letting branch return types infer or by restructuring the stream parser.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Tests:
|
|
|
|
|
- Provider config tests with `ConfigProvider.fromMap`.
|
|
|
|
|
- Storage tests with fake clients before changing real resource lifetimes.
|
|
|
|
|
|
|
|
|
|
### P2: Canonicalize MCP Around The Effect Server
|
|
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
- Status:
|
|
|
|
|
- First blocker slice complete: MCP now builds under strict tsgo and the
|
|
|
|
|
stdio server has an Effect-backed compatibility implementation.
|
|
|
|
|
- Remaining shape:
|
|
|
|
|
- Decide whether the old SDK/Zod stdio compatibility surface should stay as
|
|
|
|
|
a wrapper or be removed.
|
|
|
|
|
- Add parity tests before deleting any public entry point.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Tests:
|
2026-06-02 00:22:04 -05:00
|
|
|
- `cd ts && bun run --cwd packages/mcp build`
|
|
|
|
|
- Root `cd ts && bun run check`
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
### P2: Tighten Workbench Platform And Reactivity Usage
|
|
|
|
|
|
|
|
|
|
- TrustGraph evidence:
|
2026-06-02 00:22:04 -05:00
|
|
|
- `ts/packages/workbench/src/atoms/workbench.ts`
|
|
|
|
|
- Remaining direct browser state includes `localStorage` and DOM theme
|
|
|
|
|
inspection.
|
|
|
|
|
- Effect primitives:
|
2026-06-01 22:00:28 -05:00
|
|
|
- `BrowserKeyValueStore.layerLocalStorage`,
|
|
|
|
|
`BrowserKeyValueStore.layerSessionStorage`, `BrowserHttpClient`,
|
2026-06-02 00:22:04 -05:00
|
|
|
`Clipboard`, `AtomRpc`, `AtomHttpApi`, `AtomRegistry`, `AsyncResult`,
|
|
|
|
|
`Reactivity`.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Rewrite shape:
|
2026-06-02 00:22:04 -05:00
|
|
|
- Leave workbench out of the next backend/runtime rewrite wave.
|
|
|
|
|
- Move persistent UI state through browser platform services later.
|
2026-06-01 22:00:28 -05:00
|
|
|
- Tests:
|
2026-06-02 00:22:04 -05:00
|
|
|
- `cd ts && bun run workbench:qa`
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
## Recommended PR Order
|
|
|
|
|
|
2026-06-02 01:59:51 -05:00
|
|
|
1. Client RPC managed runtime/scoped layer cleanup.
|
|
|
|
|
2. Base processor registry and constructor shim redesign.
|
|
|
|
|
3. Gateway RPC callback and client streaming completion cleanup.
|
|
|
|
|
4. Storage/provider managed resource cleanup.
|
|
|
|
|
5. MCP parity/deletion decision and workbench platform polish.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
|
|
|
|
## No-Op Rules
|
|
|
|
|
|
|
|
|
|
Do not flag these as rewrite blockers without additional proof:
|
|
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
- Promise-returning CLI actions, MCP SDK callbacks, client compatibility
|
|
|
|
|
methods, and Fastify route handlers at true external boundaries. Boundary
|
|
|
|
|
code still must map failures into typed errors or wire errors.
|
2026-06-01 22:17:50 -05:00
|
|
|
- `try`/`catch` blocks at host/tool boundaries only when the catch maps into a
|
2026-06-02 00:22:04 -05:00
|
|
|
typed error or a wire-contract error. Internal exception capture should use
|
|
|
|
|
`Effect.try`, `Effect.tryPromise`, or `Result.try`.
|
2026-06-01 22:00:28 -05:00
|
|
|
- `S.Class`, `S.TaggedErrorClass`, `Context.Service`, `Rpc.make`, and
|
|
|
|
|
`HttpApi.make` when they are required or idiomatic for the Effect API.
|
|
|
|
|
- Plain `Map` usage for local pure transformations, such as graph utility
|
2026-06-02 00:22:04 -05:00
|
|
|
construction, unless the state is long-lived mutable service state.
|
|
|
|
|
- JSON stringification in tests or wire-contract fixtures. Production JSON
|
|
|
|
|
encode/decode should prefer schema codecs when the encoded form can be
|
|
|
|
|
preserved.
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
## Acceptance For Final Loop Completion
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
The overall playbook loop is complete only when:
|
2026-06-01 22:00:28 -05:00
|
|
|
|
2026-06-02 00:22:04 -05:00
|
|
|
- All remaining playbook signal matches are migrated or documented as no-op
|
|
|
|
|
external-boundary cases with concrete evidence.
|
|
|
|
|
- No P0/P1/P2 migration item remains in this audit.
|
|
|
|
|
- `cd ts && bun run check`, `cd ts && bun run build`, `cd ts && bun run test`,
|
|
|
|
|
and `git diff --check` pass after the final migration slice.
|