# TrustGraph Effect-Native Rewrite Opportunity Audit This is the current backlog snapshot for the playbook in `ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md`. The branch is `ts-port-effect-v4`. The unrelated local file `.idea/effect.intellij.xml` must stay uncommitted. ## Inputs Verified source roots: - TrustGraph TS port: `/home/elpresidank/YeeBois/dev/trustgraph/ts` - Effect v4 subtree: `/home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4` - Installed Effect beta used by this workspace: `ts/node_modules/effect` Current signal counts from `ts/packages` after the 2026-06-02 dispatcher Effect collections slice: | Signal | Count | | --- | ---: | | `Effect.runPromise` | 175 | | `Effect.runPromiseWith` | 0 | | `Effect.cached` | 0 | | `Layer.succeed` | 13 | | `Map<` | 86 | | `WebSocket` | 72 | | `new Map` | 56 | | `new Set` | 15 | | `Set<` | 9 | | `toPromiseRequestor` | 0 | | `makeAsyncProcessor` | 19 | | `receive(` | 17 | | `while (` | 2 | | `new Error` | 7 | | `new Promise` | 9 | | `JSON.parse` | 4 | | `localStorage` | 9 | | `JSON.stringify` | 8 | | `setTimeout` | 3 | | `process.env` | 3 | Notes: - The remaining `process.env` hits are in `packages/workbench/playwright.config.ts`. - In production `packages/base`, `packages/cli`, and `packages/mcp` sources, the strict scans for `new Error`, `new Promise`, `setTimeout`, `JSON.parse`, `JSON.stringify`, and direct `process.env` reads are clean. - `Effect.runPromise` is expected at external Promise compatibility boundaries, but each match should still be audited for avoidable internal runtime ownership. - The dispatcher Effect collections slice removed native `Map`/`Set` from the gateway service registries, streaming membership set, and scoped requestor cache. Remaining broad `Map`/`Set` matches include tests/fakes, WeakMap compatibility caches, short-lived pure traversal collections, and larger ref-backed service state that still needs focused `HashMap`/`MutableHashMap` cleanup. - The `Effect.runPromise` and `WebSocket` counts dropped in this snapshot because `EffectRpcClient` now owns its RPC/socket layer with `ManagedRuntime` and uses Effect's WebSocket constructor layer. - The raw `WebSocket` count increased in this snapshot because the adapter slice added focused tests and typed adapter names; production `websocket-adapter.ts` is now clean of `try`/`catch`, normal `Error`, and the previous constructor assertions. - The `new Error` count dropped because `websocket-adapter.ts` now throws `S.TaggedErrorClass` adapter errors. - The latest client socket slice removed the remaining production `trustgraph-socket.ts` normal `Error`, raw `JSON.parse`, and listener `try`/`catch` matches. The remaining client socket modernization signal is the shared `newableFactory` constructor assertion pattern. - The service entrypoint runtime slice dropped the `Effect.runPromise` count by replacing remaining flow service `run()` program facades with `ManagedRuntime` and routing local `ts/scripts/run-*` launchers through `runMain()`/`NodeRuntime.runMain`. - The base processor compatibility runtime slice dropped the `Effect.runPromise` count again by moving `AsyncProcessor`, `Flow`, and `FlowProcessor` Promise compatibility facades onto `ManagedRuntime`. - The base flow definition schema slice removed hand-rolled `Predicate`/object narrowing from `flow-processor.ts`; signal counts are unchanged because this was a validation-quality migration. - The text completion stream sentinel slice removed the duplicated `Effect.void as Effect.Effect` assertions from provider stream unfold branches. Counts are unchanged because this was an Effect diagnostic and type-channel cleanup. - The text completion generator boundary slice removed the `Effect.runPromise(Effect.fail(...))` fallback and the related `AsyncGenerator`/`IteratorResult` assertions from `model/text-completion/common.ts`. - The text completion provider status slice replaced manual status/statusCode record assertions with `effect/Predicate` narrowing. - The base parameter spec accessor slice added Schema-backed `ParameterSpec` values plus `flow.parameterEffect(spec)` and `flow.parameter(spec)`. Bare string parameter lookup remains available as an `unknown` compatibility escape, while typed parameter access now decodes through Schema and fails with a tagged `FlowParameterDecodeError`. - The base producer/requestor spec accessor slice added typed spec-object accessors for `ProducerSpec` and `RequestResponseSpec`, then migrated flow service producer/requestor lookups off caller-chosen generic string calls. Spec object handles are scoped per `Flow` through WeakMaps and finalizers delete only the handle they registered. - The native PubSub boundary slice removed the unused legacy `messaging/subscriber.ts` async queue/fanout implementation. Effect's native `PubSub` is an in-process hub and does not replace the broker-backed `PubSubBackend`/NATS boundary, but it should be preferred for future in-process broadcast/fanout needs. - The base producer scoped runtime slice moved the legacy `makeProducer` Promise facade onto the existing `makeEffectProducerFromPubSub` scoped factory. Public `start`/`send`/`stop` remain Promise compatibility boundaries, while producer allocation, flush, and finalizer close now go through the Effect runtime path. - The NATS typed boundary slice removed the dynamic `import("nats")` header path and maps header construction plus `ack()`/`nak()` failures into tagged `PubSubError`s with `Effect.try`. The `receive(` and `JSON.stringify` count increases are from the new mocked NATS backend test, not production code. - The NATS selective 404 slice replaced catch-all stream/consumer create fallbacks with an internal `S.TaggedErrorClass` lookup wrapper plus `Effect.catchIf` recovery only for NATS JetStream missing-resource errors. Non-missing lookup failures now stay on the typed failure path without attempting to create streams or durable consumers. - The consumer rate-limit retry slice wired the previously unused `rateLimitTimeoutMs` option and `TG_RATE_LIMIT_TIMEOUT_MS` config into both legacy and Effect-native consumers. Repeated `TooManyRequestsError` failures now retry with `Schedule.spaced` until success or a tagged rate-limit timeout. The `new Error` count dropped by one because a touched consumer test fixture no longer uses a normal `Error`. - The consumer concurrency ownership slice changed the Effect-native consumer runtime so `concurrency > 1` allocates one backend consumer per worker instead of sharing a single `BackendConsumer.receive()` handle. `stop` is now idempotent through `Ref`, so explicit stop and scoped finalizers do not close workers twice. - The request-response stop signal slice added a `Deferred` shutdown signal to `makeEffectRequestResponseFromPubSub`. Pending requests now race response waiting against runtime stop and fail promptly with a tagged `MessagingLifecycleError` instead of waiting for timeout. - The legacy consumer facade slice moved `makeConsumer` onto `makeEffectConsumerFromPubSub` with a `ManagedRuntime` Promise boundary and a closeable `Scope`. Consumer workers now use `Effect.forkScoped` so their lifetime is owned by the caller scope rather than the parent fiber. The `Effect.runPromise`, `receive(`, `new Promise`, and `setTimeout` counts dropped because the old blocking facade loop and its test timer shim were removed. - The workbench theme storage slice stopped mirroring `themeAtom` into the legacy `tg-theme` localStorage key. The canonical `trustgraph-workbench-theme-v1` value remains owned by `Atom.kvs` over `BrowserKeyValueStore.layerLocalStorage`; the first-paint host script reads that JSON-encoded key before React mounts and falls back to `tg-theme` only for legacy installs. - The Effect AI `LanguageModel` adapter slice added a reusable `makeLanguageModelProvider` bridge in text-completion common code. It maps `generateText` responses to `LlmResult`, maps streaming `text-delta` and final `finish.usage` parts to TrustGraph chunks, and converts Effect AI rate and quota failures into `TooManyRequestsError`. No concrete provider has been flipped yet. - The native request/response PubSub slice removed the local `Map` response subscriber fanout in `makeEffectRequestResponseFromPubSub`. Response dispatch now publishes `{ id, value }` envelopes through native `effect/PubSub`, and each request uses a scoped `PubSub.Subscription` plus `Stream.fromSubscription` to wait for its matching response. - The Claude Effect AI slice moved the Claude provider off the direct `@anthropic-ai/sdk` wrapper and onto `@effect/ai-anthropic` `AnthropicLanguageModel` through `makeLanguageModelProvider`. The direct SDK dependency was removed from `@trustgraph/flow`. - A focused broker-backend scout found no remaining P0 broker runtime rewrite after the producer, NATS, consumer concurrency, rate-limit, and request-response stop slices. `PubSubBackend` remains an intentional Promise-returning adapter boundary wrapped by `PubSub`/Effect services. - The gateway streaming callback slice added Effect-returning dispatcher streaming methods, switched the RPC stream server off nested `Effect.runPromiseWith(context)` queue offers, and replaced the client `StopStreaming` sentinel error with `Stream.runForEachWhile`. - The FalkorDB scoped client lifecycle slice removed the remaining `Effect.cached` matches from `ts/packages`. FalkorDB triples store/query Live layers and direct compatibility factories now acquire clients through `Effect.acquireRelease` and disconnect them on scope close. The `Effect.runPromise` count increased by two because the new lifecycle tests run scoped programs at the test boundary. - The Qdrant config/schema/fakeability slice removed direct production `new QdrantClient`, sync config loading, payload casts, and Qdrant `Layer.succeed` service construction from graph/doc store/query modules. The installed Qdrant client exposes no public close/disconnect method, so this remains a fakeable construction and Schema decode slice rather than a scoped finalizer slice. `Effect.runPromise` increased because the new tests and legacy service initialization logs run Effects at compatibility boundaries. - The client streaming facade slice did not change signal counts. It centralized the legacy streaming `{ response, complete, error }` envelope decode in `trustgraph-socket.ts`, uses Schema plus `effect/Predicate` property narrowing for streaming payload reads, and leaves service-specific legacy completion markers only where they preserve public callback behavior. - The Ollama embeddings effectful layer slice dropped one `Layer.succeed` match by making `OllamaEmbeddingsLive` effectful and mapping config/load failures to `EmbeddingsError`. The `JSON.stringify` count increased by one because the new layer test uses a JSON response fixture. - The text completion provider stream helper slice removed all provider-local `Stream.unfold` pull loops, dropped the `while (` count from 9 to 3, and removed the Mistral `content as string` assertion. The only remaining text-completion `iterator.next` match is the `toAsyncGenerator` compatibility adapter that exposes Effect streams through the public `AsyncGenerator` provider contract. - The request-response queue stream slice replaced the Effectful `waitForResponse` generator loop with `Stream.fromQueue`, `Stream.filterMapEffect`, `Result`, and `Stream.runHead`, dropping the remaining `while (` count from 3 to 2. The two remaining production `while` hits are synchronous parsing/CLI traversal loops, not async polling loops. - The gateway RPC WebSocket cause-handling slice removed the Promise `.catch` around the socket program by sandboxing the Effect and handling the resulting `Cause` in the Effect pipeline before the Fastify fire-and-forget `runPromise` boundary. - The client RPC acquisition cause tap slice removed the Promise `.catch` used only to update connection state on runtime/client acquisition failure. `effect-rpc-client.ts` now uses `Effect.tapCause` and `Cause.pretty` before the public Promise boundary. - The client socket close Effect boundary slice removed the Promise `.catch` from `BaseApi.close()`. The void public facade now runs `rpc.close()` through `Effect.tryPromise` and logs the tagged socket close error through `Effect.catch`. - The client streaming callback Effect boundary slice removed the remaining production Promise `.catch` matches from `trustgraph-socket.ts` by centralizing legacy callback request failures in `runLegacyStreamingRequest`. The public callback facades still return/ignore Promises where required, but failure mapping now uses `Effect.tryPromise` and `Effect.catch`. - The text completion provider effectful layer slice dropped six `Layer.succeed` matches by moving OpenAI, OpenAI-compatible, Azure OpenAI, Claude, Mistral, and Ollama processor layers onto `makeTextCompletionLayer(makeXProviderEffect(config))`. SDK construction and config lookup now live in Effect; sync `makeXProvider` exports remain compatibility facades. - The gateway dispatcher ownership and serialization slice did not change broad signal counts. It stopped closing injected pubsub backends, brackets one-shot publish producers with `Effect.acquireUseRelease`, and routes gateway request/response translation through `Effect.try` wrappers returning tagged `DispatchSerializationError` failures. - `Record` and `throwLibrarianServiceError` are now clean in `ts/packages`. ## Loop Passes ### 2026-06-02: Base Request/Response Facade - Status: migrated and verified. - Completed: - Request/response startup now owns a scoped Effect runtime handle and maps failures to TrustGraph tagged messaging errors. - Runtime shutdown is idempotent and uses scoped fibers. - Tests cover Promise compatibility, tagged timeout errors, and tagged lifecycle errors. - Verification: - `bun run --cwd ts/packages/base test` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts check:tsgo` - `bun run --cwd ts build` - `bun run --cwd ts test` ### 2026-06-02: Gateway Dispatcher Requestor Cache - Status: migrated and package-verified. - Completed: - Gateway dispatcher caches scoped `EffectRequestResponse` handles instead of `Promise` values. - Lazy requestor creation is serialized with `SynchronizedRef.modifyEffect`. - Streaming final-marker detection is centralized. - Dispatcher cleanup uses Effect scope/error handling instead of manual `try`/`catch`. - Verification: - `bun run --cwd ts/packages/flow test` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts check:tsgo` ### 2026-06-02: Gateway Dispatcher Effect Collections Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/gateway/dispatch/manager.ts` now stores the flow/global service registries in `effect/HashMap` instead of native `ReadonlyMap`, while explicit entry arrays preserve the public service-name ordering. - Streaming service membership now uses `effect/HashSet` instead of native `Set`. - The scoped requestor cache now stores `HashMap>` in the existing `SynchronizedRef`, replacing `new Map` cloning with immutable `HashMap.set`. - Cache hits and service topic lookups now use `HashMap.get` plus `effect/Option`, and ref update tuples use `effect/Tuple.make` instead of `as const` assertions. - Gateway dispatcher tests now cover concurrent same-key dispatches so the cache still creates exactly one scoped producer/consumer pair. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/gateway-dispatcher.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-02: Strict Base, CLI, MCP, And tsgo Slice - Status: migrated, root-verified, committed, and pushed. - Completed: - Base messaging, NATS backend, producer, consumer, subscriber, request/response, runtime factories, processor programs, flow specs, and LLM service now use Effect-native boundaries, schema codecs, scoped cleanup, and `S.TaggedErrorClass.make(...)` errors. - CLI commands now run Effect programs at the command boundary, wrap socket lifecycle with `Effect.acquireUseRelease`, encode JSON through Effect Schema, and write output without `console.log`. - MCP Effect server now loads env/config through `Config`, wraps gateway calls with `Effect.tryPromise`, constructs schema classes with `.make`, and uses tagged errors. - MCP stdio compatibility server keeps `createMcpServer` and `run`, but uses Effect callbacks/tryPromise/schema encoding internally. `run()` uses `ManagedRuntime`; `runMain()` uses `NodeRuntime.runMain`. - Flow stateful service launch sites now pass an explicit `Context.Context` into the base processor runtime instead of hiding requirements behind assertions. - Verification: - `cd ts && bun run check` - `cd ts && bun run test` - `cd ts && bun run build` - `git diff --check` ### 2026-06-02: ConfigService Ref-Backed State Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/config/service.ts` now models runtime state as a `SynchronizedRef` instead of adding mutable `store`, `version`, consumer, and producer fields onto the processor object. - Config operations have Effect-returning handlers with Promise facades only on the exported compatibility methods. - Request narrowing now uses `effect/Predicate` rather than request-record type assertions. - Persistence remains schema-backed and now reads/writes snapshots from the ref-backed state. - The consume loop now uses `Effect.whileLoop`; the remaining `consumer.receive(2000)` call is a pubsub boundary for this service. - Service startup now exposes `runMain()` through `NodeRuntime.runMain`. The legacy `run()` Promise facade uses `ManagedRuntime`, and `ts/scripts/run-config.ts` delegates directly to `runMain()` instead of owning its own catch/process-exit wrapper. - Config-service tests cover tagged invalid mutation errors, workspace persistence, legacy load, concurrent ref-backed mutations, and push publishing from the stored producer handle. - Verification: - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` - `git diff --check` ### 2026-06-02: ConfigService Operation Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/config/service.ts` now dispatches `ConfigOperation` with `effect/Match` instead of a native `switch`. - The dispatcher is a named `Effect.fn` and uses `Match.exhaustive` against the schema-derived `ConfigOperation` union. - The per-message response sender now uses `Effect.fnUntraced` instead of an arrow function returning `Effect.gen(...)`. - Config-service tests now cover all seven operations through `handleOperation`, including tagged invalid mutation failures. - Existing explicit `never` annotations on `persistEffect`, `loadFromDiskEffect`, `persistStateEffect`, and `readPersistedConfigEffect` were removed so Effect can infer the channel. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/config-service.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-02: Librarian Operation Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/librarian/service.ts` now dispatches librarian and collection-management operations with `effect/Match` instead of native `switch` statements. - Both dispatchers intentionally use `Match.orElse` rather than `Match.exhaustive` because raw broker message values can still contain unknown runtime operations before a schema boundary rejects them. - Existing tagged `LibrarianServiceError` operation labels are preserved for promise, sync, Effect-helper, stream-only, and unknown-operation branches. - Librarian-service tests now cover representative Match-backed librarian dispatch paths, collection list/update/delete dispatch, and runtime fallback errors without type assertions. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: FlowManager Operation Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/flow-manager/service.ts` now dispatches flow operations with `effect/Match` instead of a native `switch`. - The dispatcher keeps the existing config refresh behavior before routing and uses `Match.orElse` because `FlowRequest.operation` is a public wire-level `string`, not a closed schema literal union. - Existing tagged `FlowManagerError` behavior is preserved for unknown operations and branch-specific failures. - Flow-manager tests now cover all eight flow operations through `handleOperation`, including config-client blueprint mutations and the runtime unknown-operation fallback. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: Client Connection State SubscriptionRef Slice - Status: migrated and package-verified. - Completed: - `ts/packages/client/src/socket/effect-rpc-client.ts` now owns RPC connection state in `effect/SubscriptionRef` instead of a mutable state variable plus manual listener `Set`. - `ts/packages/client/src/socket/trustgraph-socket.ts` now bridges UI connection-state listeners through `SubscriptionRef.changes` instead of a hand-rolled listener array. - Both public `subscribe` APIs preserve synchronous immediate replay and unsubscribe compatibility while using Effect fibers for later updates. - Client tests now drive a fake RPC state source to prove immediate replay, connection updates, and unsubscribe behavior. - Verification: - `bun run --cwd ts/packages/client test -- src/__tests__/rpc-timeout.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: Client Callback Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/client/src/socket/trustgraph-socket.ts` now maps RPC connection status with `effect/Match` instead of a native `switch`. - Flow agent streaming chunk callbacks now use `effect/Match` for `thought`, `observation`, `answer`, `final-answer`, and `action`, while preserving ignored behavior for unknown chunk types with `Match.orElse`. - Client RPC tests now drive agent stream chunks through the fake RPC stream to prove callback dispatch, ignored fallback behavior, completion signals, and metadata forwarding. - Verification: - `bun run --cwd ts/packages/client test -- src/__tests__/rpc-timeout.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: Gateway Term Service HashSet Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/gateway/dispatch/serialize.ts` now uses `effect/HashSet` for static term-bearing request/response service membership instead of native `Set`. - Request and response translators preserve the same deep client/internal term conversion behavior via `HashSet.has` membership checks. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/gateway-dispatcher.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: Effect AI Stream Part Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/model/text-completion/common.ts` now maps `Response.StreamPart` values with `effect/Match` instead of a native `switch`. - The matcher handles `text-delta`, `finish`, and `error` explicitly, while preserving ignored behavior for other valid stream parts with `Match.orElse`. - Text-completion common tests now include an ignored `text-start` stream part before text deltas to prove the fallback path remains silent. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: MCP Tool Transport Adapter Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/agent/mcp-tool/service.ts` no longer uses an `as unknown as Parameters[0]` assertion when connecting the MCP SDK client. - The concrete `StreamableHTTPClientTransport` is wrapped in a small `Transport` adapter that forwards callbacks, send, close, and protocol version updates while avoiding the SDK's exact-optional `sessionId` type mismatch. - The transport release path still closes the concrete SDK transport. - Verification: - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check:tsgo` ### 2026-06-04: Agent Service Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/agent/react/service.ts` no longer uses native `switch` for configured tool construction, live tool wiring, or ReAct continuation parsing. - Configured tool construction now uses `Effect.fn` plus `effect/Match` and preserves unknown-tool logging/fallback behavior. - The ReAct parser is exported for focused tests and uses an exhaustive `Match` over continuation sections. - Agent service tests cover configured-tool loading, unknown configured-tool fallback, default descriptions, MCP tool args, continuation parsing, and final-answer parsing. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/agent-service.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: CLI Library MIME Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/cli/src/commands/library.ts` now uses `effect/Match` for extension-to-MIME mapping instead of a native `switch`. - `guessMimeType` is exported for focused CLI helper coverage. - CLI tests now cover known extensions, case normalization, `html`/`htm` aliases, and fallback behavior for unknown or extensionless paths. - `ts/packages/cli/package.json` excludes `dist/**` from Vitest so built test output is not executed a second time after root build lanes. - Verification: - `bun run --cwd ts/packages/cli test -- src/__tests__/library.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-04: Streaming ReAct Parser Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/agent/react/parser.ts` now uses `effect/Match` for ReAct state emission instead of a native `switch`. - Marker states now use an inferred `MarkerState` type instead of per-entry `as ReActState` assertions. - Flush now processes an unterminated final buffer line through marker detection, fixing `Final Answer:` chunks that arrive without a trailing newline. - Parser tests cover split markers, pre-marker thought fallback, continuations, action input, and final-answer routing. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/agent-parser.test.ts` - `cd ts && bun run check:tsgo` ### 2026-06-02: RAG And Agent Requestor Bridge Slice - Status: migrated, root-verified, committed, and pushed. - Completed: - `ts/packages/flow/src/retrieval/graph-rag.ts` and `ts/packages/flow/src/retrieval/document-rag.ts` now accept `EffectRequestResponse` clients directly. The engines no longer adapt Effect requestors back to Promise requestors and then wrap those calls in `Effect.tryPromise`. - `ts/packages/flow/src/retrieval/graph-rag-service.ts` and `ts/packages/flow/src/retrieval/document-rag-service.ts` now pass native flow requestors directly into the engines. - `ts/packages/flow/src/agent/react/tools.ts` now accepts `EffectRequestResponse` clients directly for graph RAG, document RAG, triples, and MCP tool calls. Tool input narrowing uses Schema and `effect/Predicate` rather than local request/response type assertions. - `ts/packages/flow/src/agent/react/service.ts` wires default and configured tools with native Effect requestors instead of `toPromiseRequestor`. - Graph RAG, document RAG, and agent service startup now expose `runMain()` through `NodeRuntime.runMain`; their legacy `run()` Promise facades use `ManagedRuntime`. - `ts/scripts/run-graph-rag.ts`, `ts/scripts/run-document-rag.ts`, and `ts/scripts/run-agent.ts` now delegate to `runMain()`. - Verification: - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: KnowledgeCore Ref-Backed State Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/cores/service.ts` now exposes a typed `KnowledgeCoreService` instead of `AsyncProcessorRuntime & Record`. - Runtime state now lives in `SynchronizedRef` with `kgCores`, `deCores`, the request consumer, and response producer. - Knowledge operations now have Effect-returning handlers with Promise facades only on exported compatibility methods. - Persistence now decodes legacy and current snapshot shapes with Effect Schema and encodes JSON through Schema rather than raw `JSON.parse`/`JSON.stringify` plus assertions. - The consume loop now uses `Effect.whileLoop`; the remaining `consumer.receive(2000)` call is a pubsub boundary for this service. - The service exposes `runMain()` through `NodeRuntime.runMain`; legacy `run()` uses `ManagedRuntime`, and `ts/scripts/run-knowledge.ts` delegates to `runMain()`. - `ts/packages/base/src/schema/messages.ts` now models legacy hyphenated knowledge request/response aliases so the service can preserve the wire shape without response type assertions. - New knowledge-core tests cover ref-backed mutation, graph embedding alias responses, concurrent state updates, and legacy persistence loading. - Verification: - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: KnowledgeCore Operation Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/cores/service.ts` now dispatches `KnowledgeOperation` with `effect/Match` instead of a native `switch`. - The dispatcher is a named `Effect.fn` and uses `Match.exhaustive` against the schema-derived `KnowledgeOperation` union, so newly modeled operations should surface as type/check failures until handled. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/knowledge-core-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Flow Manager And Librarian Runtime Normalization - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/flow-manager/service.ts` and `ts/packages/flow/src/librarian/service.ts` now expose `runMain()` through `NodeRuntime.runMain`. - Their legacy `run()` Promise facades now use `ManagedRuntime` instead of directly owning `Effect.runPromise`. - `ts/scripts/run-flow-manager.ts` and `ts/scripts/run-librarian.ts` now delegate to `runMain()` instead of wrapping startup with local `.catch(console.error/process.exit)` handlers. - Verification: - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: FlowManager Ref-Backed State Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/flow-manager/service.ts` now exposes a typed `FlowManagerService` instead of `AsyncProcessorRuntime & Record`. - Runtime state now lives in `SynchronizedRef` with `flows`, `blueprints`, the request consumer, response producer, and config request client. - Flow operations now have Effect-returning handlers with Promise facades only on exported compatibility methods. - Blueprint config loading now narrows runtime values before constructing `Blueprint` records, replacing the prior `parsed as Blueprint` shortcut. - `start-flow` and `stop-flow` mutate the flow map through `SynchronizedRef.modifyEffect`, making duplicate checks and map updates atomic. - The consume loop now uses `Effect.whileLoop`; the remaining `consumer.receive(2000)` call is a pubsub boundary for this service. - New flow-manager tests cover tagged errors, ref-backed flow mutation, config push/delete requests, blueprint narrowing, duplicate concurrent starts, and message-level flow-error responses. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: FlowManager Effect.fn Normalization Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/flow-manager/service.ts` no longer defines reusable helpers as arrow functions that immediately return `Effect.gen(...)`. - Config request, blueprint refresh, flow refresh, blueprint handlers, flow handlers, config push/delete, resource close, consume, run, and local operation handling now use named `Effect.fn` providers. - Hot local helpers for one-message consumption and response sending use `Effect.fnUntraced`. - `pushFlowsConfigEffect` keeps its best-effort logging/swallowing contract through the `Effect.fn` pipeable form instead of a wrapper generator. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Librarian Schema And Assertion Cleanup Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/schema/messages.ts` now models librarian upload and stream request/response fields directly, instead of requiring service-side `as LibrarianResponse` casts for the existing wire protocol. - `ts/packages/flow/src/librarian/service.ts` now decodes persisted librarian state through a concrete `S.fromJsonString` schema instead of a generic JSON decode plus `as A`. - Document metadata `metadata` triples now narrow through Schema decoding with `Option` before being included in normalized metadata. - Upload, stream, and complete-upload request/response constructors now rely on the schema-modeled fields instead of local type assertions. - New librarian tests cover modeled upload fields, concrete persisted-state loading, and schema-backed metadata triple normalization. - Remaining: - Resolved by the typed runtime loop and ref-backed state slices below. - Verification: - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Librarian Tagged Operation Helper Slice - Status: migrated and root-verified. - Completed: - Removed the librarian `throwLibrarianServiceError` helper. - `get-document-metadata`, `list-children`, `upload-chunk`, `get-upload-status`, and `abort-upload` now dispatch through local Effect-returning helpers that fail with `LibrarianServiceError`. - Compatibility methods for those operations now return Promise facades backed by `Effect.runPromise`. - The librarian tests now await the Promise compatibility facade for upload status. - Remaining: - Resolved by the typed runtime loop and ref-backed state slices below. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Librarian Typed Runtime Loop Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/librarian/service.ts` now exposes a typed `LibrarianService` interface instead of `AsyncProcessorRuntime & Record`. - Service construction now uses `makeAsyncProcessor` with `runEffect`; the old method-bag `run` override and `as LibrarianService` cast are gone. - The librarian startup poller now uses `Effect.whileLoop`. - The local operation helpers retrieve the initialized service through an Effect gate rather than closing over an unsafe partially built value. - Remaining: - Resolved by the ref-backed state slice below. - Verification: - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Librarian Ref-Backed State Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/librarian/service.ts` now stores documents, processing records, upload sessions, collection manager, and pubsub handles in `SynchronizedRef`. - Document, processing, upload, collection, persistence, load, and stop paths now read snapshots or mutate cloned maps/managers through the ref instead of writing fields on the service object. - Upload chunk updates clone nested `UploadSession.chunks` before replacing the upload map entry, avoiding mutable nested state hidden behind the ref. - Librarian response producers and consumers are read/nullified through ref-backed handles. - Verification: - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client RPC Managed Runtime Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/effect-rpc-client.ts` now builds one `ManagedRuntime` from the RPC client layer instead of manually creating a `Scope`, building the layer, and calling `Effect.runPromise` for every operation. - RPC dispatch and stream dispatch continue to expose the existing Promise-returning `EffectRpcClient` facade, but they run through the managed runtime and close with `runtime.dispose()`. - The Effect RPC socket path now consumes `Socket.layerWebSocketConstructorGlobal` instead of a duplicate local WebSocket constructor layer. - Dispatch payload construction now uses `DispatchPayload.make(...)` so schema classes are not instantiated with `new`. - Client socket logging and timestamp creation now use Effect `Logger` and `Clock` instead of direct console and `Date.now()` calls in the touched surface. - Verification: - `bun run --cwd ts/packages/client build` - `cd ts && bun run check` - `bun run --cwd ts/packages/client test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client WebSocket Adapter Error Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/websocket-adapter.ts` now models host fallback failures with `WebSocketAdapterError` via `S.TaggedErrorClass`. - Synchronous `getWebSocketConstructor()` and `getRandomValues()` facades keep their public signatures while using `Result.try` instead of local `try`/`catch` blocks. - Runtime predicates now narrow WebSocket constructor modules and crypto modules without the previous constructor/result type assertions. - New adapter tests cover global WebSocket selection, optional `ws` fallback, global crypto, typed crypto failure, and typed adapter errors. - Verification: - `bun run --cwd ts/packages/client build` - `bun run --cwd ts/packages/client test -- src/__tests__/websocket-adapter.test.ts` - `bun run --cwd ts/packages/client test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` - `git diff --check` ### 2026-06-02: Client Socket Tagged Error And JSON Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/trustgraph-socket.ts` now models socket API failures with `TrustGraphSocketError` via `S.TaggedErrorClass`. - Flow/blueprint JSON response parsing now uses Schema decoding through `S.UnknownFromJsonString` instead of raw `JSON.parse`. - Token-cost config JSON keeps the previous invalid-string fallback behavior while decoding through Schema/Option. - Connection-state listener isolation now uses `Result.try` and typed socket errors instead of a local `try`/`catch`. - Flow start, row embeddings, collection update, and response-error failures now reject with tagged socket errors instead of normal `Error`. - Flow API tests cover invalid JSON and response-error rejections. - Verification: - `bun run --cwd ts/packages/client build` - `bun run --cwd ts/packages/client test -- src/__tests__/flows-api.test.ts` - `cd ts && bun run check` - `bun run --cwd ts/packages/client test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client Newable Factory Compatibility Decision - Status: documented no-op for the current loop. - Evidence: - `ts/packages/workbench/src/atoms/workbench.ts` constructs `new BaseApi(...)`. - `ts/packages/client/src/__tests__/flows-api.test.ts` constructs `new FlowsApi(...)`, and sibling API facades expose the same constructor shape. - `EffectRpcClient` and `BaseApi` also preserve callable factory exports for compatibility with the vendored TrustGraph client shape. - Decision: - The remaining `newableFactory(... ) as unknown as NewableFactory<...>` assertions in client socket files are TypeScript compatibility boundaries, not Effect error/requirement channel assertions and not replacements for an Effect primitive. - Removing them safely requires a deliberate public API redesign or explicit class implementations for every API facade, not a local Effect-native rewrite. - Verification: - Current client/root verification from the tagged error slice covers this no-op decision. ### 2026-06-02: Service Entrypoint Runtime Slice - Status: migrated and root-verified. - Completed: - Remaining flow service `run(): Promise` program facades now use `ManagedRuntime.make(Layer.empty)` instead of direct `Effect.runPromise(program)`. - Remaining flow service modules now expose `runMain()` through `NodeRuntime.runMain(program)`. - Local `ts/scripts/run-*` launchers for gateway, prompt, chunker, extractor, PDF decoder, embeddings, triples, graph/document embeddings, text-completion providers, and MCP tool service now delegate directly to `runMain()`. - Direct `Effect.runPromise(program)` matches in `ts/packages/flow/src` are clean. Remaining `Effect.runPromise` matches are callback/Promise compatibility boundaries for later slices. - Verification: - `bun run --cwd ts/packages/flow build` - `git diff --check` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Base Processor Compatibility Runtime Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/processor/async-processor.ts` now uses a `ManagedRuntime` for Promise compatibility methods, signal-shutdown execution, and legacy `AsyncProcessor.launch`. - `ts/packages/base/src/processor/flow.ts` now owns a per-flow `ManagedRuntime` for `start`, `stop`, `runInCompatibilityScope`, and Promise resource facades. - `ts/packages/base/src/processor/flow-processor.ts` now uses a `ManagedRuntime` for the public `start(context)` facade instead of a local `Effect.runPromiseWith` runner. - `ts/packages/base/src/spec/parameter-spec.ts` now routes legacy `add` through `flow.runInCompatibilityScope(...)`, matching the other specs. - Subagent checks confirmed `NodeRuntime` is process-entrypoint-only here; `@trustgraph/base` should not add an `@effect/platform-node` dependency for these compatibility facades. - Remaining: - Constructor `as unknown as` shims in base processors preserve callable-plus-newable public exports and are compatibility boundaries for this loop. - Typed string lookup casts in `Flow` need a real typed-spec/key redesign; `HashMap`/`MutableHashMap` alone cannot infer `T` from a bare string. - Verification: - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `git diff --check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Base Flow Definition Schema Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/processor/flow-processor.ts` now validates `config.flows` with Effect Schema instead of local `Predicate`/object/string-record guards. - Invalid flow definition payloads still log/skip and preserve the existing config-handler and acknowledgement behavior. - `ts/packages/base/src/__tests__/flow-processor-runtime.test.ts` now covers an invalid nested flow definition that is acknowledged without starting resources. - Verification: - `bun run --cwd ts/packages/base test -- src/__tests__/flow-processor-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Text Completion Stream Sentinel Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/model/text-completion/{ollama,openai,mistral,azure-openai,claude,openai-compatible}.ts` now return the `Stream.unfold` end sentinel with `Effect.as(Effect.void, undefined)`. - Removed six `Effect.void as Effect.Effect` assertions without replacing them with `Effect.succeed(undefined)`, which `@effect/tsgo` flags as a diagnostic. - Verification: - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Text Completion Generator Boundary Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/model/text-completion/common.ts` now rejects fallback `AsyncGenerator.throw(...)` calls with the mapped tagged provider error directly instead of running `Effect.fail(...)` through `Effect.runPromise`. - The custom generator object no longer uses `as AsyncGenerator`, `as Promise>`, or `as LlmChunk` assertions. - Added a focused unit test for fallback throw mapping. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Text Completion Provider Status Narrowing Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/model/text-completion/common.ts` now uses `effect/Predicate` narrowing for provider `status` / `statusCode` inspection instead of local record assertions. - `ts/packages/flow/src/__tests__/text-completion-common.test.ts` covers both rate-limit status fields. - Verification: - `bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Base Parameter Spec Accessor Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/spec/parameter-spec.ts` now models `ParameterSpec` with an Effect Schema codec. Legacy parameter specs default to `S.Unknown`, preserving name-based registration while making typed access schema-backed. - `ts/packages/base/src/processor/flow.ts` now exposes `flow.parameterEffect(spec)` and `flow.parameter(spec)` for inferred, Schema-decoded parameter values. String lookup remains available as an `unknown` compatibility escape instead of a caller-chosen generic type. - Parameter schema failures now fail with the tagged `FlowParameterDecodeError` rather than a normal `Error`. - `ts/packages/flow/src/chunking/service.ts` now declares numeric chunk parameters once and retrieves them through the typed spec-object accessor. - `ts/packages/base/src/__tests__/flow-spec-runtime.test.ts` covers typed parameter decoding, legacy string lookup, missing parameter errors, sync accessor decoding, and schema mismatch errors. - Remaining: - Add typed spec-object accessors for producers and requestors so call sites can stop spelling generic string lookups for those registries too. - Verification: - `bun run --cwd ts/packages/base test -- src/__tests__/flow-spec-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/base test` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Base Producer And Requestor Spec Accessor Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/spec/producer-spec.ts` now exposes `ProducerSpec.producerEffect(flow)` and stores typed producer handles in a per-spec WeakMap keyed by `Flow`. - `ts/packages/base/src/spec/request-response-spec.ts` now exposes `RequestResponseSpec.requestorEffect(flow)` and stores typed requestor handles in a per-spec WeakMap keyed by `Flow`. - Spec finalizers remove only the exact handle they registered, avoiding stale finalizers deleting newer registrations for the same flow/spec pair. - `ts/packages/base/src/processor/flow.ts` now supports `flow.producerEffect(spec)`, `flow.requestorEffect(spec)`, `flow.producer(spec)`, and `flow.requestor(spec)` while keeping string accessors as untyped compatibility escapes. - Base service adapters and flow service handlers now reuse the same hoisted producer/requestor spec object in their spec arrays and handler lookups. - `ts/packages/base/src/__tests__/flow-spec-runtime.test.ts` covers typed spec-object lookups, duplicate spec identity failures, and scoped finalizer cleanup for producer and requestor handles. - Remaining: - Bare string `Flow` producer/requestor accessors remain compatibility escapes for external/legacy callers, but new Effect service code should use spec objects. - Verification: - `bun run --cwd ts/packages/base test -- src/__tests__/flow-spec-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/base test` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Native PubSub Boundary Slice - Status: migrated and package-verified. - Completed: - Confirmed Effect's native `PubSub` module is an in-process asynchronous hub with scoped subscriptions, not a NATS/Pulsar-compatible broker boundary. - Kept TrustGraph's `PubSubBackend` and `PubSub` service as the broker adapter layer because it owns topics, broker producers/consumers, acknowledgement, schema codecs, and backend lifecycle. - Removed the unused legacy `ts/packages/base/src/messaging/subscriber.ts` implementation, which duplicated in-process async queue/fanout behavior. - Removed the corresponding `makeAsyncQueue`, `makeSubscriber`, `Subscriber`, and `AsyncQueue` barrel exports from `ts/packages/base/src/messaging/index.ts`. - Remaining: - Future in-process fanout or request-streaming code should use `effect/PubSub`, `Queue`, `Stream.fromPubSub`, or `Channel.fromPubSub` rather than adding another local async queue implementation. - Do not replace `PubSubBackend` with `effect/PubSub` unless the code path is explicitly local-only and does not need broker semantics. - Verification: - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` ### 2026-06-02: Gateway Streaming Callback Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/gateway/dispatch/manager.ts` now exposes `dispatchGlobalServiceStreamingEffect` and `dispatchFlowServiceStreamingEffect` so Effect callers can handle stream chunks without Promise callback re-entry. - The existing Promise-returning streaming methods remain as compatibility facades and wrap responders with `Effect.tryPromise`. - `ts/packages/flow/src/gateway/rpc-server.ts` now writes stream chunks into the RPC queue through the dispatcher Effect path, removing the prior `Effect.context` plus `Effect.runPromiseWith(context)` bridge. - `ts/packages/client/src/socket/effect-rpc-client.ts` now uses `Stream.runForEachWhile` for early stream termination instead of throwing a synthetic `StopStreaming` tagged error. - Gateway dispatcher tests now exercise both the Promise compatibility streaming path and the Effect-native responder path. - Remaining: - `ts/packages/flow/src/gateway/rpc-protocol.ts` remains a Fastify socket compatibility bridge, not a direct replacement target for Effect RPC server layers yet. Local parser validation and collection cleanup inside that bridge remain valid migration targets. - Verification: - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/client build` - `bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts` - `bunx --bun vitest run src/__tests__/rpc-timeout.test.ts` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client Streaming Facade Normalization Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/trustgraph-socket.ts` now decodes the legacy streaming envelope with Schema before service-specific callback handling. - Streaming payload reads now use `effect/Predicate` property narrowing helpers instead of repeated response-wrapper assertions. - Graph RAG, document RAG, text completion, prompt, agent, and document stream callbacks now use a shared `streamComplete(...)` helper. The RPC `DispatchStreamChunk.complete` bit is the default transport completion source, with legacy service markers preserved for public compatibility. - Explainability triples are decoded through a recursive Schema instead of `as Triple[]`. - The focused client test now proves normalized `DispatchStreamChunk` completion flows through `graphRagStreaming` and final metadata. - Verification: - `bunx --bun vitest run src/__tests__/rpc-timeout.test.ts` - `bun run --cwd ts/packages/client build` - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/client test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Ollama Embeddings Effectful Layer Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/embeddings/ollama.ts` now exposes `makeOllamaEmbeddingsEffect` for effectful config loading and service construction. - `OllamaEmbeddingsLive` now uses `Layer.effect` and maps config/load failures into `EmbeddingsError` instead of preconstructing the service with `Layer.succeed`. - The direct `makeOllamaEmbeddings(config)` factory remains as a compatibility facade, while the canonical `program` entrypoint preserves the provider tagged error channel. - Ollama response JSON parsing no longer uses a Promise type assertion. - The focused embeddings tests now cover both direct factory use and the effectful `OllamaEmbeddingsLive` layer. - Verification: - `bunx --bun vitest run src/__tests__/ollama-embeddings.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: FalkorDB Scoped Client Lifecycle Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/storage/triples/falkordb.ts` and `ts/packages/flow/src/query/triples/falkordb.ts` now model FalkorDB client acquisition with `Effect.acquireRelease`. - FalkorDB Live layers now use `Layer.effect` and own Redis client disconnect finalizers through the layer scope. - Direct Promise compatibility factories and direct service factories now bracket each operation with scoped acquisition instead of hiding mutable `Effect.cached` connection slots. - Legacy `makeTriplesStoreService` and `makeTriplesQueryService` provider hooks now acquire scoped FalkorDB services and map acquisition failures to `ProcessorLifecycleError`; modern `program` entrypoints preserve the FalkorDB tagged layer error type. - FalkorDB query row field extraction now uses `effect/Predicate` narrowing instead of record/string type assertions. - New lifecycle tests use fake clients/graphs to prove connect on acquire and disconnect on scope close for both triples store and triples query. - Verification: - `bunx --bun vitest run src/__tests__/falkordb-lifecycle.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Qdrant Config, Schema, And Fakeable Construction Slice - Status: migrated and root-verified. - Completed: - Added `ts/packages/flow/src/qdrant/client.ts` as the narrow fakeable Qdrant surface used by graph/doc embedding store/query modules. - Graph and document Qdrant store/query constructors now create clients through `Effect.try`, load Qdrant config in Effect, and map config/client failures into their existing `S.TaggedErrorClass` errors. - Graph and document query payload extraction now uses `Schema.decodeUnknownEffect(...).pipe(Effect.option)` and skips malformed Qdrant payloads without type assertions. - Qdrant graph/doc query Live layers and graph store Live layer now use `Layer.effect` instead of preconstructing services with `Layer.succeed`. - Legacy graph store/query/doc query processor providers now acquire Qdrant services with named `Effect.fn` providers and map startup failures to `ProcessorLifecycleError`. - The installed Qdrant client still has no public close/disconnect method, so no `Effect.acquireRelease` finalizer was added for Qdrant. - Verification: - `bunx --bun vitest run src/__tests__/qdrant-embeddings.test.ts` - `bun run --cwd ts/packages/flow build` - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Text Completion Provider Stream Helper Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/model/text-completion/common.ts` now exposes `streamTextCompletionChunks`, an Effect-native helper built on `Stream.fromAsyncIterable`, `Ref`, `Stream.filterMap`/`Result`, and a final token chunk append. - OpenAI, Azure OpenAI, OpenAI-compatible, Mistral, Ollama, and Claude streaming providers now share the helper instead of each hand-rolling `Stream.unfold` plus `iterator.next` loops. - Mistral non-streaming and streaming content normalization now uses `effect/Predicate` and `Option` narrowing through `textFromContent`, removing the prior `content as string` assertion. - The helper uses the installed Effect beta's `Option.fromNullishOr` and Result-shaped `Stream.filterMap` API, verified by `check:tsgo`. - `ts/packages/flow/src/__tests__/text-completion-common.test.ts` covers token accumulation/final chunk emission and non-string content narrowing. - Remaining: - Full Effect AI provider swaps still need parity tests first; current OpenAI and Azure behavior is Chat Completions based, no installed Azure/Mistral/Ollama Effect AI provider exists, and Anthropic needs explicit text/token/streaming/rate-limit parity coverage before replacement. - Verification: - `bunx --bun vitest run src/__tests__/text-completion-common.test.ts` - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Request-Response Queue Stream Slice - Status: migrated and root-verified. - Completed: - `ts/packages/base/src/messaging/runtime.ts` now waits for accepted request-response replies by converting the response `Queue` to a `Stream.fromQueue`. - Recipient filtering now uses `Stream.filterMapEffect` with `Result` to skip partial responses until the recipient returns `true`. - `Stream.runHead` replaces the prior `while (true)`/`Queue.take` loop and preserves the existing timeout behavior around request-response calls. - `ts/packages/base/src/__tests__/messaging-runtime.test.ts` now covers recipient filtering across partial and final responses. - Remaining: - The two remaining production `while (` matches are `agent/react/parser.ts` line-buffer parsing and `cli/src/commands/util.ts` Commander parent traversal; neither is async polling or resource ownership. - Verification: - `bunx --bun vitest run src/__tests__/messaging-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Gateway RPC WebSocket Cause Handling Slice - Status: migrated and root-verified. - Completed: - `ts/packages/flow/src/gateway/server.ts` now handles RPC WebSocket program defects and interruptions inside the Effect pipeline with `Effect.sandbox`, `Effect.catch`, and `Cause.pretty`. - The previous Promise `.catch(...)` around `Effect.runPromise(...)`, plus the nested `Effect.runPromise` used only for logging and socket close, is removed. - The outer `Effect.runPromise` remains as the Fastify WebSocket host boundary. - Verification: - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client RPC Acquisition Cause Tap Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/effect-rpc-client.ts` now observes runtime/client acquisition failures with `Effect.tapCause` and `Cause.pretty`. - Removed the Promise `.catch(...)` that only updated local connection state after `runtime.runPromise(TrustGraphRpcClientService)`. - Removed the local `errorMessage` helper and its message-field assertion. - Public `dispatch`, `dispatchStream`, and `close` Promise facades remain compatibility boundaries. - Verification: - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/client build` - `bun run --cwd ts/packages/client test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client Socket Close Effect Boundary Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/trustgraph-socket.ts` now wraps `rpc.close()` with `Effect.tryPromise` inside the public `close(): void` facade. - Close failures are mapped to the existing tagged `TrustGraphSocketError` shape and logged through `Effect.catch` instead of a Promise `.catch`. - The remaining client socket Promise `.catch` matches were streaming callback compatibility bridges and are now handled by the follow-up slice. - Verification: - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/client build` - `bun run --cwd ts/packages/client test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Client Streaming Callback Effect Boundary Slice - Status: migrated and root-verified. - Completed: - `ts/packages/client/src/socket/trustgraph-socket.ts` now routes the legacy `agent`, `graphRagStreaming`, and `documentRagStreaming` callback request failures through `runLegacyStreamingRequest`. - `runLegacyStreamingRequest` uses `Effect.tryPromise` to map failures into tagged `TrustGraphSocketError` values, then uses `Effect.catch` to invoke the public legacy error callback. - Production `trustgraph-socket.ts` no longer has Promise `.catch` matches; remaining matches in that file are `Effect.catch` only. - Rechecked the PubSub replacement question against Effect v4 source: Effect's native `PubSub` is an in-process async hub over Effect queues. TrustGraph's `PubSubBackend` remains the broker adapter boundary for NATS/Pulsar-style topics, subscriptions, acknowledgement, schema codecs, and backend lifecycle. - Verification: - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/client build` - `bun run --cwd ts/packages/client test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Text Completion Provider Effectful Layer Slice - Status: migrated and root-verified. - Completed: - Added shared `makeTextCompletionLayer` for constructing `Llm` from an effectful `LlmProvider`. - Added `makeOpenAIProviderEffect`, `makeOpenAICompatibleProviderEffect`, `makeAzureOpenAIProviderEffect`, `makeClaudeProviderEffect`, `makeMistralProviderEffect`, and `makeOllamaProviderEffect`. - Processor `program.layer` definitions now use `Layer.effect` via the shared helper instead of constructing providers inside `Layer.succeed`. - Provider object assembly is split into pure `makeXProviderFromClient` helpers so Promise-returning provider methods remain external compatibility facades and do not trigger `effect(runEffectInsideEffect)`. - Added tests for explicit provider config, shared `Llm` layer provisioning, and tagged missing-config errors. - Verification: - `bunx --bun vitest run src/__tests__/text-completion-providers.test.ts src/__tests__/text-completion-common.test.ts` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check:tsgo` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Gateway Dispatcher Ownership And Serialization Slice - Status: migrated and root-verified. - Completed: - `makeDispatcherManager` now tracks whether it owns the pubsub backend and no longer closes injected `PubSubBackend` instances on `stop()`. - `publishToTopic` now uses `Effect.acquireUseRelease` so the one-shot producer is closed even when `send` fails. - Gateway dispatch paths now call `translateRequestEffect` and `translateResponseEffect`, which wrap serialization with `Effect.try` and return tagged `DispatchSerializationError` failures. - Streaming dispatch recipients are named `Effect.fn` callbacks, satisfying strict Effect diagnostics while preserving responder behavior. - Tests cover injected backend ownership, typed serialization failure before requestor startup, and producer close on send failure. - Verification: - `bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check:tsgo` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Base Producer Scoped Runtime Slice - Status: migrated and root-verified. - Completed: - Kept `PubSubBackend` as the broker adapter boundary; Effect native `PubSub` remains an in-process primitive and is not a replacement for broker-backed topics, subscriptions, acknowledgements, codecs, or backend lifecycle. - Reworked `makeProducer` so the legacy Promise facade allocates producers through `makeEffectProducerFromPubSub` inside a closeable `Scope`. - `stop()` now flushes the Effect producer and closes the scope with the registered producer finalizer, including the flush-failure path. - Added focused producer facade coverage for send routing, idempotent stop, tagged not-started lifecycle errors, and close-on-flush-failure behavior. - Verification: - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/base build` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/producer.test.ts` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: NATS Typed Boundary Slice - Status: migrated and root-verified. - Completed: - Replaced the dynamic header import inside `makeNatsProducer` with the static NATS `headers` export. - Wrapped publish header construction in `Effect.try`, so invalid header names/values fail as tagged `PubSubError` values instead of defects. - Wrapped NATS `ack()` and `nak()` calls in `Effect.try`, preserving the existing wrong-message guard and mapping thrown acknowledgement failures into tagged `PubSubError`s. - Added a mocked NATS backend test covering invalid publish headers and thrown ack/nak failures through the public `makeNatsBackend` path. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/nats-backend.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: NATS Selective 404 Slice - Status: migrated and root-verified. - Completed: - Added an internal `NatsLookupError` tagged error to preserve lookup causes without leaving `unknown` in Effect error channels. - Stream creation now happens only when `manager.streams.info()` fails with a NATS JetStream 404/missing-resource error. - Durable consumer creation now happens only when `js.consumers.get()` fails with a NATS JetStream 404/missing-resource error. - Added mocked NATS tests proving 404 lookups create resources while permission-style lookup failures do not. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/nats-backend.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Consumer Rate-Limit Retry Slice - Status: migrated and root-verified. - Completed: - Added `rateLimitTimeoutMs` to the Effect-native messaging runtime config, backed by `TG_RATE_LIMIT_TIMEOUT_MS` and the Python-compatible default of `7_200_000ms`. - Reworked legacy `makeConsumer` retry handling to use `Schedule.spaced`, retry repeated `TooManyRequestsError`s, and fail with a tagged `MessagingTimeoutError` when the rate-limit timeout elapses. - Reworked `makeEffectConsumerFromPubSub` handler retry handling with the same schedule/timeout behavior while keeping handler failures in typed Effect error channels. - Added legacy and Effect-native tests for repeated rate-limit retry until success and negative acknowledgement after retry timeout. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/consumer.test.ts src/__tests__/messaging-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Consumer Concurrency Ownership Slice - Status: migrated and root-verified. - Completed: - `makeEffectConsumerFromPubSub` now creates one backend consumer per concurrency worker rather than sharing a single backend consumer across parallel receive loops. - Consumer runtime `stop` is idempotent via `Ref.getAndSet`, so explicit `consumer.stop` and scope finalization do not double-close worker handles. - Added Effect-native runtime coverage proving `concurrency: 3` creates and closes three independent backend consumers exactly once. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Request-Response Stop Signal Slice - Status: migrated and root-verified. - Completed: - `makeEffectRequestResponseFromPubSub` now owns a `Deferred` stop signal for the lifetime of the scoped request-response runtime. - `request()` races response waiting against that stop signal before applying the request timeout, so pending calls fail promptly when the runtime stops. - `stop()` fails the stop signal with a tagged `MessagingLifecycleError` before interrupting the dispatch loop and closing the producer/consumer resources. - Flow PDF decoder and graph embeddings service error unions now include `MessagingLifecycleError` because requestor failures can surface shutdown. - Added Effect-native runtime coverage proving a pending request fails with the tagged lifecycle error when the request-response runtime stops. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `bun run --cwd ts/packages/flow build` - `bun run --cwd ts/packages/flow test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` ### 2026-06-02: Legacy Consumer Facade Slice - Status: migrated and root-verified. - Completed: - `makeConsumer` is now a Promise compatibility facade over `makeEffectConsumerFromPubSub` instead of owning a separate mutable backend, `running` flag, retry loop, and direct `BackendConsumer.receive`. - The facade uses a module `ManagedRuntime`, `Scope.make`, `Scope.provide`, and `Scope.close` to keep public `start()`/`stop()` Promises at the boundary while the actual consumer lifetime stays scoped. - Legacy Promise handlers are adapted with `Effect.tryPromise` and preserve `TooManyRequestsError` as a typed retry signal. - `makeEffectConsumerFromPubSub` now forks workers with `Effect.forkScoped`, so a caller-owned scope keeps workers alive until stop/finalization. - `consumer.test.ts` no longer encodes `start()` as the blocking consume-loop join; it waits for observable handler/ack/nak effects and then stops the scoped consumer. - Verification: - `cd ts/packages/base && bunx --bun vitest run src/__tests__/consumer.test.ts` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts src/__tests__/flow-spec-runtime.test.ts` - `cd ts && bun run check:tsgo` - `bun run --cwd ts/packages/base build` - `bun run --cwd ts/packages/base test` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `git diff --check` ### 2026-06-02: Workbench Theme KeyValueStore Slice - Status: migrated and root-verified. - Completed: - `themeClassAtom` no longer writes the legacy `tg-theme` localStorage mirror. Theme persistence stays in the canonical `trustgraph-workbench-theme-v1` `Atom.kvs` entry backed by `BrowserKeyValueStore.layerLocalStorage`. - The pre-paint host script in `index.html` now restores the canonical JSON-encoded theme key before React mounts, with `tg-theme` retained only as a legacy fallback. - Workbench QA now asserts that changing theme writes the canonical key and does not recreate `tg-theme`. - Verification: - `cd ts && bun run check` - `cd ts && bun run workbench:qa` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` ### 2026-06-02: Effect AI LanguageModel Adapter Slice - Status: migrated and package-verified. - Completed: - Added `makeLanguageModelProvider`, a bridge from `effect/unstable/ai/LanguageModel` into the existing TrustGraph `LlmProvider` contract. - Covered non-streaming text/token mapping, streaming text/final-token mapping, and Effect AI rate/quota failure mapping with fake `LanguageModel` tests. - Kept concrete provider swaps deferred until provider-specific parity is proven. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/text-completion-common.test.ts src/__tests__/text-completion-providers.test.ts` ### 2026-06-02: Native Request/Response PubSub Fanout Slice - Status: migrated and package-verified. - Completed: - Replaced the request/response runtime's hand-managed `Map` response fanout with native `effect/PubSub`. - Each request subscribes before sending, consumes through `Stream.fromSubscription`, filters by response id, and releases the subscription at scope exit. - Kept `PubSubBackend` as the broker boundary because Effect native PubSub is in-process only and does not provide NATS topics, ack/nack, durable subscriptions, schema codecs, or backend lifecycle. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts src/__tests__/request-response.test.ts src/__tests__/flow-spec-runtime.test.ts` ### 2026-06-02: Claude Effect AI Provider Slice - Status: migrated and package-verified. - Completed: - Replaced the direct `@anthropic-ai/sdk` provider implementation with `@effect/ai-anthropic` `AnthropicLanguageModel` plus the shared `makeLanguageModelProvider` adapter. - Preserved `CLAUDE_KEY`, default model, temperature, max output, processor, and public `LlmProvider` compatibility behavior. - Added Claude provider config coverage for `CLAUDE_KEY` env fallback and the missing-key tagged config error. - Removed the direct `@anthropic-ai/sdk` dependency and its lockfile entries from `@trustgraph/flow`. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/text-completion-providers.test.ts src/__tests__/text-completion-common.test.ts` - `cd ts/packages/flow && bun run build` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: RPC Dispatch Tagged Error Slice - Status: migrated and package-verified. - Completed: - Replaced the remaining production `S.ErrorClass` usage in the Flow gateway RPC contract with `S.TaggedErrorClass`. - Normalized the client-side RPC `DispatchError` counterpart to the same tagged-error schema shape so both wire contract copies stay aligned. - Closed the scratch-note `S.ErrorClass` finding for production code. The remaining plain `new Error` matches are test-only helpers or external host-boundary simulations. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts` - `cd ts/packages/client && bunx --bun vitest run src/__tests__/rpc-timeout.test.ts` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Effect Metrics Prometheus Slice - Status: migrated and root-verified. - Completed: - Replaced the `@trustgraph/base` `prom-client` metric wrappers with Effect-native `Metric.counter`, `Metric.histogram`, and `effect/unstable/observability` `PrometheusMetrics.format`. - Kept the existing Prometheus metric names and gateway `/api/v1/metrics` scrape boundary while removing the direct `prom-client` dependency and lockfile entries. - Changed producer metric recording from a sync callback to an Effect value that runs inside the producer send pipeline. - Added isolated metric-registry tests for producer and consumer Prometheus formatting. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/metrics-effect.test.ts src/__tests__/producer.test.ts src/__tests__/messaging-runtime.test.ts` - `cd ts/packages/base && bun run build` - `cd ts/packages/base && bun run test` - `cd ts/packages/flow && bun run build` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts` - `cd ts && bun run check` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: MCP Effect Stdio Entrypoint Slice - Status: migrated and root-verified. - Completed: - Added an Effect-native stdio layer and process entrypoint with `McpServer.layerStdio`, `NodeStdio.layer`, and `NodeRuntime.runMain`. - Reused the same `TrustGraphMcpToolkitLive` path for HTTP and stdio through a shared toolkit-layer helper. - Kept the legacy SDK/Zod stdio export as a compatibility surface until protocol-level `tools/list` and `tools/call` parity tests prove it can be flipped or removed. - Added focused coverage that the Effect toolkit names remain stable and the stdio layer/entrypoint are exported. - Updated the MCP test script to ignore compiled `dist/**` output so root builds do not cause duplicate Vitest runs from generated tests. - Scratch-note triage: - Metrics, in-process PubSub fanout, Claude Effect AI, RPC `S.TaggedErrorClass`, and `@effect/tsgo` setup are already migrated. - Remaining valid scratch targets are MCP protocol parity/flip, Duration config cleanup, Term/ClientTerm tagged-union matching, service `Effect.fn` normalization, `@effect/cli`, stream/RPC follow-ups, chunking `Chunk`, cores Promise APIs, and long-lived `Map`/`Set` state. - Verification: - `cd ts/packages/mcp && bun run test` - `cd ts/packages/mcp && bun run build` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Term And ClientTerm Match Slice - Status: migrated and package-verified. - Completed: - `ts/packages/base/src/schema/primitives.ts` now exposes `Term` as a recursive `S.toTaggedUnion("type")` schema and aligns `Triple.g` with the Python/client wire contract as an optional graph string. - `ts/packages/flow/src/gateway/dispatch/serialize.ts` now defines compact client-term schemas with `S.tag`, decodes unknown term-shaped values with Schema/Option, and translates terms with `Match.discriminatorsExhaustive`. - Removed the gateway serializer's native term switches and unsafe pass-through casts. Malformed known-tag objects now stay ordinary payload objects during deep translation instead of being cast into invalid terms. - Replaced pure term helper switches in FalkorDB triples store/query, Qdrant graph embeddings store, Graph RAG, agent tools, and workbench graph utilities with exhaustive `Match` discriminators. - Added tests for named graph string decoding, nested compact/internal term round trips, and malformed known-tag payload preservation. - Remaining: - The client socket streaming term schema is still a local recursive union and can be centralized later if drift appears. It has no native term switch in the current scan. - Operation dispatch switches in config, cores, librarian, and flow-manager are separate service-command refactors, not part of this term wire slice. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/schema-effect.test.ts` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts src/__tests__/falkordb-lifecycle.test.ts src/__tests__/qdrant-embeddings.test.ts src/__tests__/retrieval-rag.test.ts` - `cd ts/packages/flow && bun run test` - `cd ts/packages/base && bun run build` - `cd ts/packages/flow && bun run build` - `cd ts/packages/workbench && bun run build` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Messaging Runtime Duration Config Slice - Status: migrated and package-verified. - Completed: - `ts/packages/base/src/runtime/messaging-config.ts` now stores internal runtime timing fields as `Duration.Duration` instead of number-shaped millisecond fields. - `loadMessagingRuntimeConfig()` reads `TG_*` timing env vars with `Config.duration` while preserving legacy bare-number millisecond env values through a `Config.number(...).map(Duration.millis)` fallback. - `ts/packages/base/src/messaging/runtime.ts` now uses `Duration` values directly for sleeps, retry schedules, and timeout options. It converts back to milliseconds only at the broker `receive(timeoutMs)` boundary and for existing timeout error payloads. - Public compatibility options such as `receiveTimeoutMs`, `rateLimitRetryMs`, `rateLimitTimeoutMs`, and request `timeoutMs` remain numeric millisecond inputs. - Tests now cover both legacy numeric env values and native Effect duration strings. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/base && bunx --bun vitest run src/__tests__/schema-effect.test.ts src/__tests__/messaging-runtime.test.ts src/__tests__/consumer.test.ts` - `cd ts/packages/base && bun run build` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-02: Qdrant MutableHashSet Cache Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/storage/embeddings/qdrant-graph.ts` now uses `MutableHashSet` for its known-collection cache. - `ts/packages/flow/src/storage/embeddings/qdrant-doc.ts` now uses `MutableHashSet` inside its store effect shape instead of a native `Set`. - Collection cache membership, insertion, and invalidation now use `MutableHashSet.has`, `MutableHashSet.add`, and `MutableHashSet.remove`. - Qdrant tests now prove the graph store service avoids repeated collection existence checks while the collection is cached and checks again after delete invalidates the cache. - Remaining: - The document store's public direct facade still constructs a fresh store effect per operation, so its cache lifetime is per constructed effect shape. Changing that facade to own a shared runtime/service is a separate lifecycle design slice, not part of this collection primitive migration. - Verification: - `cd ts && bun run check:tsgo` - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/qdrant-embeddings.test.ts` - `cd ts/packages/flow && bun run build` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Workbench Match Dispatch Slice - Status: migrated and root-verified. - Completed: - `ts/packages/workbench/src/qa/mock-api.ts` now dispatches mock RPC service requests with `effect/Match` and keeps the existing unknown-service `{}` fallback. - `ts/packages/workbench/src/atoms/workbench.ts` now dispatches chat submit behavior for `graph-rag`, `document-rag`, and `agent` through `Match.exhaustive` over the `ChatMode` union. - The strict scan for native `switch` statements in `ts/packages` is clean: `rg -n "\bswitch\s*\(" ts/packages --glob '*.ts' --glob '*.tsx'` returns no matches. - Verification: - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Request-Response Stop Ref Slice - Status: migrated and package-verified. - Completed: - `ts/packages/base/src/messaging/runtime.ts` now tracks `makeEffectRequestResponseFromPubSub` stop idempotence with `Ref` instead of local mutable state. - The request-response runtime now matches the consumer runtime stop pattern: `Ref.getAndSet` gates shutdown, signal failure, PubSub shutdown, fiber interruption, producer close, and backend consumer close. - `ts/packages/base/src/__tests__/messaging-runtime.test.ts` now asserts that explicit request-response stop plus scoped finalization closes the producer and response consumer exactly once. - Verification: - `cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts src/__tests__/request-response.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Config Service HashMap State Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/config/service.ts` now stores workspace and namespace config state in nested `HashMap` values inside the existing `SynchronizedRef`. - Put/delete operations now update immutable `HashMap` snapshots with `HashMap.set` and `HashMap.remove` instead of cloning and mutating native `Map` instances. - Persistence, config dump, list, and get-values handlers keep plain `Record`/array shapes only at API and JSON boundaries, with deterministic ordering applied while converting out of `HashMap`. - The focused scan for native map state in `config/service.ts` is clean; the remaining map matches in that file are `HashMap` and `SynchronizedRef` operations. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/config-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: KnowledgeCore Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/cores/service.ts` now defines its reusable generator helpers with `Effect.fn` or `Effect.fnUntraced` instead of arrow functions returning `Effect.gen`. - `sendResponse` and `consumeOnceEffect` use `Effect.fnUntraced` because they are small hot-path helper functions. - `readPersistedKnowledgeEffect`, `persistStateEffect`, `closeKnowledgeResourcesEffect`, and `runKnowledgeCoreServiceEffect` use named `Effect.fn` wrappers while preserving their existing catch/logging behavior. - The focused scan for `=> Effect.gen` in `cores/service.ts` is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/knowledge-core-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Config Service Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/config/service.ts` now defines its reusable generator helpers with `Effect.fn` or `Effect.fnUntraced` instead of arrow functions returning `Effect.gen`. - `consumeOnceEffect` uses `Effect.fnUntraced`; persistence, push, read, put/delete, close, and run helpers use named `Effect.fn` wrappers. - Existing persistence read/write catch/logging behavior is preserved through `Effect.fn` post-processing functions. - The focused scan for config-service helper `=> Effect.gen` patterns is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/config-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: NATS MutableHashSet Stream Cache Slice - Status: migrated and package-verified. - Completed: - `ts/packages/base/src/backend/nats.ts` now tracks initialized stream names in `MutableHashSet` instead of a native `Set`. - `ensureStream` uses `MutableHashSet.has` and `MutableHashSet.add` while preserving the NATS broker adapter boundary and the selective 404 create-on-missing behavior. - `ts/packages/base/src/__tests__/nats-backend.test.ts` now proves repeated producer/consumer creation for subjects in the same stream only performs one stream lookup. - Verification: - `cd ts/packages/base && bunx --bun vitest run src/__tests__/nats-backend.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: KnowledgeCore HashMap State Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/cores/service.ts` now stores knowledge-core and document-core state in `HashMap` inside the existing `SynchronizedRef`. - Put/delete/load/get operations now read and update immutable `HashMap` snapshots with `HashMap.get`, `HashMap.set`, `HashMap.remove`, and `HashMap.has`. - Persistence and list-response helpers convert `HashMap` state to deterministic sorted records/arrays only at the API/JSON boundaries. - `ts/packages/flow/src/__tests__/knowledge-core-service.test.ts` now reads service state through `HashMap.get` and `Option`. - The focused scan for native map state in `cores/service.ts` is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/knowledge-core-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Librarian Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/librarian/service.ts` now defines its reusable librarian generator helpers with `Effect.fn` or `Effect.fnUntraced` instead of arrow functions returning `Effect.gen`. - `consumeOnceEffect` and both per-message response senders use `Effect.fnUntraced` for small hot-path callbacks. - `runLibrarianServiceEffect`, `getDocumentMetadataEffect`, `listChildrenEffect`, `uploadChunkEffect`, `getUploadStatusEffect`, and `abortUploadEffect` use named `Effect.fn` wrappers while preserving their tagged-error behavior and Promise compatibility method surfaces. - The focused scan for librarian-service helper `=> Effect.gen` patterns is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/librarian-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: FalkorDB Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/query/triples/falkordb.ts` now defines reusable connection, pattern-match, and query helper programs with `Effect.fn` or `Effect.fnUntraced` instead of arrow functions returning `Effect.gen`. - `ts/packages/flow/src/storage/triples/falkordb.ts` now uses a named `Effect.fn` for its connection helper, matching the existing scoped acquisition/finalizer boundary. - The public Promise facades and scoped layer acquisition behavior are unchanged; this slice only removes repeated helper generator construction. - The focused scan for FalkorDB helper `=> Effect.gen` patterns is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/falkordb-lifecycle.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: FlowManager HashMap State Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/flow-manager/service.ts` now stores long-lived flow and blueprint state in immutable `HashMap` snapshots inside the existing `SynchronizedRef`. - Refresh, start, stop, delete, list, get, and config-push paths now use `HashMap.get`, `HashMap.has`, `HashMap.set`, `HashMap.remove`, and `HashMap.size` instead of cloned native `Map` state. - List responses and config-push iteration convert `HashMap` state through sorted entries only at the API/config boundary for deterministic output. - `ts/packages/flow/src/__tests__/flow-manager-service.test.ts` now reads service state through `HashMap.get` plus `Option` and keeps the test fake pubsub `Map` as a compatibility fixture. - The focused scan for native flow-manager map state is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/flow-manager-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Gateway RPC Protocol Decode Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/gateway/rpc-protocol.ts` no longer casts `parser.decode(data)` to `ReadonlyArray`. - The socket protocol now validates parser output through a local Schema for client-to-server `Request`, `Ack`, `Interrupt`, `Ping`, and `Eof` envelopes before calling the server protocol write hook. - Invalid client frames and parser failures now map to a tagged `RpcProtocolDecodeError` and are written back as encoded RPC defects. - The existing server-response defect encoding no longer uses a non-null assertion on `parser.encode`. - `ts/packages/flow/src/__tests__/gateway-rpc-protocol.test.ts` covers valid request header prepending, control-frame forwarding, and rejection of server response envelopes received from a client socket. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-rpc-protocol.test.ts` - `cd ts/packages/flow && bunx --bun vitest run` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Gateway RPC Protocol Mutable Collections Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/gateway/rpc-protocol.ts` now tracks active socket clients with `MutableHashMap` and `MutableHashSet` instead of native `Map` / `Set` closure state. - The protocol still returns a native `ReadonlySet` snapshot at the `RpcServer.Protocol.clientIds` API boundary because that is the Effect RPC protocol contract. - `ts/packages/flow/src/__tests__/gateway-rpc-protocol.test.ts` now covers server response sends through the registered client and the public `clientIds` snapshot. - The focused scan for native map/set state in `gateway/rpc-protocol.ts` is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-rpc-protocol.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Retrieval RAG Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/retrieval/document-rag.ts` now defines its reusable document RAG query program with `Effect.fn` instead of a function returning `Effect.gen`. - `ts/packages/flow/src/retrieval/graph-rag.ts` now defines the graph RAG query, concept extraction, vector lookup, entity lookup, edge traversal, edge scoring, and synthesis helpers with named `Effect.fn` providers. - Public Promise facades and requestor compatibility surfaces are unchanged. - The focused scan for retrieval `return Effect.gen(...)` helper patterns is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/retrieval-rag.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Ollama Embeddings Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/embeddings/ollama.ts` now defines the `embed` service method as a named `Effect.fn` generator instead of returning a nested `Effect.gen` from an `Effect.fn` callback. - Empty embedding requests now return through the same generator path without allocating a separate `Effect.succeed` helper. - The public sync factory, effectful layer, `ManagedRuntime` facade, and `NodeRuntime.runMain` entrypoint are unchanged. - The focused scan for Ollama `return Effect.gen(...)` helper patterns is clean. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/ollama-embeddings.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Base Processor Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/base/src/processor/flow.ts` now exposes `startEffect` and `stopEffect` as Effect values with spans instead of lazy zero-argument Effect methods. - `Flow.runInCompatibilityScopeEffect` is now a named `Effect.fn` helper instead of a method returning nested `Effect.gen`. - The public `Flow` shape is explicit so Effect error and requirement channels do not widen through recursive object inference. - Producer, request/response, and parameter specs are generic over the receiving flow requirements, so reusable specs remain mountable in flows with additional service dependencies without type assertions. - `ts/packages/base/src/processor/flow-processor.ts` now defines `startFlow`, `configureFlows`, `processNextConfigPush`, the safe consume wrapper, and the Promise `start` compatibility helper with named `Effect.fn` functions. - The focused scan for callable `return Effect.gen(...)` helpers in `flow.ts` is clean. Remaining `flow-processor.ts` matches are one-shot program values / exported program factories and should be scoped separately from reusable helper normalization. - Verification: - `cd ts/packages/base && bunx --bun vitest run src/__tests__/flow-spec-runtime.test.ts src/__tests__/flow-processor-runtime.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Chunking Chunk Collection Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/chunking/recursive-splitter.ts` now returns `Chunk.Chunk` and uses `effect/Chunk` for splitter result, merge, recursive append, and overlap collections. - The chunking service behavior is unchanged; `Chunk` is iterable and still provides `length` for logging and output counting. - Splitter and service tests convert `Chunk` results to readonly arrays only at assertion boundaries. - The focused chunking scan no longer has array-backed splitter result state. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/recursive-splitter.test.ts src/__tests__/chunking-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Workbench Random ID Effect.fn Helper Slice - Status: migrated and package-verified. - Completed: - `ts/packages/workbench/src/atoms/workbench.ts` now defines the reusable random id helper with named `Effect.fn` instead of a function returning `Effect.gen`. - Existing notification, upload, chat request, and message id call sites are unchanged and still yield the helper directly. - The remaining workbench `Effect.gen` scan hit is the local `submitUploadDocument` one-shot effect value inside an existing `Effect.fn` command, not a reusable helper factory. - Verification: - `cd ts/packages/workbench && bun run build` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Librarian Collection Manager MutableHashMap Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/librarian/collection-manager.ts` now stores in-memory collection entries in `MutableHashMap` instead of native `Map`. - Public `listCollections`, `getCollection`, `updateCollection`, `deleteCollection`, `ensureCollectionExists`, `toJSON`, and `loadFromJSON` behavior stays array/boolean/undefined based at the API and persistence boundaries. - New focused coverage verifies create/update/list/get/delete and JSON restore behavior through the migrated collection state. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/collection-manager.test.ts src/__tests__/librarian-service.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Prompt Template MutableHashMap Cache Slice - Status: migrated and package-verified. - Completed: - `ts/packages/flow/src/prompt/template.ts` now stores loaded prompt templates in `MutableHashMap` instead of a native `Map`. - Config reload clears and repopulates the Effect collection, request lookup narrows through `Option`, and logging uses `MutableHashMap.size` / `keys`. - New focused coverage verifies a config-loaded prompt template renders through the service request/response flow. - Verification: - `cd ts/packages/flow && bunx --bun vitest run src/__tests__/prompt-template.test.ts` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ### 2026-06-04: Workbench Explain Triples MutableHashMap Slice - Status: migrated and package-verified. - Completed: - `ts/packages/workbench/src/atoms/workbench.ts` now stores the `explainTriplesAtom` input cache in `MutableHashMap` instead of a native `Map`. - Lookup narrows through `Option`, writes use `MutableHashMap.set`, and the adjacent `Triple[]` assertions were replaced with callback return types and inferred array copies. - Verification: - `cd ts && bun run --cwd packages/workbench build` - `cd ts && bun run check:tsgo` - `cd ts && bun run build` - `cd ts && bun run test` - `cd ts && bun run lint` - `git diff --check` ## Subagent Findings To Preserve - MCP/workbench: - Make the Effect MCP server the canonical implementation. An Effect `McpServer.layerStdio` entrypoint now exists; the old stdio server should remain only as compatibility until protocol-level `tools/list` and `tools/call` parity is proved. Do not delete `server.ts` until that parity coverage exists, with special attention to `text_completion` behavior. - Workbench BaseApi atoms can move toward `AtomRpc` or `AtomHttpApi` after the client API is less Promise-first. - MCP env is now Config-backed; continue that policy for future MCP settings. - Workbench persistent theme storage is now canonicalized through `Atom.kvs`/`BrowserKeyValueStore`. Remaining workbench `localStorage` matches are legacy migration fallbacks, QA assertions, or the pre-paint host script. - Flow stateful services: - Config service operation dispatch, schema persistence, and nested workspace state are complete: the long-lived config store now uses `HashMap` inside `SynchronizedRef`, and plain records remain only at persistence/API boundaries. - KnowledgeCore service operation dispatch, helper functions, and ref-backed core state are complete: `kgCores` and `deCores` now use `HashMap` inside `SynchronizedRef`, and plain records remain only at persistence/API boundaries. - FlowManager operation dispatch, helper functions, and ref-backed flow / blueprint state are complete: `flows` and `blueprints` now use `HashMap` inside `SynchronizedRef`, and plain records remain only at config/API boundaries. Librarian ref-backed state remains a larger collection target. Follow-up service work should focus on scoped layers, schedules where polling semantics allow, and managed persistence providers rather than direct mutable service fields. - Flow service startup facades now consistently use `ManagedRuntime`, and local scripts should delegate to `runMain()` instead of adding local `.catch(console.error/process.exit)` wrappers. - Base `Flow.startEffect` / `stopEffect` are now Effect values, not zero-argument lazy Effect methods. New runtime APIs should expose Effect values where no arguments are needed, and reserve `Effect.fn` for reusable helpers that take arguments. - Persistence IO should move toward `FileSystem` or `KeyValueStore` where the installed beta has the needed provider surface. - Base messaging/processors: - Processor/flow Promise compatibility now uses `ManagedRuntime`; keep `NodeRuntime` only for process `runMain()` entrypoints. - Subscriber queues/maps and dynamic flow state should continue moving toward `Queue`, `Deferred`, `SynchronizedRef`, `Schedule`, and scoped layers. - The legacy `messaging/subscriber.ts` async queue/fanout implementation is removed. Use native `effect/PubSub` for future in-process fanout, while keeping `PubSubBackend` for broker-backed messaging. - The legacy producer and consumer facades now delegate to scoped Effect runtime factories. Public `start()`/`send()`/`stop()` Promises remain compatibility boundaries. - NATS header construction, ack/nak operations, and lookup create-on-missing behavior now stay typed. `PubSubBackend` remains an intentional broker adapter contract rather than a direct `effect/PubSub` replacement target. - NATS initialized stream cache now uses Effect `MutableHashSet`; keep the rest of the NATS SDK connection/JetStream manager state as an external broker adapter boundary unless moving the whole backend lifecycle. - Consumer rate-limit retry timeout behavior is now wired in both legacy and Effect-native consumer paths. Effect-native consumer concurrency now owns one backend consumer per worker, and request-response pending shutdown now fails through a tagged lifecycle error. The legacy consumer facade blocking shape is complete; do not reopen it unless a public API compatibility issue appears. - Existing constructor shims preserve callable-plus-newable public exports; removing them needs a public API split or real class redesign. - Typed string registries in `Flow` now have Schema-backed parameter specs and typed producer/requestor spec-object accessors. New service handlers should hoist spec objects and use those accessors; bare string accessors remain compatibility escapes. - Base metrics are now Effect-native and Prometheus-formatted through `PrometheusMetrics.format`; do not reopen `prom-client` unless a future scrape requirement cannot be represented by Effect metrics. - The scratch note's `prom-client` claim is stale: `ts/packages` has no `prom-client` matches, and `metrics/prometheus.ts` already uses `effect/Metric` plus `PrometheusMetrics.format`. Remaining observability work is OTLP/span coverage and wiring existing consumer metrics helpers. - Numeric public timeout fields such as `timeoutMs` remain compatibility surfaces. Internal runtime config with `Config.number(...Ms)` is still a valid `Config.duration` / `Duration` cleanup target. - Gateway/client: - `EffectRpcClient` now owns its socket/RPC layer with `ManagedRuntime`. Socket errors/JSON parsing now use tagged errors and Schema decoding. The remaining client `newableFactory` assertions are documented as public API compatibility boundaries for this loop. - `EffectRpcClient` remains Promise-shaped at the public facade. A future client API slice can add an Effect/Stream-native surface beside the Promise facade, then migrate callback users incrementally. - Gateway/client `DispatchError` contracts now use `S.TaggedErrorClass`; do not reopen `S.ErrorClass` unless a new production match appears. - Gateway `DispatchStream` now uses Effect-native dispatcher streaming callbacks instead of nested `Effect.runPromiseWith`, and client streaming facade callbacks now decode the legacy envelope through Schema before applying service-specific public callback semantics. - Gateway dispatcher ownership and serialization cleanup is complete: injected pubsub backends are not closed by the manager, one-shot producers are acquire/use/release bracketed, and serialization failures are typed Effect errors. - `gateway/rpc-protocol.ts` remains a Fastify socket compatibility bridge while the public Effect RPC server layers require SocketServer or Effect HTTP routing. The parser decode assertion is complete through Schema validation; future local cleanups should focus on its connection collections rather than replacing the whole bridge. - WebSocket adapter host fallbacks now use `Result.try` and tagged adapter errors while preserving sync exports. - RAG/providers/storage: - RAG and agent requestor bridges are complete: `toPromiseRequestor` has no remaining `ts/packages` matches. - Provider SDKs and storage clients should become managed resources where they have meaningful lifecycle. - Ollama embeddings now has an effectful canonical layer. There is no installed Effect AI Ollama provider package, so future Ollama work should focus on local Effect wrappers/adapters rather than provider replacement. - Full text-completion provider swaps need parity tests first. OpenAI and Azure currently use Chat Completions while `@effect/ai-openai` is Responses API oriented, and no installed Azure/Mistral/Ollama Effect AI provider is available. Anthropic is the closest direct provider swap, but must preserve text, token counts, streaming final usage, and rate-limit mapping. The local provider layer-construction cleanup is complete; remaining provider work is adapter/parity work, not `Layer.succeed` cleanup. - The `effect/unstable/ai/LanguageModel` to TrustGraph `LlmProvider` adapter baseline is complete, and Claude now uses `@effect/ai-anthropic` through that adapter. Direct OpenAI, Azure, and OpenAI-compatible swaps are no-ops until Responses-vs-Chat-Completions parity is proven. - FalkorDB scoped lifecycle is complete for triples query/store. Use the fakeable client/graph factory pattern from that slice for future storage client tests. - Qdrant config/schema/fakeability is complete for graph/doc embedding store/query modules. Qdrant still has no close/disconnect surface in the installed client, so do not reopen it as an `acquireRelease` close slice without new SDK evidence. - Shared text-completion stream iteration and the Mistral content assertion are complete, and the Effect AI stream-part adapter now uses `effect/Match`. The remaining provider-layer item is parity-backed Effect AI adapter work, not a direct SDK swap. - AI service modernization remains partially valid: Claude already uses `@effect/ai-anthropic`, while OpenAI/Azure/OpenAI-compatible/Mistral/Ollama still need parity-backed Effect AI adapters or provider-specific bridges. - Scratch-note follow-ups: - `Term` / compact client term serialization is complete for base schema, gateway translation, and pure term helper switches. Future work should only reopen this if client socket schema drift appears or a hidden consumer needs a different named-graph shape. - Messaging runtime `Config.duration` cleanup is complete. Internal runtime config uses `Duration.Duration`; public timeout compatibility inputs and broker receive/error payload boundaries remain numeric milliseconds. - CLI modernization remains valid, but the live installed target is `effect/unstable/cli` rather than an installed `@effect/cli` package. - Chunking `effect/Chunk` migration is complete: the recursive splitter now returns `Chunk.Chunk` and converts to arrays only at test/assertion boundaries. - Knowledge core internals are largely Effect-native, but public core service facades still expose Promise methods; migrate tests to Effect-first methods before shrinking those facades. - Qdrant graph/doc known-collection caches now use `MutableHashSet`. Short-lived local traversal sets remain no-ops. - Gateway dispatcher static service registries, streaming membership, and scoped requestor cache now use Effect `HashMap`/`HashSet`; gateway term-bearing service membership sets now use Effect `HashSet` too. - FlowManager, KnowledgeCore, ConfigService, and Librarian `() => Effect.gen(...)` factories are normalized to `Effect.fn` / `Effect.fnUntraced`. - ConfigService and KnowledgeCore operation dispatch now use `effect/Match` with `Match.exhaustive`; FlowManager and Librarian operation dispatch now use `effect/Match` with runtime-preserving `Match.orElse` fallbacks. - ConfigService nested workspace config state now uses Effect `HashMap`. JSON and API response helpers intentionally convert back to sorted plain records/arrays at the boundary. - Native `switch` statements are now clean in `ts/packages`; future branch drift should keep service dispatch on `effect/Match` or Schema tagged-union helpers. - Client RPC/BaseApi connection-state fanout now uses `effect/SubscriptionRef`; remaining gateway/client P1 work is broader API design, not listener bookkeeping. - Long-lived `Map` / `Set` state in remaining ref-backed services can move toward Effect collections later; local pure traversal maps/sets remain no-ops. - Fresh strict signal sweep after the 2026-06-04 helper and collection slices found no production normal `Error`, raw `try`/`catch`, native `switch`, or Effect-focused type assertions under `ts/packages`. - Remaining real helper-normalization targets from the fresh sweep are separately scoped inline callback/program factories in messaging compatibility facades, gateway/librarian helpers, and CLI command actions. The workbench random id helper is complete; the remaining workbench `Effect.gen` match is a local one-shot command effect value. - Remaining real long-lived native collection targets include base processor registries and Librarian service state. The standalone Librarian collection manager, prompt template cache, and workbench explain triples module cache are complete. Local traversal sets and test fakes remain no-op boundaries. ## Ranked Findings ### No-op: Broker Backend Adapter Boundary - Status: - Closed as a P0 rewrite item after the 2026-06-02 broker scout and the legacy consumer facade slice. - TrustGraph evidence: - `ts/packages/base/src/backend/types.ts` - `ts/packages/base/src/backend/nats.ts` - `ts/packages/base/src/backend/pubsub.ts` - `ts/packages/base/src/messaging/runtime.ts` - Effect primitives: - `Layer`, `Scope`, `Stream`, `Schedule`, `Queue`, `Effect.acquireRelease`, and `Effect.tryPromise`. - Evidence: - `BackendProducer`, `BackendConsumer`, and `PubSubBackend` are the external Promise broker adapter contract. `backend/pubsub.ts` wraps that contract in Effect through `Context.Service`, `Layer`, `Effect.tryPromise`, and scoped finalizers. - NATS boundary failures, selective stream/consumer lookup recovery, producer sends, ack/nak, and close paths are typed with Effect wrappers. - Producer, consumer, and request-response runtime ownership now live in scoped Effect factories. - Rule: - Keep `PubSubBackend` as the compatibility adapter boundary; Effect native `PubSub` remains in-process only. - Treat the producer Promise facade as a completed compatibility wrapper; avoid reopening it unless backend runtime changes require a narrower adapter. - Keep NATS SDK boundary failures typed and avoid catch-all create-on-failure behavior. Future backend slices should move connection/stream state into scoped Effect services. - Treat rate-limit retry timeout semantics as complete; next consumer slices should focus on blocking compatibility and backend/layer ownership, not retry policy. - Treat Effect-native per-worker consumer ownership as complete; do not flag `makeEffectConsumerFromPubSub` concurrency for shared backend receive handles. - Treat request-response pending shutdown semantics as complete; do not flag `waitForResponse` timeout behavior for stopped runtimes. - Treat request-response stop idempotence as complete; stop state now uses an Effect `Ref` and explicit stop plus scoped finalizer closes resources once. - Treat request-response in-process fanout as complete: response routing now uses native `effect/PubSub` subscriptions instead of a hand-managed subscriber map. - Treat the legacy consumer facade as a completed compatibility wrapper over `makeEffectConsumerFromPubSub`; do not flag blocking `start()` semantics. ### No-op: Remaining Effect AI Provider Swaps - TrustGraph evidence: - `ts/packages/flow/src/model/text-completion/*.ts` - Effect primitives: - `effect/unstable/ai/LanguageModel`, `effect/unstable/ai/EmbeddingModel`, Effect AI OpenAI/Anthropic provider layers. - Rewrite shape: - Adapter baseline is complete: `makeLanguageModelProvider` bridges `LanguageModel` into `LlmProvider`. - Claude migration is complete through `@effect/ai-anthropic`. - Do not directly swap OpenAI, Azure, or OpenAI-compatible providers yet: current TrustGraph code uses Chat Completions/local-server semantics while `@effect/ai-openai` is Responses API backed. - Do not directly swap Mistral or Ollama until an installed Effect provider package exists or a parity-backed local Effect wrapper is designed. - Tests: - Provider parity for `LlmResult`, final streaming chunk token counts, 429 mapping, missing-token config failures, and OpenAI-compatible local-server behavior. ### No-op: Base Metrics Prometheus Wrapper - Status: - Closed as a scratch-note migration target by the Effect Metrics Prometheus slice. - TrustGraph evidence: - `ts/packages/base/src/metrics/prometheus.ts` - `ts/packages/flow/src/gateway/server.ts` - Effect primitives: - `Metric.counter`, `Metric.histogram`, and `effect/unstable/observability` `PrometheusMetrics.format`. - Rule: - Keep the gateway Fastify route as the external scrape boundary, but record TrustGraph metrics through Effect `Metric` values. - Use a fresh `Metric.MetricRegistry` in tests that assert exact scrape content. ### Complete: Term And ClientTerm Tagged-Union Normalization - TrustGraph evidence: - `ts/packages/base/src/schema/primitives.ts` - `ts/packages/flow/src/gateway/dispatch/serialize.ts` - `ts/packages/client/src/socket/trustgraph-socket.ts` - Effect primitives: - `S.toTaggedUnion(...).match` and `effect/Match` discriminator helpers. - Rewrite shape: - Add tagged-union helpers for internal `Term` and compact client terms. - Replace serializer native switches with tagged-union matching or `Match.discriminatorsExhaustive`. - Remove unsafe default pass-through casts while preserving compact `g` string compatibility. - Tests: - Base schema tests now cover recursive terms and graph strings. - Gateway dispatcher tests now cover all compact term variants, nested triples, compact graph strings, malformed known-tag payloads, and malformed client triple failures. ### Complete: Messaging Runtime Duration Config Cleanup - TrustGraph evidence: - `ts/packages/base/src/runtime/messaging-config.ts` - `ts/packages/base/src/messaging/runtime.ts` - Effect primitives: - `Config.duration`, `Duration.Duration`, and existing `Duration.millis` compatibility conversions. - Rewrite shape: - Change internal runtime config fields such as `receiveTimeoutMs`, `requestTimeoutMs`, `retryDelayMs`, and `rateLimitTimeoutMs` to `Duration.Duration`. - Load env-backed values with `Config.duration` while preserving Python-style millisecond defaults and public numeric compatibility options. - Keep external `timeoutMs` option names numeric in request/response, processor, and client boundaries unless their public API is deliberately changed. - Tests: - Base runtime config tests cover legacy millisecond env values and Effect duration string env values. - Messaging runtime and consumer tests preserve retry and timeout behavior. ### Complete: Qdrant Known-Collection MutableHashSet Cleanup - TrustGraph evidence: - `ts/packages/flow/src/storage/embeddings/qdrant-doc.ts` - `ts/packages/flow/src/storage/embeddings/qdrant-graph.ts` - Effect primitives: - `MutableHashSet` from `effect`. - Rewrite shape: - Replace long-lived `Set` known-collection caches with `MutableHashSet` in Qdrant graph/doc embedding stores. - Keep short-lived local `Set` values for pure query traversal or fixture assertions as no-op boundaries. - Tests: - Qdrant embeddings tests prove graph cache hits skip repeated collection existence checks and deletion invalidates the cache. ### P2: Canonicalize MCP Around The Effect Server - Status: - MCP now builds under strict tsgo, the stdio server has an Effect-backed compatibility implementation, and an Effect `McpServer.layerStdio` entrypoint exists. - Remaining shape: - Keep the old SDK/Zod stdio compatibility surface for now. - Prove `tools/list` and `tools/call` parity before deleting any public entry point or dropping `zod`/server-side MCP SDK dependencies. - Pay special attention to `text_completion`: legacy calls the TrustGraph gateway, while the Effect server currently uses an Effect AI `LanguageModel`/OpenAI layer. - Tests: - `cd ts && bun run --cwd packages/mcp build` - Root `cd ts && bun run check` ### No-op: Workbench Theme And Browser Storage Boundaries - Status: migrated and documented no-op for remaining direct browser matches. - Evidence: - `settingsAtom`, `themeAtom`, `flowIdAtom`, and `conversationAtom` are canonical `Atom.kvs` entries backed by `BrowserKeyValueStore.layerLocalStorage`. - The only remaining `tg-theme` production read is a legacy migration fallback. The pre-paint `index.html` script is a host boundary that restores the canonical key before React and Effect services mount. - Graph canvas DOM/class inspection remains render-only host behavior. - Rule: - Do not reopen workbench `localStorage` as a backend/runtime blocker unless a canonical `Atom.kvs` entry is bypassed or a legacy migration fallback can be intentionally removed. ## Recommended PR Order 1. MCP protocol parity tests and legacy stdio flip/removal decision. 2. Flow/client RPC stream API design beyond callback/Promise compatibility. 3. Long-lived ref-backed `HashMap` state cleanup where clone helpers remain. 4. Sibling service `Effect.fn` normalization where arrow-returned generators still appear. ## No-Op Rules Do not flag these as rewrite blockers without additional proof: - Promise-returning CLI actions, MCP SDK callbacks, client compatibility methods, and Fastify route handlers at true external boundaries. Boundary code still must map failures into typed errors or wire errors. - `try`/`catch` blocks at host/tool boundaries only when the catch maps into a typed error or a wire-contract error. Internal exception capture should use `Effect.try`, `Effect.tryPromise`, or `Result.try`. - `S.Class`, `S.TaggedErrorClass`, `Context.Service`, `Rpc.make`, and `HttpApi.make` when they are required or idiomatic for the Effect API. - Plain `Map` usage for local pure transformations, such as graph utility construction, unless the state is long-lived mutable service state. - Plain synchronous loops for parsing or tree traversal are not Effect migration blockers unless they hide async polling, resource ownership, or callback scheduling. - JSON stringification in tests or wire-contract fixtures. Production JSON encode/decode should prefer schema codecs when the encoded form can be preserved. - Workbench pre-paint `index.html` storage reads are host-boundary code used before React and Effect services mount. Keep canonical workbench state in `Atom.kvs`/`BrowserKeyValueStore`, and treat old-key reads as migration fallbacks unless backward compatibility is intentionally dropped. - The old MCP SDK/Zod stdio server is compatibility code until an Effect stdio entrypoint plus parity tests prove the same public protocol behavior. - Client `newableFactory` assertions that preserve vendored callable-plus-new API facades are compatibility boundaries unless the public constructor API is intentionally redesigned. - Base `AsyncProcessor`, `Flow`, and `FlowProcessor` callable-plus-newable export assertions are compatibility boundaries unless the public constructor API is intentionally redesigned. - TrustGraph `PubSubBackend` / backend `PubSub` service is a broker adapter boundary for NATS/Pulsar-style topics, acknowledgement, schema codecs, and backend lifecycle. Effect's native `PubSub` can replace in-process fanout helpers, but not the distributed broker abstraction by itself. This was rechecked against `/home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4/packages/effect/src/PubSub.ts`, whose exported API is local publish/subscribe over Effect queues. - Request-response pending shutdown semantics are complete in `makeEffectRequestResponseFromPubSub`: pending calls race response waiting against a `Deferred` stop signal and fail with tagged `MessagingLifecycleError`; stop idempotence is owned by an Effect `Ref`. - Legacy `makeConsumer` facade blocking-loop ownership is complete: `start()` now initializes scoped Effect consumers and returns after startup, while `stop()` closes the native consumer scope. - `ts/packages/flow/src/gateway/rpc-protocol.ts` is a Fastify socket compatibility bridge. Do not treat it as a wholesale server-layer replacement target until the gateway is ready to move onto Effect SocketServer or Effect HTTP routing; local parser validation and Effect collection cleanup inside that bridge are still valid. ## Acceptance For Final Loop Completion The overall playbook loop is complete only when: - All remaining playbook signal matches are migrated or documented as no-op external-boundary cases with concrete evidence. - No P0/P1/P2 migration item remains in this audit. - `cd ts && bun run check`, `cd ts && bun run build`, `cd ts && bun run test`, and `git diff --check` pass after the final migration slice.