trustgraph/ts/EFFECT_NATIVE_REWRITE_AUDIT.md

TrustGraph Effect-Native Rewrite Opportunity Audit

This is the current backlog snapshot for the playbook in ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md. The branch is ts-port-effect-v4. The unrelated local file .idea/effect.intellij.xml must stay uncommitted.

Inputs

Verified source roots:

• TrustGraph TS port: /home/elpresidank/YeeBois/dev/trustgraph/ts
• Effect v4 subtree: /home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4
• Installed Effect beta used by this workspace: ts/node_modules/effect

Current signal counts from ts/packages after the 2026-06-02 dispatcher Effect collections slice:

Signal	Count
Effect.runPromise	175
Effect.runPromiseWith	0
Effect.cached	0
Layer.succeed	13
Map<	86
WebSocket	72
new Map	56
new Set	15
Set<	9
toPromiseRequestor	0
makeAsyncProcessor	19
receive(	17
while (	2
new Error	7
new Promise	9
JSON.parse	4
localStorage	9
JSON.stringify	8
setTimeout	3
process.env	3

Notes:

• The remaining process.env hits are in packages/workbench/playwright.config.ts.
• In production packages/base, packages/cli, and packages/mcp sources, the strict scans for new Error, new Promise, setTimeout, JSON.parse, JSON.stringify, and direct process.env reads are clean.
• Effect.runPromise is expected at external Promise compatibility boundaries, but each match should still be audited for avoidable internal runtime ownership.
• The dispatcher Effect collections slice removed native Map/Set from the gateway service registries, streaming membership set, and scoped requestor cache. Remaining broad Map/Set matches include tests/fakes, WeakMap compatibility caches, short-lived pure traversal collections, and larger ref-backed service state that still needs focused HashMap/MutableHashMap cleanup.
• The Effect.runPromise and WebSocket counts dropped in this snapshot because EffectRpcClient now owns its RPC/socket layer with ManagedRuntime and uses Effect's WebSocket constructor layer.
• The raw WebSocket count increased in this snapshot because the adapter slice added focused tests and typed adapter names; production websocket-adapter.ts is now clean of try/catch, normal Error, and the previous constructor assertions.
• The new Error count dropped because websocket-adapter.ts now throws S.TaggedErrorClass adapter errors.
• The latest client socket slice removed the remaining production trustgraph-socket.ts normal Error, raw JSON.parse, and listener try/catch matches. The remaining client socket modernization signal is the shared newableFactory constructor assertion pattern.
• The service entrypoint runtime slice dropped the Effect.runPromise count by replacing remaining flow service run() program facades with ManagedRuntime and routing local ts/scripts/run-* launchers through runMain()/NodeRuntime.runMain.
• The base processor compatibility runtime slice dropped the Effect.runPromise count again by moving AsyncProcessor, Flow, and FlowProcessor Promise compatibility facades onto ManagedRuntime.
• The base flow definition schema slice removed hand-rolled Predicate/object narrowing from flow-processor.ts; signal counts are unchanged because this was a validation-quality migration.
• The text completion stream sentinel slice removed the duplicated Effect.void as Effect.Effect<undefined> assertions from provider stream unfold branches. Counts are unchanged because this was an Effect diagnostic and type-channel cleanup.
• The text completion generator boundary slice removed the Effect.runPromise(Effect.fail(...)) fallback and the related AsyncGenerator/IteratorResult assertions from model/text-completion/common.ts.
• The text completion provider status slice replaced manual status/statusCode record assertions with effect/Predicate narrowing.
• The base parameter spec accessor slice added Schema-backed ParameterSpec<T> values plus flow.parameterEffect(spec) and flow.parameter(spec). Bare string parameter lookup remains available as an unknown compatibility escape, while typed parameter access now decodes through Schema and fails with a tagged FlowParameterDecodeError.
• The base producer/requestor spec accessor slice added typed spec-object accessors for ProducerSpec<T> and RequestResponseSpec<TReq, TRes>, then migrated flow service producer/requestor lookups off caller-chosen generic string calls. Spec object handles are scoped per Flow through WeakMaps and finalizers delete only the handle they registered.
• The native PubSub boundary slice removed the unused legacy messaging/subscriber.ts async queue/fanout implementation. Effect's native PubSub is an in-process hub and does not replace the broker-backed PubSubBackend/NATS boundary, but it should be preferred for future in-process broadcast/fanout needs.
• The base producer scoped runtime slice moved the legacy makeProducer Promise facade onto the existing makeEffectProducerFromPubSub scoped factory. Public start/send/stop remain Promise compatibility boundaries, while producer allocation, flush, and finalizer close now go through the Effect runtime path.
• The NATS typed boundary slice removed the dynamic import("nats") header path and maps header construction plus ack()/nak() failures into tagged PubSubErrors with Effect.try. The receive( and JSON.stringify count increases are from the new mocked NATS backend test, not production code.
• The NATS selective 404 slice replaced catch-all stream/consumer create fallbacks with an internal S.TaggedErrorClass lookup wrapper plus Effect.catchIf recovery only for NATS JetStream missing-resource errors. Non-missing lookup failures now stay on the typed failure path without attempting to create streams or durable consumers.
• The consumer rate-limit retry slice wired the previously unused rateLimitTimeoutMs option and TG_RATE_LIMIT_TIMEOUT_MS config into both legacy and Effect-native consumers. Repeated TooManyRequestsError failures now retry with Schedule.spaced until success or a tagged rate-limit timeout. The new Error count dropped by one because a touched consumer test fixture no longer uses a normal Error.
• The consumer concurrency ownership slice changed the Effect-native consumer runtime so concurrency > 1 allocates one backend consumer per worker instead of sharing a single BackendConsumer.receive() handle. stop is now idempotent through Ref, so explicit stop and scoped finalizers do not close workers twice.
• The request-response stop signal slice added a Deferred shutdown signal to makeEffectRequestResponseFromPubSub. Pending requests now race response waiting against runtime stop and fail promptly with a tagged MessagingLifecycleError instead of waiting for timeout.
• The legacy consumer facade slice moved makeConsumer onto makeEffectConsumerFromPubSub with a ManagedRuntime Promise boundary and a closeable Scope. Consumer workers now use Effect.forkScoped so their lifetime is owned by the caller scope rather than the parent fiber. The Effect.runPromise, receive(, new Promise, and setTimeout counts dropped because the old blocking facade loop and its test timer shim were removed.
• The workbench theme storage slice stopped mirroring themeAtom into the legacy tg-theme localStorage key. The canonical trustgraph-workbench-theme-v1 value remains owned by Atom.kvs over BrowserKeyValueStore.layerLocalStorage; the first-paint host script reads that JSON-encoded key before React mounts and falls back to tg-theme only for legacy installs.
• The Effect AI LanguageModel adapter slice added a reusable makeLanguageModelProvider bridge in text-completion common code. It maps generateText responses to LlmResult, maps streaming text-delta and final finish.usage parts to TrustGraph chunks, and converts Effect AI rate and quota failures into TooManyRequestsError. No concrete provider has been flipped yet.
• The native request/response PubSub slice removed the local Map<string, Queue> response subscriber fanout in makeEffectRequestResponseFromPubSub. Response dispatch now publishes { id, value } envelopes through native effect/PubSub, and each request uses a scoped PubSub.Subscription plus Stream.fromSubscription to wait for its matching response.
• The Claude Effect AI slice moved the Claude provider off the direct @anthropic-ai/sdk wrapper and onto @effect/ai-anthropic AnthropicLanguageModel through makeLanguageModelProvider. The direct SDK dependency was removed from @trustgraph/flow.
• A focused broker-backend scout found no remaining P0 broker runtime rewrite after the producer, NATS, consumer concurrency, rate-limit, and request-response stop slices. PubSubBackend remains an intentional Promise-returning adapter boundary wrapped by PubSub/Effect services.
• The gateway streaming callback slice added Effect-returning dispatcher streaming methods, switched the RPC stream server off nested Effect.runPromiseWith(context) queue offers, and replaced the client StopStreaming sentinel error with Stream.runForEachWhile.
• The FalkorDB scoped client lifecycle slice removed the remaining Effect.cached matches from ts/packages. FalkorDB triples store/query Live layers and direct compatibility factories now acquire clients through Effect.acquireRelease and disconnect them on scope close. The Effect.runPromise count increased by two because the new lifecycle tests run scoped programs at the test boundary.
• The Qdrant config/schema/fakeability slice removed direct production new QdrantClient, sync config loading, payload casts, and Qdrant Layer.succeed service construction from graph/doc store/query modules. The installed Qdrant client exposes no public close/disconnect method, so this remains a fakeable construction and Schema decode slice rather than a scoped finalizer slice. Effect.runPromise increased because the new tests and legacy service initialization logs run Effects at compatibility boundaries.
• The client streaming facade slice did not change signal counts. It centralized the legacy streaming { response, complete, error } envelope decode in trustgraph-socket.ts, uses Schema plus effect/Predicate property narrowing for streaming payload reads, and leaves service-specific legacy completion markers only where they preserve public callback behavior.
• The Ollama embeddings effectful layer slice dropped one Layer.succeed match by making OllamaEmbeddingsLive effectful and mapping config/load failures to EmbeddingsError. The JSON.stringify count increased by one because the new layer test uses a JSON response fixture.
• The text completion provider stream helper slice removed all provider-local Stream.unfold pull loops, dropped the while ( count from 9 to 3, and removed the Mistral content as string assertion. The only remaining text-completion iterator.next match is the toAsyncGenerator compatibility adapter that exposes Effect streams through the public AsyncGenerator<LlmChunk> provider contract.
• The request-response queue stream slice replaced the Effectful waitForResponse generator loop with Stream.fromQueue, Stream.filterMapEffect, Result, and Stream.runHead, dropping the remaining while ( count from 3 to 2. The two remaining production while hits are synchronous parsing/CLI traversal loops, not async polling loops.
• The gateway RPC WebSocket cause-handling slice removed the Promise .catch around the socket program by sandboxing the Effect and handling the resulting Cause in the Effect pipeline before the Fastify fire-and-forget runPromise boundary.
• The client RPC acquisition cause tap slice removed the Promise .catch used only to update connection state on runtime/client acquisition failure. effect-rpc-client.ts now uses Effect.tapCause and Cause.pretty before the public Promise boundary.
• The client socket close Effect boundary slice removed the Promise .catch from BaseApi.close(). The void public facade now runs rpc.close() through Effect.tryPromise and logs the tagged socket close error through Effect.catch.
• The client streaming callback Effect boundary slice removed the remaining production Promise .catch matches from trustgraph-socket.ts by centralizing legacy callback request failures in runLegacyStreamingRequest. The public callback facades still return/ignore Promises where required, but failure mapping now uses Effect.tryPromise and Effect.catch.
• The text completion provider effectful layer slice dropped six Layer.succeed matches by moving OpenAI, OpenAI-compatible, Azure OpenAI, Claude, Mistral, and Ollama processor layers onto makeTextCompletionLayer(makeXProviderEffect(config)). SDK construction and config lookup now live in Effect; sync makeXProvider exports remain compatibility facades.
• The gateway dispatcher ownership and serialization slice did not change broad signal counts. It stopped closing injected pubsub backends, brackets one-shot publish producers with Effect.acquireUseRelease, and routes gateway request/response translation through Effect.try wrappers returning tagged DispatchSerializationError failures.
• Record<string, any> and throwLibrarianServiceError are now clean in ts/packages.

Loop Passes

2026-06-02: Base Request/Response Facade

• Status: migrated and verified.
• Completed:
  • Request/response startup now owns a scoped Effect runtime handle and maps failures to TrustGraph tagged messaging errors.
  • Runtime shutdown is idempotent and uses scoped fibers.
  • Tests cover Promise compatibility, tagged timeout errors, and tagged lifecycle errors.
• Verification:
  • bun run --cwd ts/packages/base test
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts check:tsgo
  • bun run --cwd ts build
  • bun run --cwd ts test

2026-06-02: Gateway Dispatcher Requestor Cache

• Status: migrated and package-verified.
• Completed:
  • Gateway dispatcher caches scoped EffectRequestResponse handles instead of Promise<RequestResponse> values.
  • Lazy requestor creation is serialized with SynchronizedRef.modifyEffect.
  • Streaming final-marker detection is centralized.
  • Dispatcher cleanup uses Effect scope/error handling instead of manual try/catch.
• Verification:
  • bun run --cwd ts/packages/flow test
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts check:tsgo

2026-06-02: Gateway Dispatcher Effect Collections Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/gateway/dispatch/manager.ts now stores the flow/global service registries in effect/HashMap instead of native ReadonlyMap, while explicit entry arrays preserve the public service-name ordering.
  • Streaming service membership now uses effect/HashSet instead of native Set.
  • The scoped requestor cache now stores HashMap<string, EffectRequestResponse<unknown, unknown>> in the existing SynchronizedRef, replacing new Map cloning with immutable HashMap.set.
  • Cache hits and service topic lookups now use HashMap.get plus effect/Option, and ref update tuples use effect/Tuple.make instead of as const assertions.
  • Gateway dispatcher tests now cover concurrent same-key dispatches so the cache still creates exactly one scoped producer/consumer pair.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/gateway-dispatcher.test.ts
  • cd ts && bun run check:tsgo

2026-06-02: Strict Base, CLI, MCP, And tsgo Slice

• Status: migrated, root-verified, committed, and pushed.
• Completed:
  • Base messaging, NATS backend, producer, consumer, subscriber, request/response, runtime factories, processor programs, flow specs, and LLM service now use Effect-native boundaries, schema codecs, scoped cleanup, and S.TaggedErrorClass.make(...) errors.
  • CLI commands now run Effect programs at the command boundary, wrap socket lifecycle with Effect.acquireUseRelease, encode JSON through Effect Schema, and write output without console.log.
  • MCP Effect server now loads env/config through Config, wraps gateway calls with Effect.tryPromise, constructs schema classes with .make, and uses tagged errors.
  • MCP stdio compatibility server keeps createMcpServer and run, but uses Effect callbacks/tryPromise/schema encoding internally. run() uses ManagedRuntime; runMain() uses NodeRuntime.runMain.
  • Flow stateful service launch sites now pass an explicit Context.Context into the base processor runtime instead of hiding requirements behind assertions.
• Verification:
  • cd ts && bun run check
  • cd ts && bun run test
  • cd ts && bun run build
  • git diff --check

2026-06-02: ConfigService Ref-Backed State Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/config/service.ts now models runtime state as a SynchronizedRef<ConfigServiceState> instead of adding mutable store, version, consumer, and producer fields onto the processor object.
  • Config operations have Effect-returning handlers with Promise facades only on the exported compatibility methods.
  • Request narrowing now uses effect/Predicate rather than request-record type assertions.
  • Persistence remains schema-backed and now reads/writes snapshots from the ref-backed state.
  • The consume loop now uses Effect.whileLoop; the remaining consumer.receive(2000) call is a pubsub boundary for this service.
  • Service startup now exposes runMain() through NodeRuntime.runMain. The legacy run() Promise facade uses ManagedRuntime, and ts/scripts/run-config.ts delegates directly to runMain() instead of owning its own catch/process-exit wrapper.
  • Config-service tests cover tagged invalid mutation errors, workspace persistence, legacy load, concurrent ref-backed mutations, and push publishing from the stored producer handle.
• Verification:
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check
  • git diff --check

2026-06-02: ConfigService Operation Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/config/service.ts now dispatches ConfigOperation with effect/Match instead of a native switch.
  • The dispatcher is a named Effect.fn and uses Match.exhaustive against the schema-derived ConfigOperation union.
  • The per-message response sender now uses Effect.fnUntraced instead of an arrow function returning Effect.gen(...).
  • Config-service tests now cover all seven operations through handleOperation, including tagged invalid mutation failures.
  • Existing explicit never annotations on persistEffect, loadFromDiskEffect, persistStateEffect, and readPersistedConfigEffect were removed so Effect can infer the channel.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/config-service.test.ts
  • cd ts && bun run check:tsgo

2026-06-02: Librarian Operation Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/librarian/service.ts now dispatches librarian and collection-management operations with effect/Match instead of native switch statements.
  • Both dispatchers intentionally use Match.orElse rather than Match.exhaustive because raw broker message values can still contain unknown runtime operations before a schema boundary rejects them.
  • Existing tagged LibrarianServiceError operation labels are preserved for promise, sync, Effect-helper, stream-only, and unknown-operation branches.
  • Librarian-service tests now cover representative Match-backed librarian dispatch paths, collection list/update/delete dispatch, and runtime fallback errors without type assertions.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: FlowManager Operation Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/flow-manager/service.ts now dispatches flow operations with effect/Match instead of a native switch.
  • The dispatcher keeps the existing config refresh behavior before routing and uses Match.orElse because FlowRequest.operation is a public wire-level string, not a closed schema literal union.
  • Existing tagged FlowManagerError behavior is preserved for unknown operations and branch-specific failures.
  • Flow-manager tests now cover all eight flow operations through handleOperation, including config-client blueprint mutations and the runtime unknown-operation fallback.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: Client Connection State SubscriptionRef Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/client/src/socket/effect-rpc-client.ts now owns RPC connection state in effect/SubscriptionRef instead of a mutable state variable plus manual listener Set.
  • ts/packages/client/src/socket/trustgraph-socket.ts now bridges UI connection-state listeners through SubscriptionRef.changes instead of a hand-rolled listener array.
  • Both public subscribe APIs preserve synchronous immediate replay and unsubscribe compatibility while using Effect fibers for later updates.
  • Client tests now drive a fake RPC state source to prove immediate replay, connection updates, and unsubscribe behavior.
• Verification:
  • bun run --cwd ts/packages/client test -- src/__tests__/rpc-timeout.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: Client Callback Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/client/src/socket/trustgraph-socket.ts now maps RPC connection status with effect/Match instead of a native switch.
  • Flow agent streaming chunk callbacks now use effect/Match for thought, observation, answer, final-answer, and action, while preserving ignored behavior for unknown chunk types with Match.orElse.
  • Client RPC tests now drive agent stream chunks through the fake RPC stream to prove callback dispatch, ignored fallback behavior, completion signals, and metadata forwarding.
• Verification:
  • bun run --cwd ts/packages/client test -- src/__tests__/rpc-timeout.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: Gateway Term Service HashSet Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/gateway/dispatch/serialize.ts now uses effect/HashSet for static term-bearing request/response service membership instead of native Set.
  • Request and response translators preserve the same deep client/internal term conversion behavior via HashSet.has membership checks.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/gateway-dispatcher.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: Effect AI Stream Part Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/model/text-completion/common.ts now maps Response.StreamPart values with effect/Match instead of a native switch.
  • The matcher handles text-delta, finish, and error explicitly, while preserving ignored behavior for other valid stream parts with Match.orElse.
  • Text-completion common tests now include an ignored text-start stream part before text deltas to prove the fallback path remains silent.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: MCP Tool Transport Adapter Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/agent/mcp-tool/service.ts no longer uses an as unknown as Parameters<Client["connect"]>[0] assertion when connecting the MCP SDK client.
  • The concrete StreamableHTTPClientTransport is wrapped in a small Transport adapter that forwards callbacks, send, close, and protocol version updates while avoiding the SDK's exact-optional sessionId type mismatch.
  • The transport release path still closes the concrete SDK transport.
• Verification:
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check:tsgo

2026-06-04: Agent Service Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/agent/react/service.ts no longer uses native switch for configured tool construction, live tool wiring, or ReAct continuation parsing.
  • Configured tool construction now uses Effect.fn plus effect/Match and preserves unknown-tool logging/fallback behavior.
  • The ReAct parser is exported for focused tests and uses an exhaustive Match over continuation sections.
  • Agent service tests cover configured-tool loading, unknown configured-tool fallback, default descriptions, MCP tool args, continuation parsing, and final-answer parsing.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/agent-service.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: CLI Library MIME Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/cli/src/commands/library.ts now uses effect/Match for extension-to-MIME mapping instead of a native switch.
  • guessMimeType is exported for focused CLI helper coverage.
  • CLI tests now cover known extensions, case normalization, html/htm aliases, and fallback behavior for unknown or extensionless paths.
  • ts/packages/cli/package.json excludes dist/** from Vitest so built test output is not executed a second time after root build lanes.
• Verification:
  • bun run --cwd ts/packages/cli test -- src/__tests__/library.test.ts
  • cd ts && bun run check:tsgo

2026-06-04: Streaming ReAct Parser Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/agent/react/parser.ts now uses effect/Match for ReAct state emission instead of a native switch.
  • Marker states now use an inferred MarkerState type instead of per-entry as ReActState assertions.
  • Flush now processes an unterminated final buffer line through marker detection, fixing Final Answer: chunks that arrive without a trailing newline.
  • Parser tests cover split markers, pre-marker thought fallback, continuations, action input, and final-answer routing.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/agent-parser.test.ts
  • cd ts && bun run check:tsgo

2026-06-02: RAG And Agent Requestor Bridge Slice

• Status: migrated, root-verified, committed, and pushed.
• Completed:
  • ts/packages/flow/src/retrieval/graph-rag.ts and ts/packages/flow/src/retrieval/document-rag.ts now accept EffectRequestResponse clients directly. The engines no longer adapt Effect requestors back to Promise requestors and then wrap those calls in Effect.tryPromise.
  • ts/packages/flow/src/retrieval/graph-rag-service.ts and ts/packages/flow/src/retrieval/document-rag-service.ts now pass native flow requestors directly into the engines.
  • ts/packages/flow/src/agent/react/tools.ts now accepts EffectRequestResponse clients directly for graph RAG, document RAG, triples, and MCP tool calls. Tool input narrowing uses Schema and effect/Predicate rather than local request/response type assertions.
  • ts/packages/flow/src/agent/react/service.ts wires default and configured tools with native Effect requestors instead of toPromiseRequestor.
  • Graph RAG, document RAG, and agent service startup now expose runMain() through NodeRuntime.runMain; their legacy run() Promise facades use ManagedRuntime.
  • ts/scripts/run-graph-rag.ts, ts/scripts/run-document-rag.ts, and ts/scripts/run-agent.ts now delegate to runMain().
• Verification:
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: KnowledgeCore Ref-Backed State Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/cores/service.ts now exposes a typed KnowledgeCoreService instead of AsyncProcessorRuntime & Record<string, any>.
  • Runtime state now lives in SynchronizedRef<KnowledgeCoreServiceState> with kgCores, deCores, the request consumer, and response producer.
  • Knowledge operations now have Effect-returning handlers with Promise facades only on exported compatibility methods.
  • Persistence now decodes legacy and current snapshot shapes with Effect Schema and encodes JSON through Schema rather than raw JSON.parse/JSON.stringify plus assertions.
  • The consume loop now uses Effect.whileLoop; the remaining consumer.receive(2000) call is a pubsub boundary for this service.
  • The service exposes runMain() through NodeRuntime.runMain; legacy run() uses ManagedRuntime, and ts/scripts/run-knowledge.ts delegates to runMain().
  • ts/packages/base/src/schema/messages.ts now models legacy hyphenated knowledge request/response aliases so the service can preserve the wire shape without response type assertions.
  • New knowledge-core tests cover ref-backed mutation, graph embedding alias responses, concurrent state updates, and legacy persistence loading.
• Verification:
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: KnowledgeCore Operation Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/cores/service.ts now dispatches KnowledgeOperation with effect/Match instead of a native switch.
  • The dispatcher is a named Effect.fn and uses Match.exhaustive against the schema-derived KnowledgeOperation union, so newly modeled operations should surface as type/check failures until handled.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/knowledge-core-service.test.ts
  • cd ts && bun run check:tsgo
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Flow Manager And Librarian Runtime Normalization

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/flow-manager/service.ts and ts/packages/flow/src/librarian/service.ts now expose runMain() through NodeRuntime.runMain.
  • Their legacy run() Promise facades now use ManagedRuntime instead of directly owning Effect.runPromise.
  • ts/scripts/run-flow-manager.ts and ts/scripts/run-librarian.ts now delegate to runMain() instead of wrapping startup with local .catch(console.error/process.exit) handlers.
• Verification:
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: FlowManager Ref-Backed State Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/flow-manager/service.ts now exposes a typed FlowManagerService instead of AsyncProcessorRuntime & Record<string, any>.
  • Runtime state now lives in SynchronizedRef<FlowManagerServiceState> with flows, blueprints, the request consumer, response producer, and config request client.
  • Flow operations now have Effect-returning handlers with Promise facades only on exported compatibility methods.
  • Blueprint config loading now narrows runtime values before constructing Blueprint records, replacing the prior parsed as Blueprint shortcut.
  • start-flow and stop-flow mutate the flow map through SynchronizedRef.modifyEffect, making duplicate checks and map updates atomic.
  • The consume loop now uses Effect.whileLoop; the remaining consumer.receive(2000) call is a pubsub boundary for this service.
  • New flow-manager tests cover tagged errors, ref-backed flow mutation, config push/delete requests, blueprint narrowing, duplicate concurrent starts, and message-level flow-error responses.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: FlowManager Effect.fn Normalization Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/flow-manager/service.ts no longer defines reusable helpers as arrow functions that immediately return Effect.gen(...).
  • Config request, blueprint refresh, flow refresh, blueprint handlers, flow handlers, config push/delete, resource close, consume, run, and local operation handling now use named Effect.fn providers.
  • Hot local helpers for one-message consumption and response sending use Effect.fnUntraced.
  • pushFlowsConfigEffect keeps its best-effort logging/swallowing contract through the Effect.fn pipeable form instead of a wrapper generator.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/flow-manager-service.test.ts
  • cd ts && bun run check:tsgo
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Librarian Schema And Assertion Cleanup Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/schema/messages.ts now models librarian upload and stream request/response fields directly, instead of requiring service-side as LibrarianResponse casts for the existing wire protocol.
  • ts/packages/flow/src/librarian/service.ts now decodes persisted librarian state through a concrete S.fromJsonString schema instead of a generic JSON decode plus as A.
  • Document metadata metadata triples now narrow through Schema decoding with Option before being included in normalized metadata.
  • Upload, stream, and complete-upload request/response constructors now rely on the schema-modeled fields instead of local type assertions.
  • New librarian tests cover modeled upload fields, concrete persisted-state loading, and schema-backed metadata triple normalization.
• Remaining:
  • Resolved by the typed runtime loop and ref-backed state slices below.
• Verification:
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Librarian Tagged Operation Helper Slice

• Status: migrated and root-verified.
• Completed:
  • Removed the librarian throwLibrarianServiceError helper.
  • get-document-metadata, list-children, upload-chunk, get-upload-status, and abort-upload now dispatch through local Effect-returning helpers that fail with LibrarianServiceError.
  • Compatibility methods for those operations now return Promise facades backed by Effect.runPromise.
  • The librarian tests now await the Promise compatibility facade for upload status.
• Remaining:
  • Resolved by the typed runtime loop and ref-backed state slices below.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Librarian Typed Runtime Loop Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/librarian/service.ts now exposes a typed LibrarianService interface instead of AsyncProcessorRuntime & Record<string, any>.
  • Service construction now uses makeAsyncProcessor<LibrarianServiceError> with runEffect; the old method-bag run override and as LibrarianService cast are gone.
  • The librarian startup poller now uses Effect.whileLoop.
  • The local operation helpers retrieve the initialized service through an Effect gate rather than closing over an unsafe partially built value.
• Remaining:
  • Resolved by the ref-backed state slice below.
• Verification:
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Librarian Ref-Backed State Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/librarian/service.ts now stores documents, processing records, upload sessions, collection manager, and pubsub handles in SynchronizedRef<LibrarianServiceState>.
  • Document, processing, upload, collection, persistence, load, and stop paths now read snapshots or mutate cloned maps/managers through the ref instead of writing fields on the service object.
  • Upload chunk updates clone nested UploadSession.chunks before replacing the upload map entry, avoiding mutable nested state hidden behind the ref.
  • Librarian response producers and consumers are read/nullified through ref-backed handles.
• Verification:
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test -- src/__tests__/librarian-service.test.ts
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client RPC Managed Runtime Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/effect-rpc-client.ts now builds one ManagedRuntime from the RPC client layer instead of manually creating a Scope, building the layer, and calling Effect.runPromise for every operation.
  • RPC dispatch and stream dispatch continue to expose the existing Promise-returning EffectRpcClient facade, but they run through the managed runtime and close with runtime.dispose().
  • The Effect RPC socket path now consumes Socket.layerWebSocketConstructorGlobal instead of a duplicate local WebSocket constructor layer.
  • Dispatch payload construction now uses DispatchPayload.make(...) so schema classes are not instantiated with new.
  • Client socket logging and timestamp creation now use Effect Logger and Clock instead of direct console and Date.now() calls in the touched surface.
• Verification:
  • bun run --cwd ts/packages/client build
  • cd ts && bun run check
  • bun run --cwd ts/packages/client test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client WebSocket Adapter Error Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/websocket-adapter.ts now models host fallback failures with WebSocketAdapterError via S.TaggedErrorClass.
  • Synchronous getWebSocketConstructor() and getRandomValues() facades keep their public signatures while using Result.try instead of local try/catch blocks.
  • Runtime predicates now narrow WebSocket constructor modules and crypto modules without the previous constructor/result type assertions.
  • New adapter tests cover global WebSocket selection, optional ws fallback, global crypto, typed crypto failure, and typed adapter errors.
• Verification:
  • bun run --cwd ts/packages/client build
  • bun run --cwd ts/packages/client test -- src/__tests__/websocket-adapter.test.ts
  • bun run --cwd ts/packages/client test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check
  • git diff --check

2026-06-02: Client Socket Tagged Error And JSON Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/trustgraph-socket.ts now models socket API failures with TrustGraphSocketError via S.TaggedErrorClass.
  • Flow/blueprint JSON response parsing now uses Schema decoding through S.UnknownFromJsonString instead of raw JSON.parse.
  • Token-cost config JSON keeps the previous invalid-string fallback behavior while decoding through Schema/Option.
  • Connection-state listener isolation now uses Result.try and typed socket errors instead of a local try/catch.
  • Flow start, row embeddings, collection update, and response-error failures now reject with tagged socket errors instead of normal Error.
  • Flow API tests cover invalid JSON and response-error rejections.
• Verification:
  • bun run --cwd ts/packages/client build
  • bun run --cwd ts/packages/client test -- src/__tests__/flows-api.test.ts
  • cd ts && bun run check
  • bun run --cwd ts/packages/client test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client Newable Factory Compatibility Decision

• Status: documented no-op for the current loop.
• Evidence:
  • ts/packages/workbench/src/atoms/workbench.ts constructs new BaseApi(...).
  • ts/packages/client/src/__tests__/flows-api.test.ts constructs new FlowsApi(...), and sibling API facades expose the same constructor shape.
  • EffectRpcClient and BaseApi also preserve callable factory exports for compatibility with the vendored TrustGraph client shape.
• Decision:
  • The remaining newableFactory(... ) as unknown as NewableFactory<...> assertions in client socket files are TypeScript compatibility boundaries, not Effect error/requirement channel assertions and not replacements for an Effect primitive.
  • Removing them safely requires a deliberate public API redesign or explicit class implementations for every API facade, not a local Effect-native rewrite.
• Verification:
  • Current client/root verification from the tagged error slice covers this no-op decision.

2026-06-02: Service Entrypoint Runtime Slice

• Status: migrated and root-verified.
• Completed:
  • Remaining flow service run(): Promise<void> program facades now use ManagedRuntime.make(Layer.empty) instead of direct Effect.runPromise(program).
  • Remaining flow service modules now expose runMain() through NodeRuntime.runMain(program).
  • Local ts/scripts/run-* launchers for gateway, prompt, chunker, extractor, PDF decoder, embeddings, triples, graph/document embeddings, text-completion providers, and MCP tool service now delegate directly to runMain().
  • Direct Effect.runPromise(program) matches in ts/packages/flow/src are clean. Remaining Effect.runPromise matches are callback/Promise compatibility boundaries for later slices.
• Verification:
  • bun run --cwd ts/packages/flow build
  • git diff --check
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Base Processor Compatibility Runtime Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/processor/async-processor.ts now uses a ManagedRuntime for Promise compatibility methods, signal-shutdown execution, and legacy AsyncProcessor.launch.
  • ts/packages/base/src/processor/flow.ts now owns a per-flow ManagedRuntime for start, stop, runInCompatibilityScope, and Promise resource facades.
  • ts/packages/base/src/processor/flow-processor.ts now uses a ManagedRuntime for the public start(context) facade instead of a local Effect.runPromiseWith runner.
  • ts/packages/base/src/spec/parameter-spec.ts now routes legacy add through flow.runInCompatibilityScope(...), matching the other specs.
  • Subagent checks confirmed NodeRuntime is process-entrypoint-only here; @trustgraph/base should not add an @effect/platform-node dependency for these compatibility facades.
• Remaining:
  • Constructor as unknown as shims in base processors preserve callable-plus-newable public exports and are compatibility boundaries for this loop.
  • Typed string lookup casts in Flow need a real typed-spec/key redesign; HashMap/MutableHashMap alone cannot infer T from a bare string.
• Verification:
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • git diff --check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Base Flow Definition Schema Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/processor/flow-processor.ts now validates config.flows with Effect Schema instead of local Predicate/object/string-record guards.
  • Invalid flow definition payloads still log/skip and preserve the existing config-handler and acknowledgement behavior.
  • ts/packages/base/src/__tests__/flow-processor-runtime.test.ts now covers an invalid nested flow definition that is acknowledged without starting resources.
• Verification:
  • bun run --cwd ts/packages/base test -- src/__tests__/flow-processor-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Text Completion Stream Sentinel Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/model/text-completion/{ollama,openai,mistral,azure-openai,claude,openai-compatible}.ts now return the Stream.unfold end sentinel with Effect.as(Effect.void, undefined).
  • Removed six Effect.void as Effect.Effect<undefined> assertions without replacing them with Effect.succeed(undefined), which @effect/tsgo flags as a diagnostic.
• Verification:
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Text Completion Generator Boundary Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/model/text-completion/common.ts now rejects fallback AsyncGenerator.throw(...) calls with the mapped tagged provider error directly instead of running Effect.fail(...) through Effect.runPromise.
  • The custom generator object no longer uses as AsyncGenerator, as Promise<IteratorResult<LlmChunk>>, or as LlmChunk assertions.
  • Added a focused unit test for fallback throw mapping.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Text Completion Provider Status Narrowing Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/model/text-completion/common.ts now uses effect/Predicate narrowing for provider status / statusCode inspection instead of local record assertions.
  • ts/packages/flow/src/__tests__/text-completion-common.test.ts covers both rate-limit status fields.
• Verification:
  • bun run --cwd ts/packages/flow test -- src/__tests__/text-completion-common.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Base Parameter Spec Accessor Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/spec/parameter-spec.ts now models ParameterSpec<T> with an Effect Schema codec. Legacy parameter specs default to S.Unknown, preserving name-based registration while making typed access schema-backed.
  • ts/packages/base/src/processor/flow.ts now exposes flow.parameterEffect(spec) and flow.parameter(spec) for inferred, Schema-decoded parameter values. String lookup remains available as an unknown compatibility escape instead of a caller-chosen generic type.
  • Parameter schema failures now fail with the tagged FlowParameterDecodeError rather than a normal Error.
  • ts/packages/flow/src/chunking/service.ts now declares numeric chunk parameters once and retrieves them through the typed spec-object accessor.
  • ts/packages/base/src/__tests__/flow-spec-runtime.test.ts covers typed parameter decoding, legacy string lookup, missing parameter errors, sync accessor decoding, and schema mismatch errors.
• Remaining:
  • Add typed spec-object accessors for producers and requestors so call sites can stop spelling generic string lookups for those registries too.
• Verification:
  • bun run --cwd ts/packages/base test -- src/__tests__/flow-spec-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/base test
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Base Producer And Requestor Spec Accessor Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/spec/producer-spec.ts now exposes ProducerSpec<T>.producerEffect(flow) and stores typed producer handles in a per-spec WeakMap keyed by Flow.
  • ts/packages/base/src/spec/request-response-spec.ts now exposes RequestResponseSpec<TReq, TRes>.requestorEffect(flow) and stores typed requestor handles in a per-spec WeakMap keyed by Flow.
  • Spec finalizers remove only the exact handle they registered, avoiding stale finalizers deleting newer registrations for the same flow/spec pair.
  • ts/packages/base/src/processor/flow.ts now supports flow.producerEffect(spec), flow.requestorEffect(spec), flow.producer(spec), and flow.requestor(spec) while keeping string accessors as untyped compatibility escapes.
  • Base service adapters and flow service handlers now reuse the same hoisted producer/requestor spec object in their spec arrays and handler lookups.
  • ts/packages/base/src/__tests__/flow-spec-runtime.test.ts covers typed spec-object lookups, duplicate spec identity failures, and scoped finalizer cleanup for producer and requestor handles.
• Remaining:
  • Bare string Flow producer/requestor accessors remain compatibility escapes for external/legacy callers, but new Effect service code should use spec objects.
• Verification:
  • bun run --cwd ts/packages/base test -- src/__tests__/flow-spec-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/base test
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Native PubSub Boundary Slice

• Status: migrated and package-verified.
• Completed:
  • Confirmed Effect's native PubSub module is an in-process asynchronous hub with scoped subscriptions, not a NATS/Pulsar-compatible broker boundary.
  • Kept TrustGraph's PubSubBackend and PubSub service as the broker adapter layer because it owns topics, broker producers/consumers, acknowledgement, schema codecs, and backend lifecycle.
  • Removed the unused legacy ts/packages/base/src/messaging/subscriber.ts implementation, which duplicated in-process async queue/fanout behavior.
  • Removed the corresponding makeAsyncQueue, makeSubscriber, Subscriber, and AsyncQueue barrel exports from ts/packages/base/src/messaging/index.ts.
• Remaining:
  • Future in-process fanout or request-streaming code should use effect/PubSub, Queue, Stream.fromPubSub, or Channel.fromPubSub rather than adding another local async queue implementation.
  • Do not replace PubSubBackend with effect/PubSub unless the code path is explicitly local-only and does not need broker semantics.
• Verification:
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check

2026-06-02: Gateway Streaming Callback Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/gateway/dispatch/manager.ts now exposes dispatchGlobalServiceStreamingEffect and dispatchFlowServiceStreamingEffect so Effect callers can handle stream chunks without Promise callback re-entry.
  • The existing Promise-returning streaming methods remain as compatibility facades and wrap responders with Effect.tryPromise.
  • ts/packages/flow/src/gateway/rpc-server.ts now writes stream chunks into the RPC queue through the dispatcher Effect path, removing the prior Effect.context plus Effect.runPromiseWith(context) bridge.
  • ts/packages/client/src/socket/effect-rpc-client.ts now uses Stream.runForEachWhile for early stream termination instead of throwing a synthetic StopStreaming tagged error.
  • Gateway dispatcher tests now exercise both the Promise compatibility streaming path and the Effect-native responder path.
• Remaining:
  • ts/packages/flow/src/gateway/rpc-protocol.ts remains a Fastify socket compatibility bridge, not a direct replacement target for Effect RPC server layers yet.
• Verification:
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/client build
  • bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts
  • bunx --bun vitest run src/__tests__/rpc-timeout.test.ts
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client Streaming Facade Normalization Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/trustgraph-socket.ts now decodes the legacy streaming envelope with Schema before service-specific callback handling.
  • Streaming payload reads now use effect/Predicate property narrowing helpers instead of repeated response-wrapper assertions.
  • Graph RAG, document RAG, text completion, prompt, agent, and document stream callbacks now use a shared streamComplete(...) helper. The RPC DispatchStreamChunk.complete bit is the default transport completion source, with legacy service markers preserved for public compatibility.
  • Explainability triples are decoded through a recursive Schema instead of as Triple[].
  • The focused client test now proves normalized DispatchStreamChunk completion flows through graphRagStreaming and final metadata.
• Verification:
  • bunx --bun vitest run src/__tests__/rpc-timeout.test.ts
  • bun run --cwd ts/packages/client build
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/client test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Ollama Embeddings Effectful Layer Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/embeddings/ollama.ts now exposes makeOllamaEmbeddingsEffect for effectful config loading and service construction.
  • OllamaEmbeddingsLive now uses Layer.effect and maps config/load failures into EmbeddingsError instead of preconstructing the service with Layer.succeed.
  • The direct makeOllamaEmbeddings(config) factory remains as a compatibility facade, while the canonical program entrypoint preserves the provider tagged error channel.
  • Ollama response JSON parsing no longer uses a Promise type assertion.
  • The focused embeddings tests now cover both direct factory use and the effectful OllamaEmbeddingsLive layer.
• Verification:
  • bunx --bun vitest run src/__tests__/ollama-embeddings.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: FalkorDB Scoped Client Lifecycle Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/storage/triples/falkordb.ts and ts/packages/flow/src/query/triples/falkordb.ts now model FalkorDB client acquisition with Effect.acquireRelease.
  • FalkorDB Live layers now use Layer.effect and own Redis client disconnect finalizers through the layer scope.
  • Direct Promise compatibility factories and direct service factories now bracket each operation with scoped acquisition instead of hiding mutable Effect.cached connection slots.
  • Legacy makeTriplesStoreService and makeTriplesQueryService provider hooks now acquire scoped FalkorDB services and map acquisition failures to ProcessorLifecycleError; modern program entrypoints preserve the FalkorDB tagged layer error type.
  • FalkorDB query row field extraction now uses effect/Predicate narrowing instead of record/string type assertions.
  • New lifecycle tests use fake clients/graphs to prove connect on acquire and disconnect on scope close for both triples store and triples query.
• Verification:
  • bunx --bun vitest run src/__tests__/falkordb-lifecycle.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Qdrant Config, Schema, And Fakeable Construction Slice

• Status: migrated and root-verified.
• Completed:
  • Added ts/packages/flow/src/qdrant/client.ts as the narrow fakeable Qdrant surface used by graph/doc embedding store/query modules.
  • Graph and document Qdrant store/query constructors now create clients through Effect.try, load Qdrant config in Effect, and map config/client failures into their existing S.TaggedErrorClass errors.
  • Graph and document query payload extraction now uses Schema.decodeUnknownEffect(...).pipe(Effect.option) and skips malformed Qdrant payloads without type assertions.
  • Qdrant graph/doc query Live layers and graph store Live layer now use Layer.effect instead of preconstructing services with Layer.succeed.
  • Legacy graph store/query/doc query processor providers now acquire Qdrant services with named Effect.fn providers and map startup failures to ProcessorLifecycleError.
  • The installed Qdrant client still has no public close/disconnect method, so no Effect.acquireRelease finalizer was added for Qdrant.
• Verification:
  • bunx --bun vitest run src/__tests__/qdrant-embeddings.test.ts
  • bun run --cwd ts/packages/flow build
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Text Completion Provider Stream Helper Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/model/text-completion/common.ts now exposes streamTextCompletionChunks, an Effect-native helper built on Stream.fromAsyncIterable, Ref, Stream.filterMap/Result, and a final token chunk append.
  • OpenAI, Azure OpenAI, OpenAI-compatible, Mistral, Ollama, and Claude streaming providers now share the helper instead of each hand-rolling Stream.unfold plus iterator.next loops.
  • Mistral non-streaming and streaming content normalization now uses effect/Predicate and Option narrowing through textFromContent, removing the prior content as string assertion.
  • The helper uses the installed Effect beta's Option.fromNullishOr and Result-shaped Stream.filterMap API, verified by check:tsgo.
  • ts/packages/flow/src/__tests__/text-completion-common.test.ts covers token accumulation/final chunk emission and non-string content narrowing.
• Remaining:
  • Full Effect AI provider swaps still need parity tests first; current OpenAI and Azure behavior is Chat Completions based, no installed Azure/Mistral/Ollama Effect AI provider exists, and Anthropic needs explicit text/token/streaming/rate-limit parity coverage before replacement.
• Verification:
  • bunx --bun vitest run src/__tests__/text-completion-common.test.ts
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Request-Response Queue Stream Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/base/src/messaging/runtime.ts now waits for accepted request-response replies by converting the response Queue to a Stream.fromQueue.
  • Recipient filtering now uses Stream.filterMapEffect with Result to skip partial responses until the recipient returns true.
  • Stream.runHead replaces the prior while (true)/Queue.take loop and preserves the existing timeout behavior around request-response calls.
  • ts/packages/base/src/__tests__/messaging-runtime.test.ts now covers recipient filtering across partial and final responses.
• Remaining:
  • The two remaining production while ( matches are agent/react/parser.ts line-buffer parsing and cli/src/commands/util.ts Commander parent traversal; neither is async polling or resource ownership.
• Verification:
  • bunx --bun vitest run src/__tests__/messaging-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Gateway RPC WebSocket Cause Handling Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/flow/src/gateway/server.ts now handles RPC WebSocket program defects and interruptions inside the Effect pipeline with Effect.sandbox, Effect.catch, and Cause.pretty.
  • The previous Promise .catch(...) around Effect.runPromise(...), plus the nested Effect.runPromise used only for logging and socket close, is removed.
  • The outer Effect.runPromise remains as the Fastify WebSocket host boundary.
• Verification:
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client RPC Acquisition Cause Tap Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/effect-rpc-client.ts now observes runtime/client acquisition failures with Effect.tapCause and Cause.pretty.
  • Removed the Promise .catch(...) that only updated local connection state after runtime.runPromise(TrustGraphRpcClientService).
  • Removed the local errorMessage helper and its message-field assertion.
  • Public dispatch, dispatchStream, and close Promise facades remain compatibility boundaries.
• Verification:
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/client build
  • bun run --cwd ts/packages/client test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client Socket Close Effect Boundary Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/trustgraph-socket.ts now wraps rpc.close() with Effect.tryPromise inside the public close(): void facade.
  • Close failures are mapped to the existing tagged TrustGraphSocketError shape and logged through Effect.catch instead of a Promise .catch.
  • The remaining client socket Promise .catch matches were streaming callback compatibility bridges and are now handled by the follow-up slice.
• Verification:
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/client build
  • bun run --cwd ts/packages/client test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Client Streaming Callback Effect Boundary Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/client/src/socket/trustgraph-socket.ts now routes the legacy agent, graphRagStreaming, and documentRagStreaming callback request failures through runLegacyStreamingRequest.
  • runLegacyStreamingRequest uses Effect.tryPromise to map failures into tagged TrustGraphSocketError values, then uses Effect.catch to invoke the public legacy error callback.
  • Production trustgraph-socket.ts no longer has Promise .catch matches; remaining matches in that file are Effect.catch only.
  • Rechecked the PubSub replacement question against Effect v4 source: Effect's native PubSub is an in-process async hub over Effect queues. TrustGraph's PubSubBackend remains the broker adapter boundary for NATS/Pulsar-style topics, subscriptions, acknowledgement, schema codecs, and backend lifecycle.
• Verification:
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/client build
  • bun run --cwd ts/packages/client test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Text Completion Provider Effectful Layer Slice

• Status: migrated and root-verified.
• Completed:
  • Added shared makeTextCompletionLayer for constructing Llm from an effectful LlmProvider.
  • Added makeOpenAIProviderEffect, makeOpenAICompatibleProviderEffect, makeAzureOpenAIProviderEffect, makeClaudeProviderEffect, makeMistralProviderEffect, and makeOllamaProviderEffect.
  • Processor program.layer definitions now use Layer.effect via the shared helper instead of constructing providers inside Layer.succeed.
  • Provider object assembly is split into pure makeXProviderFromClient helpers so Promise-returning provider methods remain external compatibility facades and do not trigger effect(runEffectInsideEffect).
  • Added tests for explicit provider config, shared Llm layer provisioning, and tagged missing-config errors.
• Verification:
  • bunx --bun vitest run src/__tests__/text-completion-providers.test.ts src/__tests__/text-completion-common.test.ts
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check:tsgo
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Gateway Dispatcher Ownership And Serialization Slice

• Status: migrated and root-verified.
• Completed:
  • makeDispatcherManager now tracks whether it owns the pubsub backend and no longer closes injected PubSubBackend instances on stop().
  • publishToTopic now uses Effect.acquireUseRelease so the one-shot producer is closed even when send fails.
  • Gateway dispatch paths now call translateRequestEffect and translateResponseEffect, which wrap serialization with Effect.try and return tagged DispatchSerializationError failures.
  • Streaming dispatch recipients are named Effect.fn callbacks, satisfying strict Effect diagnostics while preserving responder behavior.
  • Tests cover injected backend ownership, typed serialization failure before requestor startup, and producer close on send failure.
• Verification:
  • bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check:tsgo
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Base Producer Scoped Runtime Slice

• Status: migrated and root-verified.
• Completed:
  • Kept PubSubBackend as the broker adapter boundary; Effect native PubSub remains an in-process primitive and is not a replacement for broker-backed topics, subscriptions, acknowledgements, codecs, or backend lifecycle.
  • Reworked makeProducer so the legacy Promise facade allocates producers through makeEffectProducerFromPubSub inside a closeable Scope.
  • stop() now flushes the Effect producer and closes the scope with the registered producer finalizer, including the flush-failure path.
  • Added focused producer facade coverage for send routing, idempotent stop, tagged not-started lifecycle errors, and close-on-flush-failure behavior.
• Verification:
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/base build
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/producer.test.ts
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: NATS Typed Boundary Slice

• Status: migrated and root-verified.
• Completed:
  • Replaced the dynamic header import inside makeNatsProducer with the static NATS headers export.
  • Wrapped publish header construction in Effect.try, so invalid header names/values fail as tagged PubSubError values instead of defects.
  • Wrapped NATS ack() and nak() calls in Effect.try, preserving the existing wrong-message guard and mapping thrown acknowledgement failures into tagged PubSubErrors.
  • Added a mocked NATS backend test covering invalid publish headers and thrown ack/nak failures through the public makeNatsBackend path.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/nats-backend.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: NATS Selective 404 Slice

• Status: migrated and root-verified.
• Completed:
  • Added an internal NatsLookupError tagged error to preserve lookup causes without leaving unknown in Effect error channels.
  • Stream creation now happens only when manager.streams.info() fails with a NATS JetStream 404/missing-resource error.
  • Durable consumer creation now happens only when js.consumers.get() fails with a NATS JetStream 404/missing-resource error.
  • Added mocked NATS tests proving 404 lookups create resources while permission-style lookup failures do not.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/nats-backend.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Consumer Rate-Limit Retry Slice

• Status: migrated and root-verified.
• Completed:
  • Added rateLimitTimeoutMs to the Effect-native messaging runtime config, backed by TG_RATE_LIMIT_TIMEOUT_MS and the Python-compatible default of 7_200_000ms.
  • Reworked legacy makeConsumer retry handling to use Schedule.spaced, retry repeated TooManyRequestsErrors, and fail with a tagged MessagingTimeoutError when the rate-limit timeout elapses.
  • Reworked makeEffectConsumerFromPubSub handler retry handling with the same schedule/timeout behavior while keeping handler failures in typed Effect error channels.
  • Added legacy and Effect-native tests for repeated rate-limit retry until success and negative acknowledgement after retry timeout.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/consumer.test.ts src/__tests__/messaging-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Consumer Concurrency Ownership Slice

• Status: migrated and root-verified.
• Completed:
  • makeEffectConsumerFromPubSub now creates one backend consumer per concurrency worker rather than sharing a single backend consumer across parallel receive loops.
  • Consumer runtime stop is idempotent via Ref.getAndSet, so explicit consumer.stop and scope finalization do not double-close worker handles.
  • Added Effect-native runtime coverage proving concurrency: 3 creates and closes three independent backend consumers exactly once.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Request-Response Stop Signal Slice

• Status: migrated and root-verified.
• Completed:
  • makeEffectRequestResponseFromPubSub now owns a Deferred stop signal for the lifetime of the scoped request-response runtime.
  • request() races response waiting against that stop signal before applying the request timeout, so pending calls fail promptly when the runtime stops.
  • stop() fails the stop signal with a tagged MessagingLifecycleError before interrupting the dispatch loop and closing the producer/consumer resources.
  • Flow PDF decoder and graph embeddings service error unions now include MessagingLifecycleError because requestor failures can surface shutdown.
  • Added Effect-native runtime coverage proving a pending request fails with the tagged lifecycle error when the request-response runtime stops.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • bun run --cwd ts/packages/flow build
  • bun run --cwd ts/packages/flow test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test

2026-06-02: Legacy Consumer Facade Slice

• Status: migrated and root-verified.
• Completed:
  • makeConsumer is now a Promise compatibility facade over makeEffectConsumerFromPubSub instead of owning a separate mutable backend, running flag, retry loop, and direct BackendConsumer.receive.
  • The facade uses a module ManagedRuntime, Scope.make, Scope.provide, and Scope.close to keep public start()/stop() Promises at the boundary while the actual consumer lifetime stays scoped.
  • Legacy Promise handlers are adapted with Effect.tryPromise and preserve TooManyRequestsError as a typed retry signal.
  • makeEffectConsumerFromPubSub now forks workers with Effect.forkScoped, so a caller-owned scope keeps workers alive until stop/finalization.
  • consumer.test.ts no longer encodes start() as the blocking consume-loop join; it waits for observable handler/ack/nak effects and then stops the scoped consumer.
• Verification:
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/consumer.test.ts
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts src/__tests__/flow-spec-runtime.test.ts
  • cd ts && bun run check:tsgo
  • bun run --cwd ts/packages/base build
  • bun run --cwd ts/packages/base test
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • git diff --check

2026-06-02: Workbench Theme KeyValueStore Slice

• Status: migrated and root-verified.
• Completed:
  • themeClassAtom no longer writes the legacy tg-theme localStorage mirror. Theme persistence stays in the canonical trustgraph-workbench-theme-v1 Atom.kvs entry backed by BrowserKeyValueStore.layerLocalStorage.
  • The pre-paint host script in index.html now restores the canonical JSON-encoded theme key before React mounts, with tg-theme retained only as a legacy fallback.
  • Workbench QA now asserts that changing theme writes the canonical key and does not recreate tg-theme.
• Verification:
  • cd ts && bun run check
  • cd ts && bun run workbench:qa
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint

2026-06-02: Effect AI LanguageModel Adapter Slice

• Status: migrated and package-verified.
• Completed:
  • Added makeLanguageModelProvider, a bridge from effect/unstable/ai/LanguageModel into the existing TrustGraph LlmProvider contract.
  • Covered non-streaming text/token mapping, streaming text/final-token mapping, and Effect AI rate/quota failure mapping with fake LanguageModel tests.
  • Kept concrete provider swaps deferred until provider-specific parity is proven.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/text-completion-common.test.ts src/__tests__/text-completion-providers.test.ts

2026-06-02: Native Request/Response PubSub Fanout Slice

• Status: migrated and package-verified.
• Completed:
  • Replaced the request/response runtime's hand-managed Map<string, Queue> response fanout with native effect/PubSub.
  • Each request subscribes before sending, consumes through Stream.fromSubscription, filters by response id, and releases the subscription at scope exit.
  • Kept PubSubBackend as the broker boundary because Effect native PubSub is in-process only and does not provide NATS topics, ack/nack, durable subscriptions, schema codecs, or backend lifecycle.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/messaging-runtime.test.ts src/__tests__/request-response.test.ts src/__tests__/flow-spec-runtime.test.ts

2026-06-02: Claude Effect AI Provider Slice

• Status: migrated and package-verified.
• Completed:
  • Replaced the direct @anthropic-ai/sdk provider implementation with @effect/ai-anthropic AnthropicLanguageModel plus the shared makeLanguageModelProvider adapter.
  • Preserved CLAUDE_KEY, default model, temperature, max output, processor, and public LlmProvider compatibility behavior.
  • Added Claude provider config coverage for CLAUDE_KEY env fallback and the missing-key tagged config error.
  • Removed the direct @anthropic-ai/sdk dependency and its lockfile entries from @trustgraph/flow.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/text-completion-providers.test.ts src/__tests__/text-completion-common.test.ts
  • cd ts/packages/flow && bun run build
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: RPC Dispatch Tagged Error Slice

• Status: migrated and package-verified.
• Completed:
  • Replaced the remaining production S.ErrorClass usage in the Flow gateway RPC contract with S.TaggedErrorClass.
  • Normalized the client-side RPC DispatchError counterpart to the same tagged-error schema shape so both wire contract copies stay aligned.
  • Closed the scratch-note S.ErrorClass finding for production code. The remaining plain new Error matches are test-only helpers or external host-boundary simulations.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts
  • cd ts/packages/client && bunx --bun vitest run src/__tests__/rpc-timeout.test.ts
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Effect Metrics Prometheus Slice

• Status: migrated and root-verified.
• Completed:
  • Replaced the @trustgraph/base prom-client metric wrappers with Effect-native Metric.counter, Metric.histogram, and effect/unstable/observability PrometheusMetrics.format.
  • Kept the existing Prometheus metric names and gateway /api/v1/metrics scrape boundary while removing the direct prom-client dependency and lockfile entries.
  • Changed producer metric recording from a sync callback to an Effect value that runs inside the producer send pipeline.
  • Added isolated metric-registry tests for producer and consumer Prometheus formatting.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/metrics-effect.test.ts src/__tests__/producer.test.ts src/__tests__/messaging-runtime.test.ts
  • cd ts/packages/base && bun run build
  • cd ts/packages/base && bun run test
  • cd ts/packages/flow && bun run build
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts
  • cd ts && bun run check
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: MCP Effect Stdio Entrypoint Slice

• Status: migrated and root-verified.
• Completed:
  • Added an Effect-native stdio layer and process entrypoint with McpServer.layerStdio, NodeStdio.layer, and NodeRuntime.runMain.
  • Reused the same TrustGraphMcpToolkitLive path for HTTP and stdio through a shared toolkit-layer helper.
  • Kept the legacy SDK/Zod stdio export as a compatibility surface until protocol-level tools/list and tools/call parity tests prove it can be flipped or removed.
  • Added focused coverage that the Effect toolkit names remain stable and the stdio layer/entrypoint are exported.
  • Updated the MCP test script to ignore compiled dist/** output so root builds do not cause duplicate Vitest runs from generated tests.
• Scratch-note triage:
  • Metrics, in-process PubSub fanout, Claude Effect AI, RPC S.TaggedErrorClass, and @effect/tsgo setup are already migrated.
  • Remaining valid scratch targets are MCP protocol parity/flip, Duration config cleanup, Term/ClientTerm tagged-union matching, service Effect.fn normalization, @effect/cli, stream/RPC follow-ups, chunking Chunk, cores Promise APIs, and long-lived Map/Set state.
• Verification:
  • cd ts/packages/mcp && bun run test
  • cd ts/packages/mcp && bun run build
  • cd ts && bun run check:tsgo
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Term And ClientTerm Match Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/base/src/schema/primitives.ts now exposes Term as a recursive S.toTaggedUnion("type") schema and aligns Triple.g with the Python/client wire contract as an optional graph string.
  • ts/packages/flow/src/gateway/dispatch/serialize.ts now defines compact client-term schemas with S.tag, decodes unknown term-shaped values with Schema/Option, and translates terms with Match.discriminatorsExhaustive.
  • Removed the gateway serializer's native term switches and unsafe pass-through casts. Malformed known-tag objects now stay ordinary payload objects during deep translation instead of being cast into invalid terms.
  • Replaced pure term helper switches in FalkorDB triples store/query, Qdrant graph embeddings store, Graph RAG, agent tools, and workbench graph utilities with exhaustive Match discriminators.
  • Added tests for named graph string decoding, nested compact/internal term round trips, and malformed known-tag payload preservation.
• Remaining:
  • The client socket streaming term schema is still a local recursive union and can be centralized later if drift appears. It has no native term switch in the current scan.
  • Operation dispatch switches in config, cores, librarian, and flow-manager are separate service-command refactors, not part of this term wire slice.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/schema-effect.test.ts
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/gateway-dispatcher.test.ts src/__tests__/falkordb-lifecycle.test.ts src/__tests__/qdrant-embeddings.test.ts src/__tests__/retrieval-rag.test.ts
  • cd ts/packages/flow && bun run test
  • cd ts/packages/base && bun run build
  • cd ts/packages/flow && bun run build
  • cd ts/packages/workbench && bun run build
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Messaging Runtime Duration Config Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/base/src/runtime/messaging-config.ts now stores internal runtime timing fields as Duration.Duration instead of number-shaped millisecond fields.
  • loadMessagingRuntimeConfig() reads TG_* timing env vars with Config.duration while preserving legacy bare-number millisecond env values through a Config.number(...).map(Duration.millis) fallback.
  • ts/packages/base/src/messaging/runtime.ts now uses Duration values directly for sleeps, retry schedules, and timeout options. It converts back to milliseconds only at the broker receive(timeoutMs) boundary and for existing timeout error payloads.
  • Public compatibility options such as receiveTimeoutMs, rateLimitRetryMs, rateLimitTimeoutMs, and request timeoutMs remain numeric millisecond inputs.
  • Tests now cover both legacy numeric env values and native Effect duration strings.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/base && bunx --bun vitest run src/__tests__/schema-effect.test.ts src/__tests__/messaging-runtime.test.ts src/__tests__/consumer.test.ts
  • cd ts/packages/base && bun run build
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-02: Qdrant MutableHashSet Cache Slice

• Status: migrated and package-verified.
• Completed:
  • ts/packages/flow/src/storage/embeddings/qdrant-graph.ts now uses MutableHashSet<string> for its known-collection cache.
  • ts/packages/flow/src/storage/embeddings/qdrant-doc.ts now uses MutableHashSet<string> inside its store effect shape instead of a native Set<string>.
  • Collection cache membership, insertion, and invalidation now use MutableHashSet.has, MutableHashSet.add, and MutableHashSet.remove.
  • Qdrant tests now prove the graph store service avoids repeated collection existence checks while the collection is cached and checks again after delete invalidates the cache.
• Remaining:
  • The document store's public direct facade still constructs a fresh store effect per operation, so its cache lifetime is per constructed effect shape. Changing that facade to own a shared runtime/service is a separate lifecycle design slice, not part of this collection primitive migration.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts/packages/flow && bunx --bun vitest run src/__tests__/qdrant-embeddings.test.ts
  • cd ts/packages/flow && bun run build
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

2026-06-04: Workbench Match Dispatch Slice

• Status: migrated and root-verified.
• Completed:
  • ts/packages/workbench/src/qa/mock-api.ts now dispatches mock RPC service requests with effect/Match and keeps the existing unknown-service {} fallback.
  • ts/packages/workbench/src/atoms/workbench.ts now dispatches chat submit behavior for graph-rag, document-rag, and agent through Match.exhaustive over the ChatMode union.
  • The strict scan for native switch statements in ts/packages is clean: rg -n "\bswitch\s*\(" ts/packages --glob '*.ts' --glob '*.tsx' returns no matches.
• Verification:
  • cd ts && bun run check:tsgo
  • cd ts && bun run build
  • cd ts && bun run test
  • cd ts && bun run lint
  • git diff --check

Subagent Findings To Preserve

• MCP/workbench:
  • Make the Effect MCP server the canonical implementation. An Effect McpServer.layerStdio entrypoint now exists; the old stdio server should remain only as compatibility until protocol-level tools/list and tools/call parity is proved. Do not delete server.ts until that parity coverage exists, with special attention to text_completion behavior.
  • Workbench BaseApi atoms can move toward AtomRpc or AtomHttpApi after the client API is less Promise-first.
  • MCP env is now Config-backed; continue that policy for future MCP settings.
  • Workbench persistent theme storage is now canonicalized through Atom.kvs/BrowserKeyValueStore. Remaining workbench localStorage matches are legacy migration fallbacks, QA assertions, or the pre-paint host script.
• Flow stateful services:
  • Config service, KnowledgeCore service, FlowManager, and Librarian ref-backed state slices are complete. Follow-up service work should focus on scoped layers, schedules where polling semantics allow, and managed persistence providers rather than direct mutable service fields.
  • Flow service startup facades now consistently use ManagedRuntime, and local scripts should delegate to runMain() instead of adding local .catch(console.error/process.exit) wrappers.
  • Persistence IO should move toward FileSystem or KeyValueStore where the installed beta has the needed provider surface.
• Base messaging/processors:
  • Processor/flow Promise compatibility now uses ManagedRuntime; keep NodeRuntime only for process runMain() entrypoints.
  • Subscriber queues/maps and dynamic flow state should continue moving toward Queue, Deferred, SynchronizedRef, Schedule, and scoped layers.
  • The legacy messaging/subscriber.ts async queue/fanout implementation is removed. Use native effect/PubSub for future in-process fanout, while keeping PubSubBackend for broker-backed messaging.
  • The legacy producer and consumer facades now delegate to scoped Effect runtime factories. Public start()/send()/stop() Promises remain compatibility boundaries.
  • NATS header construction, ack/nak operations, and lookup create-on-missing behavior now stay typed. PubSubBackend remains an intentional broker adapter contract rather than a direct effect/PubSub replacement target.
  • Consumer rate-limit retry timeout behavior is now wired in both legacy and Effect-native consumer paths. Effect-native consumer concurrency now owns one backend consumer per worker, and request-response pending shutdown now fails through a tagged lifecycle error. The legacy consumer facade blocking shape is complete; do not reopen it unless a public API compatibility issue appears.
  • Existing constructor shims preserve callable-plus-newable public exports; removing them needs a public API split or real class redesign.
  • Typed string registries in Flow now have Schema-backed parameter specs and typed producer/requestor spec-object accessors. New service handlers should hoist spec objects and use those accessors; bare string accessors remain compatibility escapes.
  • Base metrics are now Effect-native and Prometheus-formatted through PrometheusMetrics.format; do not reopen prom-client unless a future scrape requirement cannot be represented by Effect metrics.
  • Numeric public timeout fields such as timeoutMs remain compatibility surfaces. Internal runtime config with Config.number(...Ms) is still a valid Config.duration / Duration cleanup target.
• Gateway/client:
  • EffectRpcClient now owns its socket/RPC layer with ManagedRuntime. Socket errors/JSON parsing now use tagged errors and Schema decoding. The remaining client newableFactory assertions are documented as public API compatibility boundaries for this loop.
  • Gateway/client DispatchError contracts now use S.TaggedErrorClass; do not reopen S.ErrorClass unless a new production match appears.
  • Gateway DispatchStream now uses Effect-native dispatcher streaming callbacks instead of nested Effect.runPromiseWith, and client streaming facade callbacks now decode the legacy envelope through Schema before applying service-specific public callback semantics.
  • Gateway dispatcher ownership and serialization cleanup is complete: injected pubsub backends are not closed by the manager, one-shot producers are acquire/use/release bracketed, and serialization failures are typed Effect errors.
  • Do not make gateway/rpc-protocol.ts the next cleanup target: it is a Fastify socket compatibility bridge while the public Effect RPC server layers require SocketServer or Effect HTTP routing.
  • WebSocket adapter host fallbacks now use Result.try and tagged adapter errors while preserving sync exports.
• RAG/providers/storage:
  • RAG and agent requestor bridges are complete: toPromiseRequestor has no remaining ts/packages matches.
  • Provider SDKs and storage clients should become managed resources where they have meaningful lifecycle.
  • Ollama embeddings now has an effectful canonical layer. There is no installed Effect AI Ollama provider package, so future Ollama work should focus on local Effect wrappers/adapters rather than provider replacement.
  • Full text-completion provider swaps need parity tests first. OpenAI and Azure currently use Chat Completions while @effect/ai-openai is Responses API oriented, and no installed Azure/Mistral/Ollama Effect AI provider is available. Anthropic is the closest direct provider swap, but must preserve text, token counts, streaming final usage, and rate-limit mapping. The local provider layer-construction cleanup is complete; remaining provider work is adapter/parity work, not Layer.succeed cleanup.
  • The effect/unstable/ai/LanguageModel to TrustGraph LlmProvider adapter baseline is complete, and Claude now uses @effect/ai-anthropic through that adapter. Direct OpenAI, Azure, and OpenAI-compatible swaps are no-ops until Responses-vs-Chat-Completions parity is proven.
  • FalkorDB scoped lifecycle is complete for triples query/store. Use the fakeable client/graph factory pattern from that slice for future storage client tests.
  • Qdrant config/schema/fakeability is complete for graph/doc embedding store/query modules. Qdrant still has no close/disconnect surface in the installed client, so do not reopen it as an acquireRelease close slice without new SDK evidence.
  • Shared text-completion stream iteration and the Mistral content assertion are complete, and the Effect AI stream-part adapter now uses effect/Match. The remaining provider-layer item is parity-backed Effect AI adapter work, not a direct SDK swap.
• Scratch-note follow-ups:
  • Term / compact client term serialization is complete for base schema, gateway translation, and pure term helper switches. Future work should only reopen this if client socket schema drift appears or a hidden consumer needs a different named-graph shape.
  • Messaging runtime Config.duration cleanup is complete. Internal runtime config uses Duration.Duration; public timeout compatibility inputs and broker receive/error payload boundaries remain numeric milliseconds.
  • Qdrant graph/doc known-collection caches now use MutableHashSet<string>. Short-lived local traversal sets remain no-ops.
  • Gateway dispatcher static service registries, streaming membership, and scoped requestor cache now use Effect HashMap/HashSet; gateway term-bearing service membership sets now use Effect HashSet too.
  • FlowManager () => Effect.gen(...) factories are normalized to Effect.fn / Effect.fnUntraced. Sibling service factories still need a focused scan before treating them as valid migration targets.
  • ConfigService and KnowledgeCore operation dispatch now use effect/Match with Match.exhaustive; FlowManager and Librarian operation dispatch now use effect/Match with runtime-preserving Match.orElse fallbacks.
  • Native switch statements are now clean in ts/packages; future branch drift should keep service dispatch on effect/Match or Schema tagged-union helpers.
  • Client RPC/BaseApi connection-state fanout now uses effect/SubscriptionRef; remaining gateway/client P1 work is broader API design, not listener bookkeeping.
  • Long-lived Map / Set state in ref-backed services can move toward Effect collections later; local pure traversal maps/sets remain no-ops.

Ranked Findings

No-op: Broker Backend Adapter Boundary

• Status:
  • Closed as a P0 rewrite item after the 2026-06-02 broker scout and the legacy consumer facade slice.
• TrustGraph evidence:
  • ts/packages/base/src/backend/types.ts
  • ts/packages/base/src/backend/nats.ts
  • ts/packages/base/src/backend/pubsub.ts
  • ts/packages/base/src/messaging/runtime.ts
• Effect primitives:
  • Layer, Scope, Stream, Schedule, Queue, Effect.acquireRelease, and Effect.tryPromise.
• Evidence:
  • BackendProducer, BackendConsumer, and PubSubBackend are the external Promise broker adapter contract. backend/pubsub.ts wraps that contract in Effect through Context.Service, Layer, Effect.tryPromise, and scoped finalizers.
  • NATS boundary failures, selective stream/consumer lookup recovery, producer sends, ack/nak, and close paths are typed with Effect wrappers.
  • Producer, consumer, and request-response runtime ownership now live in scoped Effect factories.
• Rule:
  • Keep PubSubBackend as the compatibility adapter boundary; Effect native PubSub remains in-process only.
  • Treat the producer Promise facade as a completed compatibility wrapper; avoid reopening it unless backend runtime changes require a narrower adapter.
  • Keep NATS SDK boundary failures typed and avoid catch-all create-on-failure behavior. Future backend slices should move connection/stream state into scoped Effect services.
  • Treat rate-limit retry timeout semantics as complete; next consumer slices should focus on blocking compatibility and backend/layer ownership, not retry policy.
  • Treat Effect-native per-worker consumer ownership as complete; do not flag makeEffectConsumerFromPubSub concurrency for shared backend receive handles.
  • Treat request-response pending shutdown semantics as complete; do not flag waitForResponse timeout behavior for stopped runtimes.
  • Treat request-response in-process fanout as complete: response routing now uses native effect/PubSub subscriptions instead of a hand-managed subscriber map.
  • Treat the legacy consumer facade as a completed compatibility wrapper over makeEffectConsumerFromPubSub; do not flag blocking start() semantics.

No-op: Remaining Effect AI Provider Swaps

• TrustGraph evidence:
  • ts/packages/flow/src/model/text-completion/*.ts
• Effect primitives:
  • effect/unstable/ai/LanguageModel, effect/unstable/ai/EmbeddingModel, Effect AI OpenAI/Anthropic provider layers.
• Rewrite shape:
  • Adapter baseline is complete: makeLanguageModelProvider bridges LanguageModel into LlmProvider.
  • Claude migration is complete through @effect/ai-anthropic.
  • Do not directly swap OpenAI, Azure, or OpenAI-compatible providers yet: current TrustGraph code uses Chat Completions/local-server semantics while @effect/ai-openai is Responses API backed.
  • Do not directly swap Mistral or Ollama until an installed Effect provider package exists or a parity-backed local Effect wrapper is designed.
• Tests:
  • Provider parity for LlmResult, final streaming chunk token counts, 429 mapping, missing-token config failures, and OpenAI-compatible local-server behavior.

No-op: Base Metrics Prometheus Wrapper

• Status:
  • Closed as a scratch-note migration target by the Effect Metrics Prometheus slice.
• TrustGraph evidence:
  • ts/packages/base/src/metrics/prometheus.ts
  • ts/packages/flow/src/gateway/server.ts
• Effect primitives:
  • Metric.counter, Metric.histogram, and effect/unstable/observability PrometheusMetrics.format.
• Rule:
  • Keep the gateway Fastify route as the external scrape boundary, but record TrustGraph metrics through Effect Metric values.
  • Use a fresh Metric.MetricRegistry in tests that assert exact scrape content.

Complete: Term And ClientTerm Tagged-Union Normalization

• TrustGraph evidence:
  • ts/packages/base/src/schema/primitives.ts
  • ts/packages/flow/src/gateway/dispatch/serialize.ts
  • ts/packages/client/src/socket/trustgraph-socket.ts
• Effect primitives:
  • S.toTaggedUnion(...).match and effect/Match discriminator helpers.
• Rewrite shape:
  • Add tagged-union helpers for internal Term and compact client terms.
  • Replace serializer native switches with tagged-union matching or Match.discriminatorsExhaustive.
  • Remove unsafe default pass-through casts while preserving compact g string compatibility.
• Tests:
  • Base schema tests now cover recursive terms and graph strings.
  • Gateway dispatcher tests now cover all compact term variants, nested triples, compact graph strings, malformed known-tag payloads, and malformed client triple failures.

Complete: Messaging Runtime Duration Config Cleanup

• TrustGraph evidence:
  • ts/packages/base/src/runtime/messaging-config.ts
  • ts/packages/base/src/messaging/runtime.ts
• Effect primitives:
  • Config.duration, Duration.Duration, and existing Duration.millis compatibility conversions.
• Rewrite shape:
  • Change internal runtime config fields such as receiveTimeoutMs, requestTimeoutMs, retryDelayMs, and rateLimitTimeoutMs to Duration.Duration.
  • Load env-backed values with Config.duration while preserving Python-style millisecond defaults and public numeric compatibility options.
  • Keep external timeoutMs option names numeric in request/response, processor, and client boundaries unless their public API is deliberately changed.
• Tests:
  • Base runtime config tests cover legacy millisecond env values and Effect duration string env values.
  • Messaging runtime and consumer tests preserve retry and timeout behavior.

Complete: Qdrant Known-Collection MutableHashSet Cleanup

• TrustGraph evidence:
  • ts/packages/flow/src/storage/embeddings/qdrant-doc.ts
  • ts/packages/flow/src/storage/embeddings/qdrant-graph.ts
• Effect primitives:
  • MutableHashSet from effect.
• Rewrite shape:
  • Replace long-lived Set<string> known-collection caches with MutableHashSet<string> in Qdrant graph/doc embedding stores.
  • Keep short-lived local Set values for pure query traversal or fixture assertions as no-op boundaries.
• Tests:
  • Qdrant embeddings tests prove graph cache hits skip repeated collection existence checks and deletion invalidates the cache.

P2: Canonicalize MCP Around The Effect Server

• Status:
  • MCP now builds under strict tsgo, the stdio server has an Effect-backed compatibility implementation, and an Effect McpServer.layerStdio entrypoint exists.
• Remaining shape:
  • Keep the old SDK/Zod stdio compatibility surface for now.
  • Prove tools/list and tools/call parity before deleting any public entry point or dropping zod/server-side MCP SDK dependencies.
  • Pay special attention to text_completion: legacy calls the TrustGraph gateway, while the Effect server currently uses an Effect AI LanguageModel/OpenAI layer.
• Tests:
  • cd ts && bun run --cwd packages/mcp build
  • Root cd ts && bun run check

No-op: Workbench Theme And Browser Storage Boundaries

• Status: migrated and documented no-op for remaining direct browser matches.
• Evidence:
  • settingsAtom, themeAtom, flowIdAtom, and conversationAtom are canonical Atom.kvs entries backed by BrowserKeyValueStore.layerLocalStorage.
  • The only remaining tg-theme production read is a legacy migration fallback. The pre-paint index.html script is a host boundary that restores the canonical key before React and Effect services mount.
  • Graph canvas DOM/class inspection remains render-only host behavior.
• Rule:
  • Do not reopen workbench localStorage as a backend/runtime blocker unless a canonical Atom.kvs entry is bypassed or a legacy migration fallback can be intentionally removed.

Recommended PR Order

1. MCP protocol parity tests and legacy stdio flip/removal decision.
2. Flow/client RPC stream API design beyond callback/Promise compatibility.
3. Long-lived ref-backed HashMap state cleanup where clone helpers remain.
4. Sibling service Effect.fn normalization where arrow-returned generators still appear.

No-Op Rules

Do not flag these as rewrite blockers without additional proof:

• Promise-returning CLI actions, MCP SDK callbacks, client compatibility methods, and Fastify route handlers at true external boundaries. Boundary code still must map failures into typed errors or wire errors.
• try/catch blocks at host/tool boundaries only when the catch maps into a typed error or a wire-contract error. Internal exception capture should use Effect.try, Effect.tryPromise, or Result.try.
• S.Class, S.TaggedErrorClass, Context.Service, Rpc.make, and HttpApi.make when they are required or idiomatic for the Effect API.
• Plain Map usage for local pure transformations, such as graph utility construction, unless the state is long-lived mutable service state.
• Plain synchronous loops for parsing or tree traversal are not Effect migration blockers unless they hide async polling, resource ownership, or callback scheduling.
• JSON stringification in tests or wire-contract fixtures. Production JSON encode/decode should prefer schema codecs when the encoded form can be preserved.
• Workbench pre-paint index.html storage reads are host-boundary code used before React and Effect services mount. Keep canonical workbench state in Atom.kvs/BrowserKeyValueStore, and treat old-key reads as migration fallbacks unless backward compatibility is intentionally dropped.
• The old MCP SDK/Zod stdio server is compatibility code until an Effect stdio entrypoint plus parity tests prove the same public protocol behavior.
• Client newableFactory assertions that preserve vendored callable-plus-new API facades are compatibility boundaries unless the public constructor API is intentionally redesigned.
• Base AsyncProcessor, Flow, and FlowProcessor callable-plus-newable export assertions are compatibility boundaries unless the public constructor API is intentionally redesigned.
• TrustGraph PubSubBackend / backend PubSub service is a broker adapter boundary for NATS/Pulsar-style topics, acknowledgement, schema codecs, and backend lifecycle. Effect's native PubSub can replace in-process fanout helpers, but not the distributed broker abstraction by itself. This was rechecked against /home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4/packages/effect/src/PubSub.ts, whose exported API is local publish/subscribe over Effect queues.
• Request-response pending shutdown semantics are complete in makeEffectRequestResponseFromPubSub: pending calls race response waiting against a Deferred stop signal and fail with tagged MessagingLifecycleError.
• Legacy makeConsumer facade blocking-loop ownership is complete: start() now initializes scoped Effect consumers and returns after startup, while stop() closes the native consumer scope.
• ts/packages/flow/src/gateway/rpc-protocol.ts is a Fastify socket compatibility bridge. Do not flag its internal connection maps/sets as a standalone replacement target until the gateway is ready to move onto Effect SocketServer or Effect HTTP routing.

Acceptance For Final Loop Completion

The overall playbook loop is complete only when:

• All remaining playbook signal matches are migrated or documented as no-op external-boundary cases with concrete evidence.
• No P0/P1/P2 migration item remains in this audit.
• cd ts && bun run check, cd ts && bun run build, cd ts && bun run test, and git diff --check pass after the final migration slice.