apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-01 01:19:38 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
trustgraph/ts/EFFECT_NATIVE_REWRITE_AUDIT.md

14 KiB
Raw Blame History

TrustGraph Effect-Native Rewrite Opportunity Audit

This is the first ranked audit produced from the playbook in ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md. It is an opportunity map, not a code rewrite. The branch was ts-port-effect-v4; the only unrelated local file seen during the audit was .idea/effect.intellij.xml.

Inputs

Verified source roots:

  • TrustGraph TS port: /home/elpresidank/YeeBois/dev/trustgraph/ts
  • Effect v4 subtree: /home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4
  • Reactivity fallback: ts/node_modules/effect/src/unstable/reactivity
  • Atom React fallback: ts/packages/workbench/node_modules/@effect/atom-react

Signal counts from ts/packages:

Signal Count
Effect.runPromise 71
Map< 54
JSON.stringify 50
WebSocket 45
process.env 44
new Map 42
toPromiseRequestor 19
makeAsyncProcessor 19
new Promise 18
JSON.parse 16
receive( 16
setTimeout 13
while ( 10
localStorage 8

Loop Passes

2026-06-02: Base Request/Response Facade

  • Status: migrated and verified.
  • Completed:
    • ts/packages/base/src/messaging/request-response.ts:50 now creates an explicit Scope.Closeable and :55 builds the existing EffectRequestResponse runtime.
    • ts/packages/base/src/messaging/request-response.ts:91 rejects not-started calls with MessagingLifecycleError, and :108 maps recipient callback failures into MessagingDeliveryError. It no longer constructs normal Error values.
    • ts/packages/base/src/messaging/runtime.ts:427 now lets request/response own its producer directly, :442 runs the response dispatcher with Effect.forkScoped, and :445 makes shutdown idempotent.
    • ts/packages/base/src/__tests__/request-response.test.ts:115 covers the Promise facade over the Effect runtime, :143 asserts tagged timeout errors, and :164 asserts tagged lifecycle errors.
  • Verification:
    • bun run --cwd ts/packages/base test
    • bun run --cwd ts/packages/base build
    • bun run --cwd ts check:tsgo
    • bun run --cwd ts build
    • bun run --cwd ts test
  • Remaining base evidence:
    • makeSubscriber( has no current ts/packages call sites after this slice, but ts/packages/base/src/messaging/index.ts still exports makeAsyncQueue, makeSubscriber, and related types.
    • ts/packages/base/src/messaging/consumer.ts still has a Promise polling loop and a normal Error constructor.
    • ts/packages/base/src/messaging/producer.ts still throws a normal not-started Error.
  • Decision:
    • Normal Error construction in library internals is migration evidence. Prefer existing S.TaggedErrorClass errors from ts/packages/base/src/errors.ts, adding new tagged errors when needed.

Ranked Findings

P0: Collapse Base Messaging Promise Facades

  • Impact: 5
  • Risk: 4
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/base/src/messaging/runtime.ts already defines Effect producer, consumer, request/response factories, queues, fibers, and scopes.
    • ts/packages/base/src/messaging/consumer.ts still has a manual while (running) receive loop, sleep, and Promise delay helpers.
    • ts/packages/base/src/messaging/subscriber.ts still manages resolver maps and timeout promises.
    • ts/packages/base/src/processor/flow.ts exposes compatibility scope helpers and converts Effect handles back into Promise-style handles.
  • Effect evidence:
    • effect/Queue, effect/PubSub, effect/Stream, effect/Scope, effect/Layer, effect/Schedule, effect/Ref.
    • Sources: packages/effect/src/Queue.ts, PubSub.ts, Stream.ts, Scope.ts, Layer.ts, Schedule.ts, Ref.ts.
  • Rewrite shape:
    • Make the Effect runtime factories the canonical internal surface.
    • Keep Promise adapters only at external compatibility boundaries. Rejected values at those boundaries should still be tagged TrustGraph errors.
    • Replace polling sleep loops with scheduled scoped consumers where possible.
    • Replace resolver maps with Queue, Deferred, or PubSub-backed routing.
  • Tests:
    • cd ts && bun run --cwd packages/base test
    • Existing runtime tests around request/response, flow specs, and consumers should be expanded before removing compatibility behavior.
  • Blockers:
    • Public package exports may still expect Promise-shaped producer, consumer, and request/response handles. Inventory callers before changing exports.
    • First slice completed request/response facade migration. Next base follow-up is either an Effect-backed consumer facade or a public export decision for subscriber.ts.

P0: Convert Stateful Flow Services To Scoped Effect Services

  • Impact: 5
  • Risk: 4
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/flow/src/config/service.ts uses makeAsyncProcessor, mutable nested Map state, while (this.running), receive(2000), sleep, JSON persistence, and direct process.env.
    • ts/packages/flow/src/librarian/service.ts, cores/service.ts, and flow-manager/service.ts repeat the same service-object pattern.
  • Effect evidence:
    • Context, Layer.scoped, Ref, SynchronizedRef, Schedule, Effect.addFinalizer, Config, Schema, effect/FileSystem, effect/unstable/persistence/KeyValueStore.
    • Sources: packages/effect/src/Context.ts, Layer.ts, Ref.ts, SynchronizedRef.ts, Schedule.ts, Config.ts, Schema.ts, ts/node_modules/effect/src/FileSystem.ts, ts/node_modules/effect/src/unstable/persistence/KeyValueStore.ts.
  • Rewrite shape:
    • Model each service as a Context service plus a scoped layer.
    • Store service state in Ref or SynchronizedRef, not mutable object fields.
    • Express persistence with effect/FileSystem or KeyValueStore.layerFileSystem when the installed beta exposes the needed provider.
    • Decode persisted payloads and config with schemas at boundaries.
  • Tests:
    • Service-specific tests plus cd ts && bun run --cwd packages/flow test.
    • Add persistence round-trip tests before replacing file IO.
  • Blockers:
    • These services are behavior-heavy. Do one service per PR after the shared runtime surface is stable.

P0: Make Gateway Dispatcher Effect-Native

  • Impact: 5
  • Risk: 3
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/flow/src/gateway/server.ts already builds RPC/WebSocket pieces with Effect.
    • ts/packages/flow/src/gateway/rpc-server.ts uses Queue and RPC layers.
    • ts/packages/flow/src/gateway/dispatch/manager.ts still keeps Map<string, Promise<RequestResponse<unknown, unknown>>>, manual streaming completion checks, and per-publish producer construction.
  • Effect evidence:
    • effect/unstable/rpc RpcClient, RpcServer, RpcSerialization.
    • effect/unstable/socket Socket.
    • effect/Queue, Stream, Scope, Layer.
    • Sources: ts/node_modules/effect/src/unstable/rpc/RpcClient.ts, RpcServer.ts, RpcSerialization.ts, and ts/node_modules/effect/src/unstable/socket/Socket.ts.
  • Rewrite shape:
    • Convert dispatcher manager methods to Effect-returning functions internally.
    • Cache requestors as scoped resources instead of Promise values.
    • Represent streaming dispatch as Stream or Queue instead of callback completion detection where the wire protocol allows it.
    • Keep Fastify route handlers as Promise boundaries.
  • Tests:
    • Gateway dispatch tests with fake pubsub.
    • cd ts && SKIP_LLM=1 bun run test:pipeline after implementation.
  • Blockers:
    • The gateway is an integration boundary. Preserve current HTTP and WebSocket wire behavior during the first rewrite.

P1: Remove RAG And Agent toPromiseRequestor Bridges

  • Impact: 4
  • Risk: 3
  • Confidence: 5
  • TrustGraph evidence:
    • ts/packages/flow/src/retrieval/document-rag-service.ts
    • ts/packages/flow/src/retrieval/graph-rag-service.ts
    • ts/packages/flow/src/agent/react/service.ts
    • All define toPromiseRequestor and then immediately adapt Effect requestors back to Promise-style clients.
  • Effect evidence:
    • Existing TrustGraph EffectRequestResponse in ts/packages/base/src/messaging/runtime.ts.
    • effect/Stream, Effect.fn, Effect.runPromiseWith for boundary-only execution.
  • Rewrite shape:
    • Update RAG engines and agent helpers to accept Effect requestors or functions returning Effect.
    • Keep Promise wrappers only for old public APIs or tests that explicitly verify compatibility.
    • Convert streaming agent flows to Stream where possible.
  • Tests:
    • Existing RAG and agent service tests.
    • Add tests that assert requestor errors stay typed through the Effect path.
  • Blockers:
    • Engine call signatures need a small design pass so RAG and agent rewrite in the same direction.

P1: Finish Client RPC Boundary Modernization

  • Impact: 4
  • Risk: 3
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/client/src/socket/effect-rpc-client.ts already uses Socket.makeWebSocket, RpcClient.layerProtocolSocket, and RpcSerialization.layerNdjson.
    • The same file still owns scopePromise, clientPromise, repeated Effect.runPromise, listener sets, a WebSocket constructor shim, and a Promise facade.
    • ts/packages/client/src/socket/trustgraph-socket.ts is mostly a compatibility API over the Effect RPC client.
  • Effect evidence:
    • effect/unstable/socket/Socket: makeWebSocket, fromWebSocket, toChannel, layerWebSocket.
    • effect/unstable/rpc/RpcClient: layerProtocolSocket.
    • effect/unstable/rpc/RpcSerialization: layerNdjson, layerNdJsonRpc.
  • Rewrite shape:
    • Treat EffectRpcClient as an internal managed runtime or scoped layer.
    • Expose Promise-returning methods only through a thin compatibility adapter.
    • Move browser vs Node WebSocket constructor selection into platform layers.
  • Tests:
    • cd ts && bun run --cwd packages/client test
    • Keep timeout/retry tests around withDispatchRequestPolicy.
  • Blockers:
    • Workbench and CLI still consume Promise-shaped client APIs.

P1: Make SDK, Storage, And Provider Layers Managed Resources

  • Impact: 4
  • Risk: 3
  • Confidence: 3
  • TrustGraph evidence:
    • ts/packages/flow/src/storage/triples/falkordb.ts
    • ts/packages/flow/src/storage/embeddings/qdrant-graph.ts
    • ts/packages/flow/src/storage/embeddings/qdrant-doc.ts
    • ts/packages/flow/src/model/text-completion/*.ts
    • These files create direct SDK clients and read process.env in live constructors.
  • Effect evidence:
    • Effect.acquireRelease, Layer.scoped, Config, ConfigProvider, effect/FileSystem, effect/unstable/persistence/KeyValueStore, Metric, Logger.
    • AI provider modules from installed provider packages, with subtree source proof under packages/ai/*/src, including OpenAiLanguageModel.ts, AnthropicLanguageModel.ts, and OpenRouterLanguageModel.ts.
  • Rewrite shape:
    • Move env reading into Config loaders and provider-specific layers.
    • Scope SDK clients that need explicit close/disconnect.
    • Replace console or ad hoc logging with Effect.log* and metrics where useful.
  • Tests:
    • Provider config tests with ConfigProvider.fromMap.
    • Storage tests with fake clients before changing real resource lifetimes.
  • Blockers:
    • Some third-party SDK clients may not have meaningful finalizers. Mark those no-op after proof instead of forcing fake lifecycle code.

P2: Canonicalize MCP Around The Effect Server

  • Impact: 3
  • Risk: 2
  • Confidence: 5
  • TrustGraph evidence:
    • ts/packages/mcp/src/server.ts is the old SDK/Zod server.
    • ts/packages/mcp/src/server-effect.ts has Effect AI tools, schemas, McpServer, HTTP API integration, and provider layers.
  • Effect evidence:
    • effect/unstable/ai Tool, Toolkit, McpServer, McpSchema, LanguageModel.
    • Sources: ts/node_modules/effect/src/unstable/ai/Tool.ts, Toolkit.ts, McpServer.ts, McpSchema.ts, LanguageModel.ts.
  • Rewrite shape:
    • Do not rewrite the Effect server from scratch.
    • Make the Effect server canonical after parity checks.
    • Keep the old server only as compatibility or delete it once entrypoints and tests prove the Effect path is complete.
  • Tests:
    • MCP package build/test.
    • Tool parity diff against server.ts before removal.
  • Blockers:
    • Needs a policy decision about old SDK server lifetime.

P2: Tighten Workbench Platform And Reactivity Usage

  • Impact: 3
  • Risk: 2
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/workbench/src/atoms/workbench.ts already uses Atom, AsyncResult, Reactivity, browser layers, and metrics.
    • Remaining direct browser state includes localStorage reads/writes and DOM theme inspection.
  • Effect evidence:
    • BrowserKeyValueStore.layerLocalStorage, BrowserKeyValueStore.layerSessionStorage, BrowserHttpClient, Clipboard.
    • AtomRpc, AtomHttpApi, AtomRegistry, AsyncResult, Reactivity.
  • Rewrite shape:
    • Leave the workbench out of the first rewrite wave.
    • Later, move persistent UI state through BrowserKeyValueStore and keep remote state in Atom RPC/HTTP API families if the client API becomes fully typed Effect RPC.
  • Tests:
    • cd ts && bun run workbench:qa.
  • Blockers:
    • Workbench is already the most modern surface. Backend/runtime wins should happen first.

Recommended PR Order

  1. Gateway dispatcher requestor-cache and streaming-completion migration.
  2. Config service scoped state migration.
  3. RAG and agent requestor bridge removal.
  4. Base consumer facade and subscriber export cleanup.
  5. Client compatibility facade tightening.
  6. Storage/provider managed resource cleanup.
  7. MCP canonicalization and Workbench polish.

No-Op Rules

Do not flag these as rewrite blockers without additional proof:

  • Promise-returning CLI actions and Fastify route handlers at external boundaries. This does not exempt normal Error construction inside shared library code.
  • S.Class, S.TaggedErrorClass, Context.Service, Rpc.make, and HttpApi.make when they are required or idiomatic for the Effect API.
  • Plain Map usage for local pure transformations, such as graph utility construction, unless the state is long-lived, mutable service state.
  • JSON stringification that is part of the TrustGraph wire contract, unless a schema codec can preserve the exact encoded form.

Acceptance

This audit is complete when:

  • ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md exists.
  • This ranked audit exists and cites concrete TrustGraph and Effect surfaces.
  • git diff --check passes for both files.
  • No code rewrite is mixed into this audit.