apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-01 01:19:38 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
trustgraph/ts/EFFECT_NATIVE_REWRITE_AUDIT.md
elpresidank 7f81c56c80 Add Effect-native rewrite audit playbook
2026-06-01 22:00:28 -05:00

12 KiB
Raw Blame History

TrustGraph Effect-Native Rewrite Opportunity Audit

This is the first ranked audit produced from the playbook in ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md. It is an opportunity map, not a code rewrite. The branch was ts-port-effect-v4; the only unrelated local file seen during the audit was .idea/effect.intellij.xml.

Inputs

Verified source roots:

  • TrustGraph TS port: /home/elpresidank/YeeBois/dev/trustgraph/ts
  • Effect v4 subtree: /home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4
  • Reactivity fallback: ts/node_modules/effect/src/unstable/reactivity
  • Atom React fallback: ts/packages/workbench/node_modules/@effect/atom-react

Signal counts from ts/packages:

Signal Count
Effect.runPromise 71
Map< 54
JSON.stringify 50
WebSocket 45
process.env 44
new Map 42
toPromiseRequestor 19
makeAsyncProcessor 19
new Promise 18
JSON.parse 16
receive( 16
setTimeout 13
while ( 10
localStorage 8

Ranked Findings

P0: Collapse Base Messaging Promise Facades

  • Impact: 5
  • Risk: 4
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/base/src/messaging/runtime.ts already defines Effect producer, consumer, request/response factories, queues, fibers, and scopes.
    • ts/packages/base/src/messaging/consumer.ts still has a manual while (running) receive loop, sleep, and Promise delay helpers.
    • ts/packages/base/src/messaging/subscriber.ts still manages resolver maps and timeout promises.
    • ts/packages/base/src/processor/flow.ts exposes compatibility scope helpers and converts Effect handles back into Promise-style handles.
  • Effect evidence:
    • effect/Queue, effect/PubSub, effect/Stream, effect/Scope, effect/Layer, effect/Schedule, effect/Ref.
    • Sources: packages/effect/src/Queue.ts, PubSub.ts, Stream.ts, Scope.ts, Layer.ts, Schedule.ts, Ref.ts.
  • Rewrite shape:
    • Make the Effect runtime factories the canonical internal surface.
    • Keep Promise adapters only at external compatibility boundaries.
    • Replace polling sleep loops with scheduled scoped consumers where possible.
    • Replace resolver maps with Queue, Deferred, or PubSub-backed routing.
  • Tests:
    • cd ts && bun run --cwd packages/base test
    • Existing runtime tests around request/response, flow specs, and consumers should be expanded before removing compatibility behavior.
  • Blockers:
    • Public package exports may still expect Promise-shaped producer, consumer, and request/response handles. Inventory callers before changing exports.

P0: Convert Stateful Flow Services To Scoped Effect Services

  • Impact: 5
  • Risk: 4
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/flow/src/config/service.ts uses makeAsyncProcessor, mutable nested Map state, while (this.running), receive(2000), sleep, JSON persistence, and direct process.env.
    • ts/packages/flow/src/librarian/service.ts, cores/service.ts, and flow-manager/service.ts repeat the same service-object pattern.
  • Effect evidence:
    • Context, Layer.scoped, Ref, SynchronizedRef, Schedule, Effect.addFinalizer, Config, Schema, effect/FileSystem, effect/unstable/persistence/KeyValueStore.
    • Sources: packages/effect/src/Context.ts, Layer.ts, Ref.ts, SynchronizedRef.ts, Schedule.ts, Config.ts, Schema.ts, ts/node_modules/effect/src/FileSystem.ts, ts/node_modules/effect/src/unstable/persistence/KeyValueStore.ts.
  • Rewrite shape:
    • Model each service as a Context service plus a scoped layer.
    • Store service state in Ref or SynchronizedRef, not mutable object fields.
    • Express persistence with effect/FileSystem or KeyValueStore.layerFileSystem when the installed beta exposes the needed provider.
    • Decode persisted payloads and config with schemas at boundaries.
  • Tests:
    • Service-specific tests plus cd ts && bun run --cwd packages/flow test.
    • Add persistence round-trip tests before replacing file IO.
  • Blockers:
    • These services are behavior-heavy. Do one service per PR after the shared runtime surface is stable.

P0: Make Gateway Dispatcher Effect-Native

  • Impact: 5
  • Risk: 3
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/flow/src/gateway/server.ts already builds RPC/WebSocket pieces with Effect.
    • ts/packages/flow/src/gateway/rpc-server.ts uses Queue and RPC layers.
    • ts/packages/flow/src/gateway/dispatch/manager.ts still keeps Map<string, Promise<RequestResponse<unknown, unknown>>>, manual streaming completion checks, and per-publish producer construction.
  • Effect evidence:
    • effect/unstable/rpc RpcClient, RpcServer, RpcSerialization.
    • effect/unstable/socket Socket.
    • effect/Queue, Stream, Scope, Layer.
    • Sources: ts/node_modules/effect/src/unstable/rpc/RpcClient.ts, RpcServer.ts, RpcSerialization.ts, and ts/node_modules/effect/src/unstable/socket/Socket.ts.
  • Rewrite shape:
    • Convert dispatcher manager methods to Effect-returning functions internally.
    • Cache requestors as scoped resources instead of Promise values.
    • Represent streaming dispatch as Stream or Queue instead of callback completion detection where the wire protocol allows it.
    • Keep Fastify route handlers as Promise boundaries.
  • Tests:
    • Gateway dispatch tests with fake pubsub.
    • cd ts && SKIP_LLM=1 bun run test:pipeline after implementation.
  • Blockers:
    • The gateway is an integration boundary. Preserve current HTTP and WebSocket wire behavior during the first rewrite.

P1: Remove RAG And Agent toPromiseRequestor Bridges

  • Impact: 4
  • Risk: 3
  • Confidence: 5
  • TrustGraph evidence:
    • ts/packages/flow/src/retrieval/document-rag-service.ts
    • ts/packages/flow/src/retrieval/graph-rag-service.ts
    • ts/packages/flow/src/agent/react/service.ts
    • All define toPromiseRequestor and then immediately adapt Effect requestors back to Promise-style clients.
  • Effect evidence:
    • Existing TrustGraph EffectRequestResponse in ts/packages/base/src/messaging/runtime.ts.
    • effect/Stream, Effect.fn, Effect.runPromiseWith for boundary-only execution.
  • Rewrite shape:
    • Update RAG engines and agent helpers to accept Effect requestors or functions returning Effect.
    • Keep Promise wrappers only for old public APIs or tests that explicitly verify compatibility.
    • Convert streaming agent flows to Stream where possible.
  • Tests:
    • Existing RAG and agent service tests.
    • Add tests that assert requestor errors stay typed through the Effect path.
  • Blockers:
    • Engine call signatures need a small design pass so RAG and agent rewrite in the same direction.

P1: Finish Client RPC Boundary Modernization

  • Impact: 4
  • Risk: 3
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/client/src/socket/effect-rpc-client.ts already uses Socket.makeWebSocket, RpcClient.layerProtocolSocket, and RpcSerialization.layerNdjson.
    • The same file still owns scopePromise, clientPromise, repeated Effect.runPromise, listener sets, a WebSocket constructor shim, and a Promise facade.
    • ts/packages/client/src/socket/trustgraph-socket.ts is mostly a compatibility API over the Effect RPC client.
  • Effect evidence:
    • effect/unstable/socket/Socket: makeWebSocket, fromWebSocket, toChannel, layerWebSocket.
    • effect/unstable/rpc/RpcClient: layerProtocolSocket.
    • effect/unstable/rpc/RpcSerialization: layerNdjson, layerNdJsonRpc.
  • Rewrite shape:
    • Treat EffectRpcClient as an internal managed runtime or scoped layer.
    • Expose Promise-returning methods only through a thin compatibility adapter.
    • Move browser vs Node WebSocket constructor selection into platform layers.
  • Tests:
    • cd ts && bun run --cwd packages/client test
    • Keep timeout/retry tests around withDispatchRequestPolicy.
  • Blockers:
    • Workbench and CLI still consume Promise-shaped client APIs.

P1: Make SDK, Storage, And Provider Layers Managed Resources

  • Impact: 4
  • Risk: 3
  • Confidence: 3
  • TrustGraph evidence:
    • ts/packages/flow/src/storage/triples/falkordb.ts
    • ts/packages/flow/src/storage/embeddings/qdrant-graph.ts
    • ts/packages/flow/src/storage/embeddings/qdrant-doc.ts
    • ts/packages/flow/src/model/text-completion/*.ts
    • These files create direct SDK clients and read process.env in live constructors.
  • Effect evidence:
    • Effect.acquireRelease, Layer.scoped, Config, ConfigProvider, effect/FileSystem, effect/unstable/persistence/KeyValueStore, Metric, Logger.
    • AI provider modules from installed provider packages, with subtree source proof under packages/ai/*/src, including OpenAiLanguageModel.ts, AnthropicLanguageModel.ts, and OpenRouterLanguageModel.ts.
  • Rewrite shape:
    • Move env reading into Config loaders and provider-specific layers.
    • Scope SDK clients that need explicit close/disconnect.
    • Replace console or ad hoc logging with Effect.log* and metrics where useful.
  • Tests:
    • Provider config tests with ConfigProvider.fromMap.
    • Storage tests with fake clients before changing real resource lifetimes.
  • Blockers:
    • Some third-party SDK clients may not have meaningful finalizers. Mark those no-op after proof instead of forcing fake lifecycle code.

P2: Canonicalize MCP Around The Effect Server

  • Impact: 3
  • Risk: 2
  • Confidence: 5
  • TrustGraph evidence:
    • ts/packages/mcp/src/server.ts is the old SDK/Zod server.
    • ts/packages/mcp/src/server-effect.ts has Effect AI tools, schemas, McpServer, HTTP API integration, and provider layers.
  • Effect evidence:
    • effect/unstable/ai Tool, Toolkit, McpServer, McpSchema, LanguageModel.
    • Sources: ts/node_modules/effect/src/unstable/ai/Tool.ts, Toolkit.ts, McpServer.ts, McpSchema.ts, LanguageModel.ts.
  • Rewrite shape:
    • Do not rewrite the Effect server from scratch.
    • Make the Effect server canonical after parity checks.
    • Keep the old server only as compatibility or delete it once entrypoints and tests prove the Effect path is complete.
  • Tests:
    • MCP package build/test.
    • Tool parity diff against server.ts before removal.
  • Blockers:
    • Needs a policy decision about old SDK server lifetime.

P2: Tighten Workbench Platform And Reactivity Usage

  • Impact: 3
  • Risk: 2
  • Confidence: 4
  • TrustGraph evidence:
    • ts/packages/workbench/src/atoms/workbench.ts already uses Atom, AsyncResult, Reactivity, browser layers, and metrics.
    • Remaining direct browser state includes localStorage reads/writes and DOM theme inspection.
  • Effect evidence:
    • BrowserKeyValueStore.layerLocalStorage, BrowserKeyValueStore.layerSessionStorage, BrowserHttpClient, Clipboard.
    • AtomRpc, AtomHttpApi, AtomRegistry, AsyncResult, Reactivity.
  • Rewrite shape:
    • Leave the workbench out of the first rewrite wave.
    • Later, move persistent UI state through BrowserKeyValueStore and keep remote state in Atom RPC/HTTP API families if the client API becomes fully typed Effect RPC.
  • Tests:
    • cd ts && bun run workbench:qa.
  • Blockers:
    • Workbench is already the most modern surface. Backend/runtime wins should happen first.

Recommended PR Order

  1. Base messaging/runtime convergence design and tests.
  2. Gateway dispatcher internal Effect conversion.
  3. RAG and agent requestor bridge removal.
  4. One stateful Flow service conversion, starting with config or cores.
  5. Client compatibility facade tightening.
  6. Storage/provider managed resource cleanup.
  7. MCP canonicalization and Workbench polish.

No-Op Rules

Do not flag these as rewrite blockers without additional proof:

  • Promise-returning CLI actions and Fastify route handlers at external boundaries.
  • S.Class, S.TaggedErrorClass, Context.Service, Rpc.make, and HttpApi.make when they are required or idiomatic for the Effect API.
  • Plain Map usage for local pure transformations, such as graph utility construction, unless the state is long-lived, mutable service state.
  • JSON stringification that is part of the TrustGraph wire contract, unless a schema codec can preserve the exact encoded form.

Acceptance

This audit is complete when:

  • ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md exists.
  • This ranked audit exists and cites concrete TrustGraph and Effect surfaces.
  • git diff --check passes for both files.
  • No code rewrite is mixed into this audit.