mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-01 01:19:38 +02:00
chore(ts): consolidate effect native closeout
This commit is contained in:
elpresidank
parent
cd6c9107d7
commit
fab718dce8
21 changed files with 199 additions and 3533 deletions
|
|
@ -1,151 +0,0 @@
|
|||
# TrustGraph TS Native Class To Functional Effect Services Goal
|
||||
|
||||
Use the `grill-me` skill from:
|
||||
|
||||
`/home/elpresidank/YeeBois/projects/beep-effect/.agents/skills/grill-me/SKILL.md`
|
||||
|
||||
## Objective
|
||||
|
||||
Refactor the TrustGraph TypeScript port so production runtime code no longer
|
||||
uses native TypeScript or JavaScript class syntax except where an Effect module
|
||||
API truly requires class syntax and no practical functional form exists.
|
||||
|
||||
The preferred style is functional and pure:
|
||||
|
||||
- Functions, factories, closures, and object-literal service implementations.
|
||||
- Functions returning functions or service objects instead of constructors.
|
||||
- `Context.Service` tags plus `Layer` builders for dependency provision.
|
||||
- `Effect.gen`, `Effect.fn`, `Effect.acquireRelease`, `Effect.addFinalizer`,
|
||||
and scoped Layers for lifecycle.
|
||||
- Top-level `Effect.runPromise` only at CLI/bootstrap boundaries.
|
||||
|
||||
Terminal condition: repeated inventory agents find zero blocking native classes
|
||||
in production runtime source like these current smells:
|
||||
|
||||
- `ts/packages/base/src/processor/async-processor.ts#L38-162`
|
||||
- `ts/packages/base/src/processor/flow-processor.ts#L284-358`
|
||||
|
||||
## Class Policy
|
||||
|
||||
Blocking scope is production runtime source under `ts/packages/**/src`,
|
||||
excluding `__tests__`, `*.test.ts`, and `*.spec.ts`.
|
||||
|
||||
Tests and scripts must still be inventoried separately, but they do not block
|
||||
completion unless they are required to verify migrated production behavior.
|
||||
|
||||
Allowed class syntax is intentionally narrow and proof-based. Do not preserve
|
||||
class syntax just because it is Effect-adjacent. For every candidate exemption,
|
||||
prove that the specific Effect API requires class syntax and that a functional
|
||||
alternative is not better.
|
||||
|
||||
Candidate exemptions to investigate, not blindly preserve:
|
||||
|
||||
- `S.Class`, `S.TaggedClass`, `S.TaggedErrorClass`, `S.ErrorClass`
|
||||
- `Data.TaggedError`
|
||||
- `Context.Service`
|
||||
- Effect RPC / HTTP API class forms such as `Rpc.make` and `HttpApi.make`
|
||||
|
||||
If a functional Effect equivalent exists, use the functional form. The target
|
||||
style is functions returning functions or service objects, not class-shaped
|
||||
Effect code.
|
||||
|
||||
Blocking class forms include:
|
||||
|
||||
- Abstract base classes and inheritance-based processors.
|
||||
- Service classes extending `AsyncProcessor`, `FlowProcessor`, or `LlmService`.
|
||||
- Client API wrapper classes.
|
||||
- Backend adapter classes.
|
||||
- Spec/resource classes.
|
||||
- Query/store/engine classes.
|
||||
- Metrics/parser/lifecycle classes.
|
||||
- React class components such as error boundaries.
|
||||
|
||||
## Required Workflow
|
||||
|
||||
1. Read and follow `grill-me`.
|
||||
2. Before asking the user questions, use parallel read-only sub-agents to
|
||||
inventory class declarations.
|
||||
3. Use AST-based inspection where possible; regex alone is not enough.
|
||||
4. Split inventory agents by subsystem:
|
||||
- Base processor/messaging/runtime/specs/backend.
|
||||
- Flow service processors and LLM/embedding/RAG services.
|
||||
- Query/store engines and adapters.
|
||||
- Client socket/API wrappers.
|
||||
- Workbench production source.
|
||||
5. Consolidate findings into:
|
||||
- Blocking production native classes.
|
||||
- Effect-native class forms with proof.
|
||||
- Functional replacements available for prior exemptions.
|
||||
- Non-blocking test/script classes.
|
||||
6. Use `grill-me` to resolve any remaining API or rollout tradeoffs.
|
||||
7. Produce a decision-complete implementation plan.
|
||||
8. After approval, implement in phases and rerun inventory until blocking count
|
||||
is zero.
|
||||
|
||||
## Refactor Direction
|
||||
|
||||
Replace inheritance and mutable class instances with:
|
||||
|
||||
- Plain TypeScript service interfaces.
|
||||
- `Context.Service` tags only where needed for Effect dependency injection.
|
||||
- `Layer.succeed`, `Layer.effect`, and `Layer.scoped` for construction.
|
||||
- Closure-held state only where necessary, preferably scoped and finalized.
|
||||
- Object-literal services instead of class instances.
|
||||
- Factory functions like `makeXService`, `makeXLayer`, and `runX`.
|
||||
- `Effect.tryPromise` for external Promise APIs.
|
||||
- `Effect.fn` for reusable effectful service methods.
|
||||
|
||||
Preserve behavior and in-repo call semantics, but do not preserve native class
|
||||
constructors as public API. Replace exported classes with interfaces, service
|
||||
tags, factory functions, or layer builders, and update all in-repo callers.
|
||||
|
||||
## Verification Requirements
|
||||
|
||||
Required gates:
|
||||
|
||||
- `cd ts && bun run check:tsgo`
|
||||
- `cd ts && bun run build`
|
||||
- `cd ts && bun run test`
|
||||
- `cd ts && bun run workbench:qa`
|
||||
- Add or provide an inventory command/script that fails on blocking production
|
||||
classes.
|
||||
- Run the inventory command after every migration phase.
|
||||
- Final inventory must report zero blocking production native classes.
|
||||
|
||||
Live smoke if service surfaces changed:
|
||||
|
||||
- `cd ts/deploy && docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build`
|
||||
- `cd ts && bun run seed`
|
||||
- `cd ts && bun run seed:demo`
|
||||
- `cd ts && bun run seed:flows`
|
||||
- `cd ts && SKIP_LLM=1 bun run test:pipeline`
|
||||
|
||||
## New Session Prompt
|
||||
|
||||
Paste this into a new Codex session:
|
||||
|
||||
```text
|
||||
Use the grill-me skill from /home/elpresidank/YeeBois/projects/beep-effect/.agents/skills/grill-me/SKILL.md.
|
||||
|
||||
The repo is /home/elpresidank/YeeBois/dev/trustgraph and the TypeScript port is in ./ts. I want to remove native TypeScript/JavaScript class syntax from production runtime code and convert the port to functional, pure Effect services.
|
||||
|
||||
Before asking me questions, use parallel read-only sub-agents to inventory every class declaration under ts/packages/**/src. Production runtime source is the blocking scope; tests and scripts should be inventoried separately but are non-blocking. Use AST-based inspection where possible, not regex alone.
|
||||
|
||||
Allowed class syntax is extremely narrow and proof-based. Even Effect-native-looking classes such as S.Class, S.TaggedClass, S.TaggedErrorClass, S.ErrorClass, Data.TaggedError, Context.Service, Rpc.make, and HttpApi.make must be treated as candidate exemptions, not automatic exemptions. If a functional Effect equivalent exists, use the functional form. I prefer functions, factories, closures, object-literal service implementations, Context.Service tags, and Layers over all class-shaped code.
|
||||
|
||||
Use these files as representative smells:
|
||||
- ts/packages/base/src/processor/async-processor.ts#L38-162
|
||||
- ts/packages/base/src/processor/flow-processor.ts#L284-358
|
||||
|
||||
Inventory by subsystem: base processor/messaging/runtime/specs/backend; flow service processors and LLM/embedding/RAG services; query/store engines and adapters; client socket/API wrappers; workbench production source.
|
||||
|
||||
After inventory, use grill-me to resolve any remaining API or rollout tradeoffs, then produce a decision-complete implementation plan. After I approve the plan, implement in phases and repeat inventory until zero blocking production native classes remain.
|
||||
|
||||
Required verification: cd ts && bun run check:tsgo; cd ts && bun run build; cd ts && bun run test; cd ts && bun run workbench:qa; add or provide an inventory command/script that fails on blocking production classes. If service surfaces changed, also run the live docker/seed/pipeline smoke from this file.
|
||||
```
|
||||
|
||||
## Goal Command
|
||||
|
||||
```text
|
||||
/goal Use ts/CLASS_EFFECT_GOAL.md to run the TrustGraph TS native-class-to-functional-Effect-services migration. First use grill-me and parallel read-only sub-agents to inventory every native class declaration. Then produce and execute a decision-complete refactor plan that replaces classes with pure functions, factories, closures, object-literal services, Context.Service tags, and Layers. Terminal condition: repeated inventory agents and the repo inventory command report zero blocking native classes in production runtime source under ts/packages/**/src. Even Effect-native class forms such as S.Class, TaggedError classes, Context.Service, Rpc.make, and HttpApi.make must be treated as narrow proof-based exemptions; if a functional Effect equivalent exists, use it. Verify with check:tsgo, build, test, workbench:qa, and live smoke when service surfaces change. Preserve unrelated local files.
|
||||
```
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,127 +0,0 @@
|
|||
# TrustGraph TS Effect-Native Rewrite Plan
|
||||
|
||||
## Summary
|
||||
|
||||
Bring the TypeScript port to a genuinely Effect-native shape while keeping existing TrustGraph capabilities working. The current native MCP rewrite should be preserved. The remaining work is to remove Promise-first TrustGraph APIs and replace manual host/framework plumbing with Effect v4-native backend, messaging, processor, HTTP/RPC, and CLI layers.
|
||||
|
||||
The implementation should proceed in green checkpoints rather than one broad sweep. Each stage should leave the repo closer to Effect-native and should run the smallest relevant gate before moving on.
|
||||
|
||||
## Non-Negotiable Architecture Decisions
|
||||
|
||||
- TrustGraph-authored production APIs should be Effect-first, not Promise-first.
|
||||
- Remove exported Promise APIs from backend, messaging, processor, flow service, gateway, CLI, and service runner surfaces.
|
||||
- Do not keep dual Promise and Effect methods on the same TrustGraph objects.
|
||||
- Do not rely on Fastify, `@fastify/websocket`, Commander, or production `Effect.runPromise` / `ManagedRuntime` compatibility bridges.
|
||||
- External JavaScript SDKs that are inherently Promise-based may be wrapped with `Effect.tryPromise`, but that Promise shape must stay behind Effect APIs.
|
||||
- Gateway HTTP and WebSocket behavior should move to Effect v4 native modules: `effect/unstable/httpapi`, `effect/unstable/http`, `effect/unstable/rpc`, `effect/unstable/socket`, and platform Bun/Node HTTP/socket layers.
|
||||
- CLI behavior should move to `effect/unstable/cli`.
|
||||
- The MCP package should stay on the native `effect/unstable/ai/McpServer` implementation and must not reintroduce server-side `@modelcontextprotocol/sdk` or `zod`.
|
||||
|
||||
## Current Grounding
|
||||
|
||||
- `bun run check:tsgo` is green.
|
||||
- `bun run build` is green.
|
||||
- `bun run --cwd packages/mcp test` is green.
|
||||
- `bun run test` currently fails only in `@trustgraph/base`, in `packages/base/src/__tests__/nats-backend.test.ts`.
|
||||
- The NATS failures are a symptom of the backend still sitting between old Promise-shaped abstractions and newer JetStream calls. The rewrite should fix the contract, not only update the mock.
|
||||
- The Effect v4 source of truth for APIs and examples is available at `/home/elpresidank/YeeBois/projects/beep-effect/.repos/effect-v4`.
|
||||
|
||||
## Stage 1: Backend And NATS
|
||||
|
||||
- Redefine `Message`, `BackendProducer`, `BackendConsumer`, and `PubSubBackend` around `Effect.Effect` return values.
|
||||
- Make NATS connection, JetStream manager/client, stream initialization, and durable consumer handles scoped resources.
|
||||
- Replace deprecated receive plumbing with modern JetStream consumer APIs.
|
||||
- Preserve behavior for stream creation, durable consumer creation, headers, JSON/schema encode/decode, receive timeout returning `null`, ack/nak, close/drain, and tagged `PubSubError` mapping.
|
||||
- Update NATS tests and in-memory fake backends to the new Effect-native interface.
|
||||
|
||||
Verification:
|
||||
|
||||
- `bun run --cwd packages/base test src/__tests__/nats-backend.test.ts`
|
||||
- `bun run --cwd packages/base test`
|
||||
- `bun run check:tsgo`
|
||||
|
||||
## Stage 2: Messaging And Processor Runtime
|
||||
|
||||
- Remove Promise facades from `makeProducer`, `makeConsumer`, `makeRequestResponse`, processor `start` / `stop` / `run`, and flow compatibility wrappers.
|
||||
- Expose Effect values, scoped constructors, Context services, Layers, and finalizers as the only TrustGraph API shape.
|
||||
- Convert message handlers, config handlers, shutdown callbacks, producer send/flush/close, consumer receive/ack/nak/close, and request/response operations to Effect functions.
|
||||
- Remove internal `ManagedRuntime.make(Layer.empty)` compatibility runtimes from production source.
|
||||
|
||||
Verification:
|
||||
|
||||
- `bun run --cwd packages/base test`
|
||||
- `bun run check:tsgo`
|
||||
|
||||
## Stage 3: Flow Service Call Sites
|
||||
|
||||
- Convert flow service state methods that currently call `Effect.runPromise` into Effect-valued service methods.
|
||||
- Keep public feature behavior unchanged for config, flow manager, librarian, knowledge cores, retrieval, embeddings, triples, agent, MCP tool, and text-completion services.
|
||||
- Convert test fakes to the same Effect-native backend/messaging contracts.
|
||||
|
||||
Verification:
|
||||
|
||||
- `bun run --cwd packages/flow test`
|
||||
- `bun run test`
|
||||
- `bun run check:tsgo`
|
||||
|
||||
## Stage 4: Gateway HTTP And RPC
|
||||
|
||||
- Remove Fastify and `@fastify/websocket` from `packages/flow`.
|
||||
- Rebuild the gateway with Effect v4 `HttpApi` groups/endpoints and native RPC/socket layers.
|
||||
- Preserve existing behavior:
|
||||
- `POST /api/v1/workbench/dispatch`
|
||||
- `POST /api/v1/:kind`
|
||||
- `POST /api/v1/flow/:flow/service/:kind`
|
||||
- `POST /api/v1/flow/:flow/load`
|
||||
- `GET /api/v1/rpc`
|
||||
- `GET /api/v1/metrics`
|
||||
- bearer auth behavior and RPC token behavior
|
||||
- existing response/error shapes expected by clients and workbench
|
||||
- Use native Effect HTTP test utilities or route-level protocol tests instead of Fastify injection.
|
||||
|
||||
Verification:
|
||||
|
||||
- `bun run --cwd packages/flow test`
|
||||
- `bun run --cwd packages/client test`
|
||||
- `bun run test`
|
||||
- `bun run build`
|
||||
|
||||
## Stage 5: CLI And Runner Scripts
|
||||
|
||||
- Remove Commander from `packages/cli`.
|
||||
- Rebuild CLI commands with `effect/unstable/cli` and Effect-native socket/API clients.
|
||||
- Convert service runner scripts to launch Effect programs directly with platform runtime `runMain` style execution.
|
||||
- Remove `run(): Promise<void>` exports from flow services; export Effect programs/layers and Effect-native `runMain` helpers instead.
|
||||
- Leave script-only demo/seed tools as follow-up only if they are outside production package source, but do not let production packages depend on Promise facades.
|
||||
|
||||
Verification:
|
||||
|
||||
- `bun run --cwd packages/cli test`
|
||||
- `bun run check:tsgo`
|
||||
- `bun run build`
|
||||
- `bun run test`
|
||||
|
||||
## Stage 6: Cleanup And Acceptance
|
||||
|
||||
- Remove obsolete dependencies from package manifests and lockfile:
|
||||
- `fastify`
|
||||
- `@fastify/websocket`
|
||||
- `commander`
|
||||
- any legacy dependency made unnecessary by the rewrite
|
||||
- Preserve unrelated dirty work. Do not revert user changes.
|
||||
- Use parallel agents for bounded audits or disjoint rewrite slices when useful.
|
||||
- Use Graphiti memory if available; if unavailable, continue safely and report it skipped.
|
||||
|
||||
Final verification:
|
||||
|
||||
- `bun run check:tsgo`
|
||||
- `bun run build`
|
||||
- `bun run lint`
|
||||
- `bun run test`
|
||||
- `bun run --cwd packages/mcp test`
|
||||
- `bun run workbench:qa` after installing the matching Playwright browser if needed
|
||||
- `git diff --check`
|
||||
- `rg -n "fastify|@fastify/websocket|commander" packages package.json bun.lock` has no production dependency/use hits
|
||||
- `rg -n "Effect\\.runPromise|ManagedRuntime\\.make|Promise<|async function main" packages/base/src packages/flow/src packages/cli/src scripts -g "*.ts"` has no production-source hits except tests or unavoidable external type declarations that are not TrustGraph APIs
|
||||
|
||||
Do not mark the goal complete until all required gates are green, or until a real external blocker is reported with the exact failing command, error, and smallest next action.
|
||||
|
|
@ -1,225 +0,0 @@
|
|||
# TrustGraph Effect-Native Rewrite Playbook
|
||||
|
||||
This playbook is the context packet for read-only sub-agents that audit the
|
||||
TrustGraph TypeScript port for code that hand-rolls behavior already provided by
|
||||
Effect v4. It is not an implementation plan for a single rewrite. Its job is to
|
||||
make future scouts fast, grounded, and allergic to invented APIs.
|
||||
|
||||
## Source Baseline
|
||||
|
||||
Verify these paths at the start of every audit run:
|
||||
|
||||
- TrustGraph TS port: `/home/elpresidank/YeeBois/dev/trustgraph/ts`
|
||||
- Effect v4 subtree: `/home/elpresidank/YeeBois/projects/beep-effect2/.repos/effect-v4`
|
||||
- Installed Effect fallback: `ts/node_modules/effect`
|
||||
- Installed Atom React fallback: `ts/packages/workbench/node_modules/@effect/atom-react`
|
||||
- Native TypeScript checker: `ts/node_modules/.bin/tsgo`
|
||||
|
||||
The prompt typo path `~/YeeBois/projecects/...` is not valid on this machine.
|
||||
Use `~/YeeBois/projects/...`.
|
||||
|
||||
When a package is not present in the subtree, verify it from TrustGraph's
|
||||
installed package sources before proposing a replacement. This matters for
|
||||
`effect/unstable/reactivity`, `effect/unstable/ai`, `effect/unstable/rpc`,
|
||||
`effect/unstable/socket`, `effect/unstable/http`, `effect/unstable/httpapi`,
|
||||
and `@effect/atom-react`.
|
||||
|
||||
Important import rule: the Effect v4 subtree contains separate packages such as
|
||||
`@effect/ai`, `@effect/rpc`, and `@effect/platform`, but TrustGraph currently
|
||||
resolves many beta APIs through `effect/unstable/*` import paths. Prefer the
|
||||
installed TrustGraph import path when it exists; use the subtree package path as
|
||||
source proof, not as an automatic import recommendation.
|
||||
|
||||
TrustGraph's TypeScript check must use `@typescript/native-preview` patched by
|
||||
`@effect/tsgo`. The expected local version line is currently:
|
||||
|
||||
```sh
|
||||
cd ts && ./node_modules/.bin/tsgo --version
|
||||
# Version 7.0.0-dev+effect-tsgo.0.13.0
|
||||
```
|
||||
|
||||
The root `ts/tsconfig.base.json` language-service plugin should stay aligned
|
||||
with `/home/elpresidank/YeeBois/projects/beep-effect/tsconfig.base.json`,
|
||||
using the same diagnostic severities and no test-only diagnostic downgrades.
|
||||
The repo-specific `namespaceImportPackages` entry should use `@trustgraph/*`.
|
||||
The root `check` script should delegate to `check:tsgo`, and `check:tsgo` must
|
||||
run `effect-tsgo patch` before `tsgo -b` so agents get Effect diagnostics
|
||||
instead of plain TypeScript. Treat every `effect(...)` diagnostic as a hard
|
||||
blocker.
|
||||
|
||||
## Primitive Map
|
||||
|
||||
Use this map as the starting baseline. A finding is only valid when it cites the
|
||||
TrustGraph path, the import path, and the Effect source path that proves the
|
||||
primitive exists.
|
||||
|
||||
| Handrolled pattern | Preferred Effect primitive | Import path | Verified source |
|
||||
| --- | --- | --- | --- |
|
||||
| Promise loops, top-level async orchestration | `Effect`, `Effect.fn`, `Effect.scoped`, `Effect.runPromiseWith` at boundaries | `effect` | `packages/effect/src/Effect.ts` |
|
||||
| Resource construction and teardown | `Layer`, `Scope`, `Effect.acquireRelease`, `Effect.addFinalizer` | `effect` | `packages/effect/src/Layer.ts`, `packages/effect/src/Scope.ts` |
|
||||
| Mutable service state | `Ref`, `SynchronizedRef`, `SubscriptionRef` | `effect` | `packages/effect/src/Ref.ts`, `packages/effect/src/SynchronizedRef.ts` |
|
||||
| Long-lived keyed state and set membership | `HashMap`, `MutableHashMap`, `HashSet`, `MutableHashSet` | `effect` | `packages/effect/src/HashMap.ts`, `packages/effect/src/MutableHashMap.ts`, `packages/effect/src/HashSet.ts`, `packages/effect/src/MutableHashSet.ts` |
|
||||
| Polling, delays, retry/backoff | `Schedule`, `Effect.sleep`, `Effect.retry` | `effect` | `packages/effect/src/Schedule.ts`, `packages/effect/src/Effect.ts` |
|
||||
| Callback queues and streaming fanout | `Queue`, `PubSub`, `Stream`, `Channel` | `effect` | `packages/effect/src/Queue.ts`, `packages/effect/src/PubSub.ts`, `packages/effect/src/Stream.ts`, `packages/effect/src/Channel.ts` |
|
||||
| Env/config decoding | `Config`, `ConfigProvider`, platform config providers | `effect`, `effect/ConfigProvider`, provider packages | `packages/effect/src/Config.ts`, `packages/effect/src/ConfigProvider.ts`, `packages/platform/src/PlatformConfigProvider.ts` |
|
||||
| JSON/wire schemas | `Schema`, `ParseResult`, `RpcSchema` | `effect/Schema`, `effect/unstable/rpc/RpcSchema` | `packages/effect/src/Schema.ts`, `ts/node_modules/effect/src/unstable/rpc/RpcSchema.ts` |
|
||||
| WebSocket lifecycle and framing | `Socket`, `RpcClient`, `RpcServer`, `RpcSerialization` | `effect/unstable/socket`, `effect/unstable/rpc` | `ts/node_modules/effect/src/unstable/socket/*.ts`, `ts/node_modules/effect/src/unstable/rpc/*.ts` |
|
||||
| HTTP servers/clients and typed APIs | `HttpApi`, `HttpApiClient`, `HttpApiBuilder`, `HttpClient`, `HttpServer` | `effect/unstable/http`, `effect/unstable/httpapi`, platform providers | `ts/node_modules/effect/src/unstable/http/*.ts`, `ts/node_modules/effect/src/unstable/httpapi/*.ts` |
|
||||
| File, storage, and process IO | `FileSystem`, `KeyValueStore`, `ChildProcess`, `ChildProcessSpawner` | `effect/FileSystem`, `effect/unstable/persistence/KeyValueStore`, `effect/unstable/process/*`, provider packages | `ts/node_modules/effect/src/FileSystem.ts`, `ts/node_modules/effect/src/unstable/persistence/KeyValueStore.ts`, `ts/node_modules/effect/src/unstable/process/*.ts` |
|
||||
| Browser local storage and clipboard | `BrowserKeyValueStore`, `Clipboard`, `BrowserHttpClient`, `BrowserSocket` | `@effect/platform-browser/*` | `ts/node_modules/.bun/@effect+platform-browser@4.0.0-beta.75+a5c1409dbf4ddafe/node_modules/@effect/platform-browser/src/*.ts` |
|
||||
| AI tools, MCP, and model calls | `Tool`, `Toolkit`, `McpServer`, `McpSchema`, `LanguageModel`, provider layers | `effect/unstable/ai`, provider packages such as `@effect/ai-openai` | `ts/node_modules/effect/src/unstable/ai/*.ts`, `packages/ai/ai/src/*.ts` |
|
||||
| Workbench async state | `Atom`, `AtomRpc`, `AtomHttpApi`, `AsyncResult`, `AtomRegistry`, `Reactivity` | `effect/unstable/reactivity`, `@effect/atom-react` | `ts/node_modules/effect/src/unstable/reactivity/*.ts`, `ts/packages/workbench/node_modules/@effect/atom-react/src/*.ts` |
|
||||
| Metrics and logs | `Metric`, `Logger`, `Effect.log*` | `effect`, `@effect/opentelemetry` | `packages/effect/src/Metric.ts`, `packages/effect/src/Logger.ts` |
|
||||
| Normal internal errors | `S.TaggedErrorClass` and existing TrustGraph tagged errors | `effect/Schema`, `@trustgraph/base/errors` | `packages/effect/src/Schema.ts`, `ts/packages/base/src/errors.ts` |
|
||||
| Imperative exception capture | `Effect.try`, `Effect.tryPromise`, or `Result.try` | `effect`, `effect/Result` | `packages/effect/src/Effect.ts`, `packages/effect/src/Result.ts` |
|
||||
|
||||
Known concrete exports useful to scouts:
|
||||
|
||||
- `Socket.makeWebSocket`, `Socket.fromWebSocket`, `Socket.toChannel`,
|
||||
`Socket.toChannelString`, `Socket.layerWebSocket`.
|
||||
- `RpcClient.layerProtocolSocket`, `RpcServer.layerProtocolWebsocket`,
|
||||
`RpcServer.layerProtocolSocketServer`, `RpcSerialization.layerNdjson`,
|
||||
`RpcSerialization.layerNdJsonRpc`.
|
||||
- `BunFileSystem`, `BunSocket`, `BunHttpClient`, `BunHttpServer`,
|
||||
`BunRuntime`, `BunChildProcessSpawner` from `@effect/platform-bun`.
|
||||
- `BrowserKeyValueStore.layerLocalStorage`,
|
||||
`BrowserKeyValueStore.layerSessionStorage`,
|
||||
`BrowserHttpClient.layerXMLHttpRequest`.
|
||||
- `Atom.make`, `Atom.runtime`, `Atom.fn`, `Atom.family`,
|
||||
`Atom.subscriptionRef`, `AtomRegistry.layer`, `Reactivity.layer`,
|
||||
`Reactivity.query`, `Reactivity.stream`, `Reactivity.mutation`.
|
||||
- `Tool.make`, `Toolkit.make`, `McpServer.registerToolkit`,
|
||||
`LanguageModel.generateText`, `LanguageModel.streamText`.
|
||||
- `S.TaggedErrorClass` for internal/library errors. Treat `new Error` inside
|
||||
library internals as migration evidence unless it is a host/tool boundary,
|
||||
test-only helper, or externally mandated error shape.
|
||||
- `Effect.try`, `Effect.tryPromise`, and `Result.try` for exception capture.
|
||||
Treat `try`/`catch` blocks as migration evidence unless they are host/tool
|
||||
boundaries or test-only helpers.
|
||||
- `S.toTaggedUnion(...).match` and `effect/Match` discriminator helpers for
|
||||
discriminated unions. Treat native `switch` statements as migration evidence
|
||||
unless the code is a host parser/traversal boundary with no useful Effect
|
||||
schema or match primitive.
|
||||
|
||||
## Scout Workflow
|
||||
|
||||
1. Confirm repo state with `git status -sb`. Preserve unrelated files,
|
||||
especially `.idea/effect.intellij.xml`.
|
||||
2. Refresh the source baseline above. If a path moved, record the corrected path
|
||||
in the report before making any recommendations.
|
||||
3. Verify the tsgo baseline:
|
||||
|
||||
```sh
|
||||
cd ts && ./node_modules/.bin/tsgo --version
|
||||
bun run check
|
||||
```
|
||||
|
||||
4. Run quick signal scans:
|
||||
|
||||
```sh
|
||||
rg -n "S\\.ErrorClass|try \\{|catch \\(|new Error|new Promise|setTimeout|while \\(|switch \\(|receive\\(|Effect\\.runPromise|toPromiseRequestor|makeAsyncProcessor|process\\.env|JSON\\.parse|JSON\\.stringify|localStorage|new Map|new Set|Map<|Set<|WebSocket" ts/packages --glob '*.ts' --glob '*.tsx'
|
||||
```
|
||||
|
||||
5. Split scouts by lane. If the thread cannot spawn every scout in parallel,
|
||||
run them in batches using the same report schema.
|
||||
6. Every finding must include both:
|
||||
- Evidence of the handrolled TrustGraph pattern.
|
||||
- Evidence of the exact Effect primitive that could replace it.
|
||||
7. Do not rewrite code in this audit. The output is a ranked opportunity map and
|
||||
a recommended PR order.
|
||||
|
||||
## Agent Lanes
|
||||
|
||||
Use these lane prompts as the durable starting point.
|
||||
|
||||
### Base Messaging And Processor Runtime
|
||||
|
||||
Inspect `ts/packages/base/src/messaging`, `ts/packages/base/src/processor`, and
|
||||
`ts/packages/base/src/spec`. Find Promise compatibility facades, polling
|
||||
receivers, manual request/response caches, top-level `Effect.runPromise`, and
|
||||
mutable maps. Compare against `Queue`, `PubSub`, `Stream`, `Scope`, `Layer`,
|
||||
`Schedule`, `Ref`, and `SynchronizedRef`.
|
||||
|
||||
### Flow Stateful Services
|
||||
|
||||
Inspect `ts/packages/flow/src/config`, `librarian`, `cores`, `flow-manager`,
|
||||
`prompt`, and service entrypoints. Find `makeAsyncProcessor` object services,
|
||||
`while (this.running)`, `sleep`, `receive(2000)`, local persistence, and
|
||||
process-env config. Compare against scoped layers, state refs, schedules,
|
||||
`FileSystem`, `KeyValueStore`, `Config`, and schema codecs. In TrustGraph's
|
||||
installed beta, prefer `effect/FileSystem` and
|
||||
`effect/unstable/persistence/KeyValueStore` plus runtime-specific provider
|
||||
packages.
|
||||
|
||||
### Gateway And RPC Boundaries
|
||||
|
||||
Inspect `ts/packages/flow/src/gateway` and `ts/packages/client/src/socket`.
|
||||
Find manual requestor caches, streaming completion detectors, WebSocket
|
||||
constructor shims, Promise-returning compatibility APIs, and repeated
|
||||
`Effect.runPromise`. Compare against `Socket`, `RpcClient`, `RpcServer`,
|
||||
`RpcSerialization`, `Stream`, `Queue`, and `Scope`.
|
||||
|
||||
### RAG, Agent, Provider, And Storage Layers
|
||||
|
||||
Inspect `ts/packages/flow/src/retrieval`, `agent`, `storage`, `query`, `model`,
|
||||
and `embeddings`. Find `toPromiseRequestor`, direct SDK resource management,
|
||||
ambient config, JSON parsing, and manual telemetry. Compare against
|
||||
`EffectRequestResponse`, `Stream`, provider `Layer`s, `Config`, `Schema`,
|
||||
`Metric`, `Logger`, and `effect/unstable/ai`.
|
||||
|
||||
### MCP And Workbench
|
||||
|
||||
Inspect `ts/packages/mcp` and `ts/packages/workbench`. Treat these as
|
||||
lower-priority unless a handrolled pattern clearly remains. Prefer making the
|
||||
Effect MCP server canonical over rewriting it from scratch. In workbench,
|
||||
compare local storage and remote state wiring against `BrowserKeyValueStore`,
|
||||
`AtomRpc`, `AtomHttpApi`, `AsyncResult`, and `Reactivity`.
|
||||
|
||||
## Report Schema
|
||||
|
||||
Each scout must return findings in this shape:
|
||||
|
||||
```md
|
||||
## Finding: <short title>
|
||||
|
||||
- Priority: P0 | P1 | P2 | No-op
|
||||
- Impact: 1-5
|
||||
- Risk: 1-5
|
||||
- Confidence: 1-5
|
||||
- TrustGraph evidence: <path:line> and pattern name
|
||||
- Effect evidence: <import path> and source path
|
||||
- Current behavior: <what must be preserved>
|
||||
- Rewrite shape: <Effect-native replacement direction>
|
||||
- Tests: <specific existing or required tests>
|
||||
- Blockers: <unknowns, API constraints, compatibility requirements>
|
||||
```
|
||||
|
||||
Scoring:
|
||||
|
||||
- P0: large handrolled lifecycle/concurrency/transport surface with a verified
|
||||
Effect primitive and clear behavior-preserving route.
|
||||
- P1: real replacement opportunity, but either risk or compatibility needs a
|
||||
focused design pass.
|
||||
- P2: cleanup or local modernization; useful but not strategic.
|
||||
- No-op: current code is already Effect-native, public-boundary Promise code is
|
||||
intentional, or the Effect primitive does not actually fit.
|
||||
|
||||
## Acceptance Checks
|
||||
|
||||
For an audit-only pass:
|
||||
|
||||
```sh
|
||||
git status -sb
|
||||
git diff --check -- ts/EFFECT_NATIVE_REWRITE_PLAYBOOK.md ts/EFFECT_NATIVE_REWRITE_AUDIT.md
|
||||
```
|
||||
|
||||
For any later implementation PR, rerun the relevant package tests and at least:
|
||||
|
||||
```sh
|
||||
cd ts && bun run check
|
||||
cd ts && bun run build
|
||||
cd ts && bun run test
|
||||
```
|
||||
|
||||
If gateway, live services, or workbench behavior changes, also run the existing
|
||||
smoke lanes from `ts/CLASS_EFFECT_GOAL.md`.
|
||||
80
ts/EFFECT_NATIVE_STATUS.md
Normal file
80
ts/EFFECT_NATIVE_STATUS.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
# TrustGraph TS Effect-Native Status
|
||||
|
||||
## Current State
|
||||
|
||||
The TypeScript port on `ts-port-effect-v4` is Effect-first across the production
|
||||
runtime surfaces completed in Phases 0-6. The law baseline is empty:
|
||||
`scripts/effect-laws.allowlist.json` has no baseline entries and one documented
|
||||
host-boundary exemption. The native-class inventory reports zero blocking
|
||||
production native classes.
|
||||
|
||||
The client package's public barrel exports the Effect RPC gateway surface and
|
||||
shared data models. The legacy `trustgraph-socket.ts` Promise `BaseApi` shim is
|
||||
kept out of the package barrel and remains only for direct client tests.
|
||||
Workbench QA uses its own minimal mock API shape.
|
||||
|
||||
## Adapted Effect Laws
|
||||
|
||||
- Data that crosses wires, persistence, or domain boundaries uses `S.Class`,
|
||||
`S.Struct`, or tagged schema unions. Service contracts and option bags may
|
||||
remain interfaces.
|
||||
- Production code reads configuration through `Config`, not `process.env`.
|
||||
- JSON encode/decode goes through Schema codecs such as
|
||||
`S.UnknownFromJsonString`.
|
||||
- HTTP clients use `effect/unstable/http` with platform layers.
|
||||
- Effects run at entrypoints through platform `runMain`; libraries return
|
||||
`Effect`, `Stream`, `Layer`, or service values.
|
||||
- Deterministic ordering uses `effect/Array` and explicit `Order`s.
|
||||
- File/path work uses platform services rather than `node:fs` or `node:path`.
|
||||
- Errors owned by TrustGraph code use tagged errors on the Effect channel.
|
||||
|
||||
## Standing Exemptions
|
||||
|
||||
- `packages/workbench/src/main.tsx` may throw if `#root` is missing. This is a
|
||||
browser host boundary and the app cannot render without the mount point.
|
||||
- `packages/flow/src/agent/mcp-tool/service.ts` may use the
|
||||
`@modelcontextprotocol/sdk` client behind Effect wrappers. Effect v4 provides
|
||||
server-side MCP support here, not a replacement client.
|
||||
- Provider SDK integrations for OpenAI, Azure OpenAI, OpenAI-compatible,
|
||||
Mistral, and Ollama stay behind `Effect.tryPromise` and
|
||||
`makeLanguageModelProvider`. Claude already uses `@effect/ai-anthropic`.
|
||||
- `PubSubBackend` remains a broker adapter contract. Effect `PubSub` is
|
||||
in-process only and is not a NATS replacement.
|
||||
- Tests and QA fixtures are excluded from production law checks.
|
||||
|
||||
## Gate Commands
|
||||
|
||||
Phase 6 validation:
|
||||
|
||||
```sh
|
||||
bun run check
|
||||
bun run lint
|
||||
bunx --bun turbo test build --force
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Final Phase 7 acceptance, all no-cache where applicable:
|
||||
|
||||
```sh
|
||||
bun run check:tsgo
|
||||
bunx --bun turbo build lint test --force
|
||||
bun run --cwd packages/mcp test
|
||||
bun run workbench:qa
|
||||
cd deploy && docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
|
||||
bun run seed && bun run seed:flows && bun run seed:demo
|
||||
SKIP_LLM=1 bun run test:pipeline
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Final source audits:
|
||||
|
||||
```sh
|
||||
bun scripts/check-effect-laws.ts
|
||||
bun scripts/inventory-native-classes.ts
|
||||
rg -n "fastify|@fastify/websocket|commander|zod" package.json packages scripts bun.lock
|
||||
rg -n "@modelcontextprotocol/sdk" packages scripts package.json bun.lock
|
||||
```
|
||||
|
||||
Expected result: no law baseline debt, zero blocking native classes, no Fastify,
|
||||
Commander, Zod, or server-side MCP SDK reintroduction. The MCP SDK client
|
||||
exemption remains documented above.
|
||||
|
|
@ -75,12 +75,12 @@ export const Field = S.Struct({
|
|||
});
|
||||
export type Field = typeof Field.Type;
|
||||
|
||||
export const RowSchema = S.Struct({
|
||||
export const Row = S.Struct({
|
||||
name: S.String,
|
||||
description: S.optionalKey(S.String),
|
||||
fields: S.Array(Field).pipe(S.mutable),
|
||||
});
|
||||
export type RowSchema = typeof RowSchema.Type;
|
||||
export type Row = typeof Row.Type;
|
||||
|
||||
export const LlmResult = S.Struct({
|
||||
text: S.String,
|
||||
|
|
|
|||
|
|
@ -6,8 +6,13 @@ export * from "./models/Triple.js";
|
|||
export * from "./models/messages.js";
|
||||
export * from "./models/namespaces.js";
|
||||
|
||||
// Export socket client
|
||||
export * from "./socket/trustgraph-socket.js";
|
||||
// Export retained compatibility types from the legacy socket shim.
|
||||
export type {
|
||||
ConnectionState,
|
||||
ExplainEvent,
|
||||
GraphRagOptions,
|
||||
StreamingMetadata,
|
||||
} from "./socket/trustgraph-socket.js";
|
||||
export * from "./socket/effect-rpc-client.js";
|
||||
export * from "./rpc/contract.js";
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ import type {
|
|||
RpcConnectionState,
|
||||
} from "./effect-rpc-client.js";
|
||||
import { getDefaultSocketUrl, getRandomValues } from "./websocket-adapter.js";
|
||||
import { Match, Option, Schema as S } from "effect";
|
||||
import { Array as A, Match, Option, Order, Schema as S } from "effect";
|
||||
import * as Predicate from "effect/Predicate";
|
||||
|
||||
// Import all message types for different services
|
||||
|
|
@ -1240,9 +1240,7 @@ export function makeFlowsApi(api: BaseApi) {
|
|||
return this.getConfigAll().then((r) => {
|
||||
const config = r as { config?: { prompt?: Record<string, unknown> } };
|
||||
const promptNs = config.config?.prompt ?? {};
|
||||
return Object.keys(promptNs)
|
||||
.filter((k) => k !== "system")
|
||||
.sort()
|
||||
return A.sort(Object.keys(promptNs).filter((k) => k !== "system"), Order.String)
|
||||
.map((id) => ({ id, name: id }));
|
||||
});
|
||||
},
|
||||
|
|
@ -2204,9 +2202,7 @@ export function makeConfigApi(api: BaseApi) {
|
|||
return this.getConfigAll().then((r) => {
|
||||
const config = r as { config?: { prompt?: Record<string, unknown> } };
|
||||
const promptNs = config.config?.prompt ?? {};
|
||||
return Object.keys(promptNs)
|
||||
.filter((k) => k !== "system")
|
||||
.sort()
|
||||
return A.sort(Object.keys(promptNs).filter((k) => k !== "system"), Order.String)
|
||||
.map((id) => ({ id, name: id }));
|
||||
});
|
||||
},
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
*/
|
||||
|
||||
import {NodeRuntime} from "@effect/platform-node";
|
||||
import {Duration, Effect, HashMap, Match, Option, SynchronizedRef} from "effect";
|
||||
import {Array as A, Duration, Effect, HashMap, Match, Option, Order, SynchronizedRef} from "effect";
|
||||
import * as Predicate from "effect/Predicate";
|
||||
import * as S from "effect/Schema";
|
||||
import type {
|
||||
|
|
@ -122,26 +122,27 @@ const initialState = (): ConfigServiceState => ({
|
|||
const getHashMapValue = <K, V>(store: HashMap.HashMap<K, V>, key: K): V | undefined =>
|
||||
Option.getOrUndefined(HashMap.get(store, key));
|
||||
|
||||
const compareText = (left: string, right: string): number =>
|
||||
left.localeCompare(right);
|
||||
|
||||
const compareWorkspace = (left: string, right: string): number =>
|
||||
const workspaceOrder = Order.make<string>((left, right) =>
|
||||
left === right
|
||||
? 0
|
||||
: left === DEFAULT_WORKSPACE
|
||||
? -1
|
||||
: right === DEFAULT_WORKSPACE
|
||||
? 1
|
||||
: compareText(left, right);
|
||||
: Order.String(left, right)
|
||||
);
|
||||
|
||||
const orderByKey = <A>(order: Order.Order<string>): Order.Order<readonly [string, A]> =>
|
||||
Order.mapInput(order, ([key]) => key);
|
||||
|
||||
const workspaceEntries = (store: ConfigStore): ReadonlyArray<readonly [string, WorkspaceStore]> =>
|
||||
HashMap.toEntries(store).sort(([left], [right]) => compareWorkspace(left, right));
|
||||
A.sort(HashMap.toEntries(store), orderByKey<WorkspaceStore>(workspaceOrder));
|
||||
|
||||
const namespaceEntries = (store: WorkspaceStore): ReadonlyArray<readonly [string, NamespaceStore]> =>
|
||||
HashMap.toEntries(store).sort(([left], [right]) => compareText(left, right));
|
||||
A.sort(HashMap.toEntries(store), orderByKey<NamespaceStore>(Order.String));
|
||||
|
||||
const valueEntries = (store: NamespaceStore): ReadonlyArray<readonly [string, unknown]> =>
|
||||
HashMap.toEntries(store).sort(([left], [right]) => compareText(left, right));
|
||||
A.sort(HashMap.toEntries(store), orderByKey<unknown>(Order.String));
|
||||
|
||||
const toPersistedWorkspaces = (
|
||||
store: ConfigStore,
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ import {
|
|||
processorLifecycleError,
|
||||
topics,
|
||||
} from "@trustgraph/base";
|
||||
import {Duration, Effect, HashMap, Match, SynchronizedRef} from "effect";
|
||||
import {Array as A, Duration, Effect, HashMap, Match, Order, SynchronizedRef} from "effect";
|
||||
import * as O from "effect/Option";
|
||||
import * as S from "effect/Schema";
|
||||
import {ensureDirectoryEffect, joinPath, readTextFileEffect, writeTextFileEffect} from "../runtime/effect-files.js";
|
||||
|
|
@ -137,8 +137,11 @@ const cloneKnowledgeCore = (core: KnowledgeCore): KnowledgeCore => ({
|
|||
})),
|
||||
});
|
||||
|
||||
const sortedEntries = <A>(store: HashMap.HashMap<string, A>): ReadonlyArray<readonly [string, A]> =>
|
||||
HashMap.toEntries(store).sort(([left], [right]) => left.localeCompare(right));
|
||||
const sortedEntries = <Value>(store: HashMap.HashMap<string, Value>): ReadonlyArray<readonly [string, Value]> =>
|
||||
A.sort(
|
||||
HashMap.toEntries(store),
|
||||
Order.make<readonly [string, Value]>(([left], [right]) => Order.String(left, right)),
|
||||
);
|
||||
|
||||
const toPersistedSnapshot = (state: KnowledgeCoreServiceState): PersistedKnowledgeSnapshot => {
|
||||
const kg: Record<string, KnowledgeCore> = {};
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ import {
|
|||
import { makeProcessorProgram } from "@trustgraph/base";
|
||||
import type { Message } from "@trustgraph/base";
|
||||
import { NodeRuntime } from "@effect/platform-node";
|
||||
import { Duration, Effect, HashMap, Match, Option, SynchronizedRef } from "effect";
|
||||
import { Array as A, Duration, Effect, HashMap, Match, Option, Order, SynchronizedRef } from "effect";
|
||||
import * as S from "effect/Schema";
|
||||
|
||||
// ---------- Internal state types ----------
|
||||
|
|
@ -236,8 +236,11 @@ const isStringRecord = (value: unknown): value is Record<string, string> =>
|
|||
const getHashMapValue = <K, V>(store: HashMap.HashMap<K, V>, key: K): V | undefined =>
|
||||
Option.getOrUndefined(HashMap.get(store, key));
|
||||
|
||||
const sortedEntries = <A>(store: HashMap.HashMap<string, A>): ReadonlyArray<readonly [string, A]> =>
|
||||
HashMap.toEntries(store).sort(([left], [right]) => left.localeCompare(right));
|
||||
const sortedEntries = <Value>(store: HashMap.HashMap<string, Value>): ReadonlyArray<readonly [string, Value]> =>
|
||||
A.sort(
|
||||
HashMap.toEntries(store),
|
||||
Order.make<readonly [string, Value]>(([left], [right]) => Order.String(left, right)),
|
||||
);
|
||||
|
||||
const sortedKeys = <A>(store: HashMap.HashMap<string, A>): Array<string> =>
|
||||
sortedEntries(store).map(([key]) => key);
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ import {
|
|||
} from "@trustgraph/base";
|
||||
import type { Message } from "@trustgraph/base";
|
||||
import { NodeRuntime } from "@effect/platform-node";
|
||||
import { Clock, Config, DateTime, Duration, Effect, Match, Option, Random, SynchronizedRef } from "effect";
|
||||
import { Clock, Config, DateTime, Duration, Effect, Match, Option, Order, Random, SynchronizedRef } from "effect";
|
||||
import * as A from "effect/Array";
|
||||
import * as MutableHashMap from "effect/MutableHashMap";
|
||||
import * as S from "effect/Schema";
|
||||
|
|
@ -479,7 +479,7 @@ export function makeLibrarianService(config: LibrarianServiceConfig): LibrarianS
|
|||
if (session === undefined) {
|
||||
return yield* librarianServiceError("get-upload-status", `Upload not found: ${uploadId}`);
|
||||
}
|
||||
const receivedChunks = Array.from(MutableHashMap.keys(session.chunks)).sort((a, b) => a - b);
|
||||
const receivedChunks = A.sort(Array.from(MutableHashMap.keys(session.chunks)), Order.Number);
|
||||
const receivedSet = new Set(receivedChunks);
|
||||
const missingChunks = Array.from({ length: session.totalChunks }, (_, i) => i).filter((i) => !receivedSet.has(i));
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ import type {
|
|||
TriplesQueryResponse,
|
||||
} from "@trustgraph/base";
|
||||
import { Triple, errorMessage } from "@trustgraph/base";
|
||||
import { Context, Effect, Layer, Match } from "effect";
|
||||
import { Array as A, Context, Effect, Layer, Match, Order } from "effect";
|
||||
import * as O from "effect/Option";
|
||||
import * as S from "effect/Schema";
|
||||
|
||||
|
|
@ -348,8 +348,10 @@ const scoreEdges = Effect.fn("GraphRagEngine.scoreEdges")(function* (
|
|||
|
||||
yield* Effect.log(`[GraphRag] Edge scoring LLM response (first 500 chars): ${llmResp.response.slice(0, 500)}`);
|
||||
|
||||
const scored = parseScoredEdges(llmResp.response);
|
||||
scored.sort((a, b) => b.score - a.score);
|
||||
const scored = A.sort(
|
||||
parseScoredEdges(llmResp.response),
|
||||
Order.make<typeof ScoredEdge.Type>((left, right) => Order.Number(right.score, left.score)),
|
||||
);
|
||||
const topN = scored.slice(0, config.edgeLimit);
|
||||
|
||||
const result: Triple[] = [];
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import type {
|
|||
import {
|
||||
makeTrustGraphGatewayClientScoped,
|
||||
} from "@trustgraph/client";
|
||||
import {Clock, Config, Context, Effect, Layer} from "effect";
|
||||
import {Array as A, Clock, Config, Context, Effect, Layer, Order} from "effect";
|
||||
import * as O from "effect/Option";
|
||||
import * as Predicate from "effect/Predicate";
|
||||
import {McpServer, Tool, Toolkit} from "effect/unstable/ai";
|
||||
|
|
@ -1775,9 +1775,7 @@ export const TrustGraphMcpToolkitLive = TrustGraphMcpToolkit.toLayer(
|
|||
).pipe(
|
||||
Effect.map((response) => {
|
||||
const promptNs = asRecord(asRecord(response.config).prompt)
|
||||
const prompts = Object.keys(promptNs)
|
||||
.filter((key) => key !== "system")
|
||||
.sort()
|
||||
const prompts = A.sort(Object.keys(promptNs).filter((key) => key !== "system"), Order.String)
|
||||
.map((id) => ({id, name: id}))
|
||||
return GetPromptsSuccess.make({prompts})
|
||||
}),
|
||||
|
|
|
|||
|
|
@ -19,8 +19,9 @@ import {
|
|||
GatewayWorkbenchHttpApi,
|
||||
TrustGraphRpcs,
|
||||
} from "@trustgraph/client";
|
||||
import type { WorkbenchQaApi } from "@/qa/mock-api";
|
||||
import type { Scope, } from "effect";
|
||||
import { Cause, Clock, Context, Effect, Layer, Match, Metric, Option, Random, Schema as S, Stream } from "effect";
|
||||
import { Cause, Clock, Context, Effect, Layer, Match, Metric, Option, Order, Random, Schema as S, Stream } from "effect";
|
||||
import * as A from "effect/Array";
|
||||
import * as MutableHashMap from "effect/MutableHashMap";
|
||||
import * as Predicate from "effect/Predicate";
|
||||
|
|
@ -802,9 +803,9 @@ function ensureNoGatewayResponseError<A>(operation: string, value: A): Effect.Ef
|
|||
: Effect.fail(WorkbenchPromiseError.make({ cause: value, message: `${operation}: ${message}` }));
|
||||
}
|
||||
|
||||
function qaBaseApi(): import("@trustgraph/client").BaseApi | undefined {
|
||||
function qaBaseApi(): WorkbenchQaApi | undefined {
|
||||
if (typeof window === "undefined") return undefined;
|
||||
return (window as Window & { __TRUSTGRAPH_WORKBENCH_QA_API__?: import("@trustgraph/client").BaseApi }).__TRUSTGRAPH_WORKBENCH_QA_API__;
|
||||
return (window as Window & { __TRUSTGRAPH_WORKBENCH_QA_API__?: WorkbenchQaApi }).__TRUSTGRAPH_WORKBENCH_QA_API__;
|
||||
}
|
||||
|
||||
function makeWorkbenchGatewayApi(settings: Settings) {
|
||||
|
|
@ -897,9 +898,7 @@ function makeWorkbenchGatewayApi(settings: Settings) {
|
|||
Effect.map((response) => {
|
||||
const config = asJsonRecord(response.config);
|
||||
const promptNs = asJsonRecord(config.prompt);
|
||||
return Object.keys(promptNs)
|
||||
.filter((key) => key !== "system")
|
||||
.sort()
|
||||
return A.sort(Object.keys(promptNs).filter((key) => key !== "system"), Order.String)
|
||||
.map((id) => ({ id, name: id }));
|
||||
}),
|
||||
),
|
||||
|
|
@ -1826,9 +1825,9 @@ function explainTriplesKey(input: ExplainTriplesInput): string {
|
|||
const graphs = input.events
|
||||
.map((event) => event.explainGraph ?? event.explainId ?? "")
|
||||
.filter((id) => id.length > 0)
|
||||
.sort()
|
||||
const sortedGraphs = A.sort(graphs, Order.String)
|
||||
.join(explainGraphSeparator);
|
||||
return [input.flowId, input.collection, graphs].join(atomFamilyKeySeparator);
|
||||
return [input.flowId, input.collection, sortedGraphs].join(atomFamilyKeySeparator);
|
||||
}
|
||||
|
||||
const graphTriplesAtomByKey = Atom.family((key: string) => {
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import { lazy, Suspense } from "react";
|
||||
import { useAtom, useAtomValue } from "@effect/atom-react";
|
||||
import { Array as A, Order } from "effect";
|
||||
import { Network, ChevronRight, ChevronDown, Loader2 } from "lucide-react";
|
||||
import * as Atom from "effect/unstable/reactivity/Atom";
|
||||
import {
|
||||
|
|
@ -74,7 +75,7 @@ export function ExplainGraph({ explainEvents, collection }: ExplainGraphProps) {
|
|||
const loading = expanded && resultLoading(result, triples);
|
||||
const error = resultError(result);
|
||||
const { data: graphData, typeMap } = triplesToGraph(triples);
|
||||
const uniqueTypes = Array.from(new Set(Array.from(typeMap.values()).map(localName))).sort();
|
||||
const uniqueTypes = A.sort(Array.from(new Set(Array.from(typeMap.values()).map(localName))), Order.String);
|
||||
|
||||
return (
|
||||
<div className="mt-2 rounded-md border border-border/50">
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import { lazy, Suspense } from "react";
|
||||
import { useAtom, useAtomRefresh, useAtomValue } from "@effect/atom-react";
|
||||
import { Array as A, Order } from "effect";
|
||||
import {
|
||||
Rotate3d,
|
||||
Search,
|
||||
|
|
@ -180,7 +181,7 @@ export default function GraphPage() {
|
|||
const selectedNode = view.selectedNodeId !== null
|
||||
? data.nodes.find((node) => node.id === view.selectedNodeId)
|
||||
: undefined;
|
||||
const uniqueTypes = Array.from(new Set(Array.from(typeMap.values()).map(localName))).sort();
|
||||
const uniqueTypes = A.sort(Array.from(new Set(Array.from(typeMap.values()).map(localName))), Order.String);
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import {
|
|||
flowIdAtom,
|
||||
settingsAtom,
|
||||
} from "@/atoms/workbench";
|
||||
import type { BaseApi } from "@trustgraph/client";
|
||||
import type { WorkbenchQaApi } from "@/qa/mock-api";
|
||||
import { MockWorkbenchFixture, makeMockBaseApi, qaSettingsFromFixture, } from "@/qa/mock-api";
|
||||
import { Schema as S } from "effect";
|
||||
|
||||
|
|
@ -21,7 +21,7 @@ export class WorkbenchQaWindowConfig extends S.Class<WorkbenchQaWindowConfig>("W
|
|||
declare global {
|
||||
interface Window {
|
||||
__TRUSTGRAPH_WORKBENCH_QA__?: WorkbenchQaWindowConfig;
|
||||
__TRUSTGRAPH_WORKBENCH_QA_API__?: BaseApi;
|
||||
__TRUSTGRAPH_WORKBENCH_QA_API__?: WorkbenchQaApi;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,26 @@
|
|||
import type { BaseApi, DocumentMetadata, ProcessingMetadata, StreamingMetadata, Triple } from "@trustgraph/client";
|
||||
import { makeBaseApiWithRpc, } from "@trustgraph/client";
|
||||
import { Match, Option, Schema as S } from "effect";
|
||||
import type { DocumentMetadata, ProcessingMetadata, StreamingMetadata, Triple } from "@trustgraph/client";
|
||||
import { Array as A, Match, Option, Order, Schema as S } from "effect";
|
||||
|
||||
type ConfigValues = Record<string, Record<string, unknown>>;
|
||||
|
||||
export interface WorkbenchQaApi {
|
||||
readonly makeRequest: <RequestType extends object, ResponseType>(
|
||||
service: string,
|
||||
request: RequestType,
|
||||
timeout?: number,
|
||||
retries?: number,
|
||||
flow?: string,
|
||||
) => Promise<ResponseType>;
|
||||
readonly makeRequestMulti: <RequestType extends object, ResponseType>(
|
||||
service: string,
|
||||
request: RequestType,
|
||||
receiver: (resp: unknown) => boolean,
|
||||
timeout?: number,
|
||||
retries?: number,
|
||||
flow?: string,
|
||||
) => Promise<ResponseType>;
|
||||
}
|
||||
|
||||
const UnknownRecord = S.Record(S.String, S.Unknown);
|
||||
const ConfigValuesRecord = S.Record(S.String, UnknownRecord);
|
||||
|
||||
|
|
@ -358,7 +375,9 @@ function dispatchFlow(state: MockState, request: Record<string, unknown>): unkno
|
|||
const id = stringValue(request["flow-id"], "default");
|
||||
return { flow: encodeJson(state.flows.definitions[id] ?? { id, description: "Mock flow" }) };
|
||||
}
|
||||
if (operation === "list-blueprints") return { "blueprint-names": Object.keys(state.flows.blueprints).sort() };
|
||||
if (operation === "list-blueprints") {
|
||||
return { "blueprint-names": A.sort(Object.keys(state.flows.blueprints), Order.String) };
|
||||
}
|
||||
if (operation === "get-blueprint") {
|
||||
const name = stringValue(request["blueprint-name"], "qa-blueprint");
|
||||
return { "blueprint-definition": encodeJson(state.flows.blueprints[name] ?? {}) };
|
||||
|
|
@ -563,33 +582,37 @@ function dispatchStream<ResponseType>(
|
|||
return Promise.resolve({} as ResponseType);
|
||||
}
|
||||
|
||||
export function makeMockBaseApi(fixture: MockWorkbenchFixture = {}): BaseApi {
|
||||
export function makeMockBaseApi(fixture: MockWorkbenchFixture = {}): WorkbenchQaApi {
|
||||
const state = createState(fixture);
|
||||
const token = state.settings.apiKey.length > 0 ? state.settings.apiKey : undefined;
|
||||
return makeBaseApiWithRpc(state.settings.user, token, state.settings.gatewayUrl, {
|
||||
dispatch: (input) =>
|
||||
return {
|
||||
makeRequest: <RequestType extends object, ResponseType>(
|
||||
service: string,
|
||||
request: RequestType,
|
||||
_timeout?: number,
|
||||
_retries?: number,
|
||||
flow?: string,
|
||||
): Promise<ResponseType> =>
|
||||
Promise.resolve(
|
||||
dispatchRequest(
|
||||
state,
|
||||
input.service,
|
||||
input.request,
|
||||
input.flow,
|
||||
),
|
||||
service,
|
||||
request as Record<string, unknown>,
|
||||
flow,
|
||||
) as ResponseType,
|
||||
),
|
||||
dispatchStream: (input, receiver) =>
|
||||
dispatchStream(state, input.service, (message) => {
|
||||
makeRequestMulti: <RequestType extends object, ResponseType>(
|
||||
service: string,
|
||||
_request: RequestType,
|
||||
receiver: (resp: unknown) => boolean,
|
||||
): Promise<ResponseType> =>
|
||||
dispatchStream(state, service, (message) => {
|
||||
const chunk = message as { response?: unknown; complete?: boolean };
|
||||
return receiver({
|
||||
response: chunk.response,
|
||||
complete: chunk.complete === true,
|
||||
});
|
||||
}).then(() => undefined),
|
||||
subscribe: (listener) => {
|
||||
listener({ status: token === undefined ? "connected" : "connected" });
|
||||
return () => {};
|
||||
},
|
||||
close: () => Promise.resolve(),
|
||||
});
|
||||
}).then(() => ({}) as ResponseType),
|
||||
};
|
||||
}
|
||||
|
||||
export function qaSettingsFromFixture(fixture: MockWorkbenchFixture = {}) {
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
{"exemptions":[],"baseline":[{"rule":"no-error-throw","path":"packages/workbench/src/main.tsx","count":1},{"rule":"no-native-sort","path":"packages/client/src/socket/trustgraph-socket.ts","count":2},{"rule":"no-native-sort","path":"packages/flow/src/config/service.ts","count":3},{"rule":"no-native-sort","path":"packages/flow/src/cores/service.ts","count":1},{"rule":"no-native-sort","path":"packages/flow/src/flow-manager/service.ts","count":1},{"rule":"no-native-sort","path":"packages/flow/src/librarian/service.ts","count":1},{"rule":"no-native-sort","path":"packages/flow/src/retrieval/graph-rag.ts","count":1},{"rule":"no-native-sort","path":"packages/mcp/src/server-effect.ts","count":1},{"rule":"no-native-sort","path":"packages/workbench/src/atoms/workbench.ts","count":2},{"rule":"no-native-sort","path":"packages/workbench/src/components/chat/explain-graph.tsx","count":1},{"rule":"no-native-sort","path":"packages/workbench/src/pages/graph.tsx","count":1},{"rule":"no-native-sort","path":"packages/workbench/src/qa/mock-api.ts","count":1},{"rule":"no-schema-suffix","path":"packages/base/src/schema/primitives.ts","count":1}]}
|
||||
{"exemptions":[{"rule":"no-error-throw","path":"packages/workbench/src/main.tsx","reason":"Host boundary: Workbench cannot render without the #root mount element."}],"baseline":[]}
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@
|
|||
|
||||
import { BunRuntime } from "@effect/platform-bun";
|
||||
import * as BunHttpClient from "@effect/platform-bun/BunHttpClient";
|
||||
import { DispatchInput, makeEffectRpcClient } from "@trustgraph/client";
|
||||
import { Config, Effect, Option as O, Schema as S } from "effect";
|
||||
import { HttpClient, HttpClientRequest } from "effect/unstable/http";
|
||||
|
||||
|
|
@ -19,17 +20,6 @@ const DEFAULT_LLM_MODEL = "qwen2.5:0.5b";
|
|||
const DEFAULT_FALKORDB_URL = "redis://localhost:6380";
|
||||
const DEFAULT_PIPELINE_WAIT = 20;
|
||||
|
||||
interface RpcSocket {
|
||||
close: () => void;
|
||||
makeRequest: <RequestType extends object, ResponseType>(
|
||||
service: string,
|
||||
request: RequestType,
|
||||
timeout?: number,
|
||||
retries?: number,
|
||||
flow?: string,
|
||||
) => Promise<ResponseType>;
|
||||
}
|
||||
|
||||
interface PipelineConfig {
|
||||
readonly gatewayUrl: string;
|
||||
readonly gatewaySecret: string | undefined;
|
||||
|
|
@ -299,33 +289,24 @@ const testTextCompletion = (config: PipelineConfig) => catchTest("Text completio
|
|||
}));
|
||||
|
||||
const testWebSocket = (config: PipelineConfig) => catchTest("Effect RPC WebSocket", Effect.gen(function* () {
|
||||
let socket: RpcSocket | undefined;
|
||||
return yield* Effect.gen(function* () {
|
||||
const { createTrustGraphSocket } = yield* Effect.tryPromise({
|
||||
try: () => import("../packages/client/src/socket/trustgraph-socket.js"),
|
||||
catch: (cause) => pipelineError("websocket.import", cause),
|
||||
});
|
||||
const gatewayWsUrl = config.gatewayUrl.replace(/^http/, "ws").replace(/\/$/, "");
|
||||
const response = yield* Effect.scoped(
|
||||
Effect.gen(function*() {
|
||||
const client = yield* makeEffectRpcClient(`${gatewayWsUrl}/api/v1/rpc`);
|
||||
return yield* client.dispatch(
|
||||
DispatchInput.make({
|
||||
scope: "global",
|
||||
service: "config",
|
||||
request: { operation: "list", keys: [] },
|
||||
}),
|
||||
{ timeoutMs: 5_000 },
|
||||
);
|
||||
}),
|
||||
).pipe(Effect.timeout("5 seconds"));
|
||||
|
||||
const gatewayWsUrl = config.gatewayUrl.replace(/^http/, "ws").replace(/\/$/, "");
|
||||
socket = createTrustGraphSocket(
|
||||
"pipeline",
|
||||
config.gatewaySecret,
|
||||
`${gatewayWsUrl}/api/v1/rpc`,
|
||||
);
|
||||
const response = yield* Effect.tryPromise({
|
||||
try: () =>
|
||||
socket?.makeRequest<Record<string, unknown>, Record<string, unknown>>(
|
||||
"config",
|
||||
{ operation: "list", keys: [] },
|
||||
5000,
|
||||
) ?? Promise.resolve({}),
|
||||
catch: (cause) => pipelineError("websocket.request", cause),
|
||||
}).pipe(Effect.timeout("5 seconds"));
|
||||
|
||||
log("websocket/rpc-response", response);
|
||||
pass("Effect RPC WebSocket round-trip works");
|
||||
return true;
|
||||
}).pipe(Effect.ensuring(Effect.sync(() => socket?.close())));
|
||||
log("websocket/rpc-response", response);
|
||||
pass("Effect RPC WebSocket round-trip works");
|
||||
return true;
|
||||
}));
|
||||
|
||||
// ─── Librarian Tests ──────────────────────────────────────────────────
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue