Commit graph

59 commits

Author SHA1 Message Date
elpresidank
d19167b566 Use MutableHashSet for Qdrant collection caches 2026-06-02 09:21:19 -05:00
elpresidank
09d34fb4d4 Normalize term translation with Effect Match 2026-06-02 09:11:33 -05:00
elpresidank
0fb10aca73 Migrate metrics to Effect primitives 2026-06-02 08:52:29 -05:00
elpresidank
39db6d8235 Use tagged errors for dispatch RPC 2026-06-02 08:41:36 -05:00
elpresidank
1d45307387 Migrate Claude provider to Effect AI 2026-06-02 08:36:55 -05:00
elpresidank
24a2447cc3 Use Effect primitives for AI and response fanout 2026-06-02 08:26:50 -05:00
elpresidank
1218e827d4 Fail pending request responses on stop 2026-06-02 06:19:32 -05:00
elpresidank
89ef3dbbbf Harden gateway dispatcher effects 2026-06-02 05:14:58 -05:00
elpresidank
fe4f5777c9 Make text completion provider layers effectful 2026-06-02 05:09:15 -05:00
elpresidank
f7f29c4df9 Handle gateway RPC socket causes in Effect 2026-06-02 04:42:32 -05:00
elpresidank
32fc7ea32d Share text completion stream helpers 2026-06-02 04:33:48 -05:00
elpresidank
9d3f745fb0 Make Ollama embeddings layer effectful 2026-06-02 04:24:14 -05:00
elpresidank
8287e1cf93 Add fakeable Qdrant Effect services 2026-06-02 04:10:03 -05:00
elpresidank
d38ce475fd Scope FalkorDB triples clients 2026-06-02 03:55:39 -05:00
elpresidank
ce5838db1d Add Effect-native gateway streaming path 2026-06-02 03:43:21 -05:00
elpresidank
44110c5bb4 Add typed flow spec accessors 2026-06-02 03:23:23 -05:00
elpresidank
abb6f3aed0 Add schema-backed parameter spec accessors 2026-06-02 03:10:43 -05:00
elpresidank
b51dc33786 Narrow provider status errors with Predicate 2026-06-02 03:00:52 -05:00
elpresidank
b3aaa9a4af Simplify text completion generator boundary 2026-06-02 02:58:19 -05:00
elpresidank
bfb5494552 Remove provider stream sentinel assertions 2026-06-02 02:55:06 -05:00
elpresidank
74955d6041 Use native runtimes for flow entrypoints 2026-06-02 02:34:03 -05:00
elpresidank
710656be26 Move librarian state into synchronized refs 2026-06-02 01:59:51 -05:00
elpresidank
5f783832e9 Type librarian service runtime loop 2026-06-02 01:47:15 -05:00
elpresidank
e6384e65b9 Replace librarian throw helpers with Effect failures 2026-06-02 01:39:04 -05:00
elpresidank
459347ad12 Model librarian upload schema boundaries 2026-06-02 01:34:46 -05:00
elpresidank
3809a38c46 Migrate flow manager to ref-backed Effect state 2026-06-02 01:26:40 -05:00
elpresidank
ba64fc5add Normalize flow service runtime entrypoints 2026-06-02 01:13:40 -05:00
elpresidank
0da0df81c4 Migrate knowledge core service to ref-backed Effect state 2026-06-02 01:07:12 -05:00
elpresidank
5979d38b99 Remove RAG requestor Promise bridges 2026-06-02 00:54:47 -05:00
elpresidank
88db18fbda Migrate config service to ref-backed Effect runtime 2026-06-02 00:40:44 -05:00
elpresidank
b4ee2b691f Migrate strict Effect runtime surfaces 2026-06-02 00:22:04 -05:00
elpresidank
f6878d4dd7 Enforce strict Effect tsgo migrations 2026-06-01 23:19:54 -05:00
elpresidank
64fb23e7d0 Make gateway dispatcher requestors Effect-scoped 2026-06-01 22:17:50 -05:00
elpresidank
b58e51bf75 Update Effect packages to beta 75 2026-06-01 21:16:38 -05:00
elpresidank
dca2786828 Remove native classes from TS runtime 2026-06-01 20:26:47 -05:00
elpresidank
952daf325d Stabilize TS workbench QA and RPC timeouts 2026-06-01 17:23:34 -05:00
elpresidank
3515106670 Advance TS port Effect workbench 2026-06-01 16:22:25 -05:00
elpresidank
5776171d27 saving 2026-05-30 09:56:23 -05:00
elpresidank
ffd97375a8 saving 2026-05-12 08:06:58 -05:00
elpresidank
6ac5446a76 feat(mcp-tool): wire McpToolService into deploy stack
Three pieces, all required for an end-to-end MCP tool call:

* McpToolService used generic spec names "request"/"response" instead of
  "mcp-tool-request"/"mcp-tool-response", so RequestResponseSpec's
  flow-config topic lookup never matched and consumers bound to literal
  subjects nobody else publishes to.

* Add entrypoints/mcp-tool.mjs (mirrors agent/librarian entrypoints) so
  the service can be launched in the prebuilt trustgraph-ts image.

* Add a `mcp-tool` service block to deploy/docker-compose.yml.

With these three fixes plus a `mcp-tool-request`/`mcp-tool-response`
entry in each flow's topics map, the agent ReAct loop can now invoke
remote MCP tools (verified end-to-end against Brave Search and FireCrawl).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 22:16:37 -05:00
elpresidank
ee45cb4850 feat: fix RAG pipelines, Beep Graph branding, PWA, and ambient glow UI
Pipeline fixes:
- Fix agent getting empty response from graph-rag by combining answer +
  explain data in single message (RequestResponse returns first msg)
- Fix Doc RAG pipeline: add content field to Qdrant doc payload, seed 10
  document chunks, fix type mismatches across base/flow/client
- Forward explainability events from agent's KnowledgeQuery to client
- Add "agent" to TERM_BEARING_RESPONSE_SERVICES for triple translation
- Fix embeddings env var (OLLAMA_URL), user/collection threading, edge
  scoring threshold, and various protocol mismatches

Branding:
- Rename TrustGraph → Beep Graph (title, sidebar, settings, about)
- Custom lambda + ThugLife pixel glasses SVG logo component
- Forest green color palette (brand-50 through brand-900)
- SVG favicon + PNG icons (16/32/180/192/512)
- PWA manifest with service worker for offline shell caching
- Splash screen with animated logo pulse on app load
- Ambient glow background with drifting green radial blobs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 10:19:10 -05:00
elpresidank
b854b56558 feat: MCP tool client infrastructure for agent extensibility
Add the full MCP tool pipeline enabling agents to invoke external tools
(like Brave Search) via MCP servers:

- Add ToolRequest/ToolResponse types and mcp-tool topics to @trustgraph/base
- Create McpToolService (FlowProcessor) that connects to external MCP servers
  via @modelcontextprotocol/sdk StreamableHTTP transport
- Add createMcpTool() to wire MCP tools into the agent's ReAct loop
- Implement config-driven tool registration in AgentService with backward-
  compatible fallback to hardcoded tools
- Add tool filtering by group and state (port of Python tool_filter.py)
- Register mcp-tool in gateway dispatcher and export from @trustgraph/flow
- Fix flow restart race condition: skip restart when flow definitions unchanged
- Update seed config with MCP server config and tool definitions
- Add run scripts for MCP tool service and Brave Search MCP server

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 05:45:46 -05:00
elpresidank
f2b376abef fix: FalkorDB result parsing, embeddings routing, triples query response, graph visualization
- Fix FalkorDB triples query: client v5 returns objects not arrays, use named field access
- Fix embeddings service: align spec names to "embeddings-request"/"embeddings-response"
- Fix client triplesQuery: read `triples` field instead of `response` from backend
- Fix graph page crash: guard against non-array triples, accept literals as entity nodes
- Add seed:demo script for AI industry knowledge graph (254 triples, 64 entities)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 04:59:36 -05:00
elpresidank
580ee319a3 fix: prevent dispatcher race condition via promise-based lazy init
Store the initialization Promise in the requestors map synchronously
before yielding, so concurrent callers for the same key await the same
instance — prevents orphaned RequestResponse objects and duplicate NATS
subscriptions. Mirrors upstream fix 8f18ba02.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:11:11 -05:00
elpresidank
5e3929a883 fix: comprehensive QA audit — light mode, accessibility, error handling, code quality
- Fix light mode: theme-aware graph node labels, remove prose-invert for
  theme-safe markdown, add brand/semantic color overrides for light backgrounds
- Add 404 catch-all route redirecting unknown paths to /chat
- FalkorDB: add .catch() to connectPromise, add ensureConnected() to all
  store methods (createLiteral, relateNode, relateLiteral, deleteCollection)
- Accessibility: dialog role/aria-modal, toast aria-live, dismiss/zoom/search
  button aria-labels, close panel aria-label
- Lazy-load ForceGraph2D (splits 189KB into separate chunk, main bundle -26%)
- Cap conversation localStorage at 200 messages to prevent quota overflow
- Fix pnpm test: add --passWithNoTests to cli/mcp packages
- Add upload error notification instead of silent catch
- Remove unused class-variance-authority dep and dead tabs.tsx component
- Add @types/node to flow package devDependencies
- Remove stale FIXME comment in messages.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 09:15:59 -05:00
elpresidank
3a80872482 fix: comprehensive QA — resolve 13 bugs, add UX improvements across all services
Client SDK: add .catch() to graphRagStreaming/documentRagStreaming (silent timeout),
null-guard JSON.parse in getPrompts/getSystemPrompt/getPrompt.

Backend: implement "getvalues" config operation for token costs, null-check
createTerm() in FalkorDB triples query, add knowledge-cores service entrypoint
and Docker entry, return proper HTTP 400/404 for gateway error responses.

Workbench: cancel button + elapsed timer for chat, clear agent spinner on error,
flow dialog inline validation, responsive header wrapping, knowledge cores
loading timeout, sidebar/page naming consistency, theme toggle indicator.

Infrastructure: enable Grafana Explore for viewers, add gateway Prometheus
scrape target, fix RAG pipeline dashboard layout (6 panels visible),
filter Service Health to configured targets only.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 05:20:10 -05:00
elpresidank
72870a7e2e feat: add unit tests, Docker polish, and workbench UX improvements
Unit tests: Consumer class (7), recursive-splitter (10), parseJsonResponse (11) — 28 total.
Docker: add 5 commented LLM provider services, dev compose override, .env.example.
Workbench: chat persistence, error boundary, disconnect banner, prompts error handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 03:51:29 -05:00
elpresidank
c7eefee607 feat: add Docker entrypoints, LLM providers, pipeline hardening, workbench pages
Phase 9 — four parallel workstreams:

- Stream A: 14 Docker entrypoints for containerized deployment
- Stream B: Pipeline hardening — robust JSON parsing, LLM retry logic,
  consumer negative-ack, FalkorDB test import fix
- Stream C: Azure OpenAI, OpenAI-compatible, and Mistral LLM providers
- Stream D: Workbench Prompts, Token Cost, Knowledge Cores pages +
  Settings feature switches

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 03:22:55 -05:00
elpresidank
50fb311d2d feat: real PDF pipeline test — end-to-end knowledge extraction working
Add full pipeline test that generates a real PDF, processes it through
the entire pipeline, and verifies knowledge lands in FalkorDB:

- Create test PDF generator using pdf-lib (2-page doc about Acme Corp)
- Add testFullPipeline() to integration tests with store verification
- Fix FalkorDB client connect() — createClient returns unconnected client
  in both TriplesStore and TriplesQuery classes

Results: PDF decoded (2 pages) → chunked (2 chunks) → extracted
(4 relationships) → 16 triples stored in FalkorDB including:
  alice-johnson → is-a-senior-engineer → acme-corporation
  cloudsync → uses-aws-for-hosting → amazon-web-services
  provenance: pages → prov:wasDerivedFrom → source document

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 02:19:12 -05:00
elpresidank
5bc7a1b6fc fix: resolve FlowProcessor topic collisions, librarian timeout, tests
Two bugs found during end-to-end testing:

1. FlowProcessor never restarted flows when config changed — it only
   started them once. Stale NATS JetStream data from previous sessions
   caused services to bind to wrong topics. Fix: stop and restart flows
   on every config push that includes flow definitions.

2. Gateway publishToTopic sent messages without an id property. Pipeline
   FlowProcessor handlers check properties.id and silently return if
   missing. Fix: auto-generate a message id when publishing to topics.

Both fixes validated: 13/13 integration tests passing, PDF decoder
correctly receives and processes document messages through the pipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:53:55 -05:00