diff --git a/docs/README.api-docs.md b/docs/README.api-docs.md new file mode 100644 index 00000000..8f7fecb8 --- /dev/null +++ b/docs/README.api-docs.md @@ -0,0 +1,48 @@ + +# Auto-generating docs + +## REST and WebSocket API Documentation + +- `specs/build-docs.sh` - Builds the REST and websocket documentation from the + OpenAPI and AsyncAPI specs. + +## Python API Documentation + +The Python API documentation is generated from docstrings using a custom Python script that introspects the `trustgraph.api` package. + +### Prerequisites + +The trustgraph package must be importable. If you're working in a development environment: + +```bash +cd trustgraph-base +pip install -e . +``` + +### Generating Documentation + +From the docs directory: + +```bash +cd docs +python3 generate-api-docs.py > python-api.md +``` + +This generates a single markdown file with complete API documentation showing: +- Installation and quick start guide +- Import statements for each class/type +- Full docstrings with examples +- Table of contents organized by category + +### Documentation Style + +All docstrings follow Google-style format: +- Brief one-line summary +- Detailed description +- Args section with parameter descriptions +- Returns section +- Raises section (when applicable) +- Example code blocks with proper syntax highlighting + +The generated documentation shows the public API exactly as users import it from `trustgraph.api`, without exposing internal module structure. + diff --git a/docs/README.development.md b/docs/README.development.md deleted file mode 100644 index 82e41e86..00000000 --- a/docs/README.development.md +++ /dev/null @@ -1,82 +0,0 @@ - -# Contributing - -## Generally - -Branching is good discipline to get into with multiple people working -on the same repo for different reasons. - -To create a branch... - -- `git checkout -b etl` # to create the branch and check it out -- `git push` # to push the branch head to the upstream repo. You get an error and a command to run. You don't have to do this straight away, but I like to get the BS admin out the way. At this stage your branch HEAD points to the head of main. - -## Adding a new module - -So, to add a new module... - -- It needs a name. Say `kg-mymodule` but you can call it what you like. -- It also needs a place in the Python package hierarchy, because it's - basically going to be its own loadable module. We have a `trustgraph.kg` - module it can be a child of. So, you need a directory - `trustgraph/kg/mymodule` -- You need three files: - - `__init__.py` which defines the module entry point. - - Then, `__main__.py` means the module is executable. - - Finally a module to contain the code, let's call it `extract.py`. - The name doesn't matter but it has to match what's in `__init__.py` and - `__main__.py`. -- The easiest way to get start is maybe make a copy of an existing module. -- `cp -r trustgraph/kg/extract_relationships trustgraph/kg/mymodule/` -- Finally you need a script entry point, in `scripts`. Copy - `scripts/kg-extract-relationships` to `scripts/kg-mymodule` -- In that `kg-mymodule` file, change the import line to import your module, - `trustgraph.kg.mymodule`. - -## Development testing - -To run your module, you don't need to have it running in a container. -It can connect to Pulsar. - -The plumbing for your new module pretty needs to be right. Look at the -input_queue, output_queue and subscriber settings near the top of your -new module code. - -So, before changing the code any more, if you copied an existing module, -check the plumbing works with your renamed module. - -To run standalone, it is recommended to take an existing docker-compose -file, run everything you need except the module you're developing. - -Then when you launch with docker compose, you'll get everything running -except your module. - -To run your module, you need to set up the Python environment as you did -in the quickstart e.g. run `. env/bin/activate` and `export PYTHONPATH=.` - -You're not running kg-mymodule in a container, so it can't use docker -internal DNS to get to the containers, but the docker compose file -exposes everything to the host anyway. You should be able to access Pulsar -on localhost port 6650, for instance. - -You should be able to run your module on the host and point at Pulsar thus: - -```bash -scripts/kg-mymodule -p pulsar://localhost:6650 -``` - -You could try loading data, and check some stuff ends up in the graph. If you get that far you're ready to hack the contents of extract.py to -do what you want. - -## Structure of the code - -The Processor class, `run` method is where all the fun takes place. - -``` - while True: - msg = self.consumer.receive() -``` - -That bit :point_up: is a loop which is executed every time a new message -arrives. - diff --git a/docs/README.md b/docs/README.md index f760d55c..35d70763 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,59 +1,21 @@ -# TrustGraph Documentation Index +# TrustGraph Documentation -Welcome to the TrustGraph documentation. This directory contains comprehensive guides for using TrustGraph's APIs and command-line tools. +Welcome to TrustGraph! For comprehensive documentation, please visit: -## Documentation Overview +## 📖 [https://docs.trustgraph.ai](https://docs.trustgraph.ai) -### 📚 [API Documentation](apis/README.md) -Complete reference for TrustGraph's APIs, including REST, WebSocket, Pulsar, and Python SDK interfaces. Learn how to integrate TrustGraph services into your applications. +The main documentation site includes: -### 🖥️ [CLI Documentation](cli/README.md) -Comprehensive guide to TrustGraph's command-line interface. Includes detailed documentation for all CLI commands, from system administration to knowledge graph management. - -### 🚀 [Quick Start Guide](README.quickstart-docker-compose.md) -Step-by-step guide to get TrustGraph running using Docker Compose. Perfect for first-time users who want to quickly deploy and test TrustGraph. +- **[Overview](https://docs.trustgraph.ai/overview)** - Introduction to TrustGraph concepts and architecture +- **[Guides](https://docs.trustgraph.ai/guides)** - Step-by-step tutorials and how-to guides +- **[Deployment](https://docs.trustgraph.ai/deployment)** - Deployment options and configuration +- **[Reference](https://docs.trustgraph.ai/reference)** - API specifications and CLI documentation ## Getting Started -If you're new to TrustGraph, we recommend starting with the -[Compose - Quick Start Guide](README.quickstart-docker-compose.md) -to get a working system up and running quickly. +**New to TrustGraph?** Start with the [Overview](https://docs.trustgraph.ai/overview) to understand the system. -For developers integrating TrustGraph into applications, check out the -[API Documentation](apis/README.md) to understand the available interfaces. +**Ready to deploy?** Check out the [Deployment Guide](https://docs.trustgraph.ai/deployment). -For system administrators and power users, the -[CLI Documentation](cli/README.md) provides detailed information about all -command-line tools. +**Integrating with code?** See the [API Reference](https://docs.trustgraph.ai/reference) for REST, WebSocket, and SDK documentation. -## Ways to deploy - -If you haven't deployed TrustGraph before, the 'compose' deployment -mentioned above is going to be the least commitment of setting things up: -See [Quick Start Guide](README.quickstart-docker-compose.md) - -Other deployment mechanisms include: -- [Scaleway Kubernetes deployment using Pulumi](https://github.com/trustgraph-ai/pulumi-trustgraph-scaleway) -- [Intel Gaudi and GPU](https://github.com/trustgraph-ai/trustgraph-tiber-cloud) - tested on Intel Tiber cloud -- [Azure Kubernetes deployment using Pulumi](https://github.com/trustgraph-ai/pulumi-trustgraph-aks) -- [AWS EC2 single instance deployment using Pulumi](https://github.com/trustgraph-ai/pulumi-trustgraph-ec2) -- [GCP GKE cloud deployment using Pulumi](https://github.com/trustgraph-ai/pulumi-trustgraph-gke) -- [RKE Kubernetes on AWS deployment using Pulumi](https://github.com/trustgraph-ai/pulumi-trustgraph-aws-rke) -- It should be possible to deploy on AWS EKS, but we haven't been able to - script anything reliable so far. - -## Support - -For questions, issues, or contributions: - -- **GitHub Issues**: Report bugs and feature requests -- **Documentation**: This documentation covers most use cases -- **Community**: Join discussions and share experiences - -## Related Resources - -- [TrustGraph GitHub Repository](https://github.com/trustgraph-ai/trustgraph) -- [Docker Hub Images](https://hub.docker.com/u/trustgraph) -- [Example Notebooks](https://github.com/trustgraph-ai/example-notebooks) - - shows some example use of various APIs. - diff --git a/docs/README.quickstart-docker-compose.md b/docs/README.quickstart-docker-compose.md deleted file mode 100644 index cf8a042f..00000000 --- a/docs/README.quickstart-docker-compose.md +++ /dev/null @@ -1,185 +0,0 @@ - -# Getting Started - -## Preparation - -> [!TIP] -> Before launching `TrustGraph`, be sure to have the `Docker Engine` or `Podman Machine` installed and running on the host machine. -> -> - [Install the Docker Engine](https://docs.docker.com/engine/install/) -> - [Install the Podman Machine](http://podman.io/) - -> [!NOTE] -> `TrustGraph` has been tested on `Linux` and `MacOS` with `Docker` and `Podman`. `Windows` deployments have not been tested. - -> [!TIP] -> If using `Podman`, the only change will be to substitute `podman` instead of `docker` in all commands. - -## Create the configuration - -This guide talks you through the Compose file launch, which is the easiest -way to lauch on a standalone machine, or a single cloud instance. -See [README](README.md) for links to other deployment mechanisms. - -To create the deployment configuration, go to the -[deployment portal](https://config-ui.demo.trustgraph.ai/) and follow the -instructions. -- Select Docker Compose or Podman Compose as the deployment - mechanism. -- Use Cassandra for the graph store, it's easiest and most tested. -- Use Qdrant for the vector store, it's easiest and most tested. -- Chunker: Recursive, chunk size of 1000, 50 overlap should be fine. -- Pick your favourite LLM model: - - If you have enough horsepower in a local GPU, LMStudio is an easy - starting point for a local model deployment. Ollama is fairly easy. - - VertexAI on Google is relatively straightforward for a cloud - model-as-a-service LLM, and you can get some free credits. -- Max output tokens as per the model, 2048 is safe. -- Customisation, check LLM Prompt Manager and Agent Tools. -- Finish deployment, Generate and download the deployment bundle. - Read the extra deploy steps on that page. - -## Preparing TrustGraph - -Below is a step-by-step guide to deploy `TrustGraph`, extract knowledge from a PDF, build the vector and graph stores, and finally generate responses with Graph RAG. - -### Install requirements - -``` -python3 -m venv env -. env/bin/activate -pip install trustgraph-cli -``` -## Running TrustGraph - -``` -docker-compose -f docker-compose.yaml up -d -``` - -After running the chosen `Docker Compose` file, all `TrustGraph` services will launch and be ready to run `Naive Extraction` jobs and provide `RAG` responses using the extracted knowledge. - -### Verify TrustGraph Containers - -On first running a `Docker Compose` file, it may take a while (depending on your network connection) to pull all the necessary components. Once all of the components have been pulled. - -A quick check that TrustGraph processors have started: - -``` -tg-show-processor-state -``` - -Processors start quickly, but can take a while (~60 seconds) for -Pulsar and Cassandra to start. - -If you have any concerns, -check that the TrustGraph containers are running: - -``` -docker ps -``` - -Any containers that have exited unexpectedly can be found by checking the `STATUS` field using the following: - -``` -docker ps -a -``` - -> [!TIP] -> Before proceeding, allow the system to stabilize. A safe warm up period is `120 seconds`. If services seem to be "stuck", it could be because services did not have time to initialize correctly and are trying to restart. Waiting `120 seconds` before launching any scripts should provide much more reliable operation. - -### Everything running - -An easy way to check all the main start is complete: - -``` -tg-show-flows -``` - -You should see a default flow. If you see an error, leave it and try again. - -### Load some sample documents - -``` -tg-load-sample-documents -``` - -### Workbench - -A UI is launched on port 8888, see if you can see it at -[http://localhost:8888/](http://localhost:8888/) - -Verify things are working: -- Go to the prompts page see that you can see some prompts -- Go to the library page, and check you can see the sample documents you - just loaded. - -### Load a document - -- On the library page, select a document. Beyond State Vigilance is a - smallish doc to work with. -- Select the doc by clicking on it. -- Select Submit at the bottom of the screen on the action bar. -- Select a processing flow, use the default. -- Click submit. - -### Look in Grafana - -A Grafana is launched on port 3000, see if you can see it at -[http://localhost:3000/](http://localhost:3000/) - -- Login as admin, password admin. -- Skip the password change screen / change the password. -- Verify things are working by selecting the TrustGraph dashboard -- After a short while, you should see the backlog rise to a few hundred - document chunks. - -Once some chunks are loaded, you can start to work with the document. - -### Graph Parsing - -To check that the knowledge graph is successfully parsing data: - -``` -tg-show-graph -``` - -The output should be a set of semantic triples in [N-Triples](https://www.w3.org/TR/rdf12-n-triples/) format. - -``` -http://trustgraph.ai/e/enterprise http://trustgraph.ai/e/was-carried to altitude and released for a gliding approach and landing at the Mojave Desert test center. -http://trustgraph.ai/e/enterprise http://www.w3.org/2000/01/rdf-schema#label Enterprise. -http://trustgraph.ai/e/enterprise http://www.w3.org/2004/02/skos/core#definition A prototype space shuttle orbiter used for atmospheric flight testing. -``` - -### Work with the document - -Back on the workbench, click on the 'Vector search' tab, and -search for something e.g. state. You should see some search results. -Click on results to start exploring the knowledge graph. - -Click on Graph view on an explored page to visualize the graph. - -### Queries over the document - -On workbench, click Graph RAG and enter a question e.g. -What is this document about? - -### Shutting Down TrustGraph - -When shutting down `TrustGraph`, it's best to shut down all Docker containers and volumes. Run the `docker compose down` command that corresponds to your model and graph store deployment: - -``` -docker compose -f document-compose.yaml down -v -t 0 -``` - -> [!TIP] -> To confirm all Docker containers have been shut down, check that the following list is empty: -> ``` -> docker ps -> ``` -> -> To confirm all Docker volumes have been removed, check that the following list is empty: -> ``` -> docker volume ls -> ``` - diff --git a/docs/api.html b/docs/api.html new file mode 100644 index 00000000..201771ec --- /dev/null +++ b/docs/api.html @@ -0,0 +1,3497 @@ + + + +
+ +Download OpenAPI specification:
REST API for TrustGraph - an AI-powered knowledge graph and RAG system.
+The API provides access to:
+Fixed endpoints accessible via /api/v1/{kind}:
config - Configuration managementflow - Flow lifecycle and blueprintslibrarian - Document library managementknowledge - Knowledge graph core managementcollection-management - Collection metadataRequire running flow instance, accessed via /api/v1/flow/{flow}/service/{kind}:
Bearer token authentication when GATEWAY_SECRET environment variable is set.
+Include token in Authorization header:
Authorization: Bearer <token>
+
+If GATEWAY_SECRET is not set, API runs without authentication (development mode).
All endpoints may return errors in this format:
+{
+ "error": {
+ "type": "gateway-error",
+ "message": "Timeout"
+ }
+}
+
+Manage TrustGraph configuration including flows, prompts, token costs, parameter types, and more.
+Get the complete system configuration including all flows, prompts, token costs, etc.
+List all configuration items of a specific type (e.g., all flows, all prompts).
+Retrieve specific configuration items by type and key.
+Create or update configuration values.
+Delete configuration items.
+flow - Flow instance definitionsflow-blueprint - Flow blueprint definitions (stored separately from flow instances)prompt - Prompt templatestoken-cost - Model token pricingparameter-type - Parameter type definitionsinterface-description - Interface descriptionsThe config service manages stored configuration.
+The flow service (/api/v1/flow) manages running flow instances.
| operation required | string Enum: "config" "list" "get" "put" "delete" Operation to perform: +
|
| type | string Configuration type (required for list, get, put, delete operations). +Common types: flow, prompt, token-cost, parameter-type, interface-description + |
Array of objects Keys to retrieve (for get operation) or delete (for delete operation) + | |
Array of objects Values to set/update (for put operation) + |
{- "operation": "config"
}{- "version": 42,
- "config": {
- "flow": {
- "default": {
- "blueprint-name": "document-rag+graph-rag",
- "description": "Default flow",
- "interfaces": {
- "agent": {
- "request": "non-persistent://tg/request/agent:default",
- "response": "non-persistent://tg/response/agent:default"
}
}
}
}, - "prompt": {
- "system": "You are a helpful AI assistant"
}, - "token-cost": {
- "gpt-4": {
- "prompt": 0.03,
- "completion": 0.06
}
}
}
}Manage flow instances and blueprints.
+The flow service manages running flow instances.
+The config service (/api/v1/config) manages stored configuration.
Start a new flow instance from a blueprint. The blueprint must exist (either built-in or created via put-blueprint).
+Parameters are resolved from:
+Stop a running flow instance. This terminates all processors and releases resources.
+List all currently running flow instances.
+Get details of a running flow including its configuration, parameters, and interface queue names.
+List all available flow blueprints (built-in and custom).
+Retrieve a blueprint definition showing its structure, parameters, processors, and interfaces.
+Create or update a flow blueprint definition.
+Blueprints define:
+Delete a custom blueprint definition. Built-in blueprints cannot be deleted.
+| operation required | string Enum: "start-flow" "stop-flow" "list-flows" "get-flow" "list-blueprints" "get-blueprint" "put-blueprint" "delete-blueprint" Flow operation: +
|
| flow-id | string Flow instance ID (required for start-flow, stop-flow, get-flow) + |
| blueprint-name | string Flow blueprint name (required for start-flow, get-blueprint, put-blueprint, delete-blueprint) + |
object Flow blueprint definition (required for put-blueprint) + | |
| description | string Flow description (optional for start-flow) + |
object Flow parameters (for start-flow). +All values are stored as strings, regardless of input type. + |
{- "operation": "start-flow",
- "flow-id": "my-flow",
- "blueprint-name": "document-rag",
- "description": "My document processing flow",
- "parameters": {
- "model": "gpt-4",
- "temperature": "0.7"
}
}{- "flow-id": "my-flow"
}Manage document library: add, remove, list documents, and control processing.
+The librarian service manages a persistent library of documents that can be:
+Add a document to the library with metadata (URL, title, author, etc.). +Documents can be added by URL or with inline content.
+Remove a document from the library by document ID or URL.
+List all documents in the library, optionally filtered by criteria.
+Start processing library documents through a flow. Documents are queued +for processing and handled asynchronously.
+Stop ongoing library document processing.
+List current processing tasks and their status.
+| operation required | string Enum: "add-document" "remove-document" "list-documents" "start-processing" "stop-processing" "list-processing" Library operation: +
|
| flow | string Flow ID + |
| collection | string Default: "default" Collection identifier + |
| user | string Default: "trustgraph" User identifier + |
| document-id | string Document identifier + |
| processing-id | string Processing task identifier + |
object (DocumentMetadata) Document metadata for library management + | |
object (ProcessingMetadata) Processing metadata for library document processing + | |
| content | string Document content (for add-document with inline content) + |
Array of objects Search criteria for filtering documents + |
{- "operation": "add-document",
- "flow": "my-flow",
- "collection": "default",
- "document-metadata": {
- "title": "Example Document",
- "author": "John Doe",
- "metadata": {
- "department": "Engineering",
- "category": "Technical"
}
}
}{- "document-metadatas": [
- {
- "title": "Document 1",
- "author": "John Doe",
- "metadata": {
- "department": "Engineering"
}
}, - {
- "title": "Document 2",
- "author": "Jane Smith",
- "metadata": {
- "department": "Research"
}
}
]
}Manage knowledge graph cores - persistent storage of triples and embeddings.
+Knowledge cores are the foundational storage units for:
+Each core has an ID, user, and collection for organization.
+List all knowledge cores for a user. Returns array of core IDs.
+Retrieve a knowledge core by ID. Returns triples and/or graph embeddings. +Response is streamed - may receive multiple messages followed by EOS marker.
+Store triples and/or graph embeddings. Creates new core or updates existing. +Can store triples only, embeddings only, or both together.
+Delete a knowledge core by ID. Removes all associated data.
+Load a knowledge core into a running flow's collection. +Makes the data available for querying within that flow instance.
+Unload a knowledge core from a flow's collection. +Removes data from flow instance but doesn't delete the core.
+The get-kg-core operation streams data in chunks:
triples or graph-embeddingseos: true to signal completion| operation required | string Enum: "list-kg-cores" "get-kg-core" "put-kg-core" "delete-kg-core" "load-kg-core" "unload-kg-core" Knowledge core operation: +
|
| user | string Default: "trustgraph" User identifier (for list-kg-cores, put-kg-core, delete-kg-core) + |
| id | string Knowledge core ID (for get, put, delete, load, unload) + |
| flow | string Flow ID (for load-kg-core) + |
| collection | string Default: "default" Collection identifier (for load-kg-core) + |
object Triples to store (for put-kg-core) + | |
object Graph embeddings to store (for put-kg-core) + |
{- "operation": "list-kg-cores",
- "user": "alice"
}{- "ids": [
- "core-123",
- "core-456",
- "core-789"
]
}Manage collection metadata for organizing documents and knowledge.
+Collections are organizational units for grouping:
+Each collection has:
+List all collections for a user. Optionally filter by tags and limit results. +Returns array of collection metadata.
+Create or update collection metadata. If collection doesn't exist, it's created. +If it exists, metadata is updated. Allows setting name, description, and tags.
+Delete a collection by user and collection ID. This removes the metadata but +typically does not delete the associated data (documents, knowledge cores).
+| operation required | string Enum: "list-collections" "update-collection" "delete-collection" Collection operation: +
|
| user | string Default: "trustgraph" User identifier + |
| collection | string Collection identifier (for update, delete) + |
| timestamp | string <date-time> ISO timestamp + |
| name | string Human-readable collection name (for update) + |
| description | string Collection description (for update) + |
| tags | Array of strings Collection tags for organization (for update) + |
| tag-filter | Array of strings Filter collections by tags (for list) + |
| limit | integer Default: 0 Maximum number of results (for list) + |
{- "operation": "list-collections",
- "user": "alice"
}{- "timestamp": "2024-01-15T10:30:00Z",
- "collections": [
- {
- "user": "alice",
- "collection": "research",
- "name": "Research Papers",
- "description": "Academic research papers on AI and ML",
- "tags": [
- "research",
- "AI",
- "academic"
]
}, - {
- "user": "alice",
- "collection": "personal",
- "name": "Personal Documents",
- "description": "Personal notes and documents",
- "tags": [
- "personal"
]
}
]
}AI agent that can understand questions, reason about them, and take actions.
+The agent service provides a conversational AI that:
+Send a question with optional:
+Responses arrive as chunks with chunk-type:
thought: Agent's reasoning processaction: Action being takenobservation: Result from actionanswer: Final response to usererror: Error occurredEach chunk may have multiple messages. Check flags:
+end-of-message: Current chunk type completeend-of-dialog: Entire conversation completeSingle response with:
+answer: Complete answerthought: Reasoning (if any)observation: Observations (if any)Include history array with previous steps to maintain context.
+Each step has: thought, action, arguments, observation.
| flow required | string Example: my-flow Flow instance ID + |
| question required | string User question or prompt for the agent + |
| state | string Agent state for continuation (optional, for multi-turn) + |
| group | Array of strings Group identifiers for collaborative agents (optional) + |
Array of objects Conversation history (optional, list of previous agent steps) + | |
| user | string Default: "trustgraph" User identifier for multi-tenancy + |
| streaming | boolean Default: false Enable streaming response delivery + |
{- "question": "What is the capital of France?",
- "user": "alice"
}{- "chunk-type": "thought",
- "content": "I need to search for information about quantum computing",
- "end-of-message": false,
- "end-of-dialog": false
}Retrieval-Augmented Generation over document embeddings.
+Document RAG combines:
+This provides grounded answers based on your document corpus.
+Enable streaming: true to receive the answer as it's generated:
response contentend-of-stream: trueWithout streaming, returns complete answer in single response.
+| flow required | string Example: my-flow Flow instance ID + |
| query required | string User query or question + |
| user | string Default: "trustgraph" User identifier for multi-tenancy + |
| collection | string Default: "default" Collection to search within + |
| doc-limit | integer [ 1 .. 100 ] Default: 20 Maximum number of documents to retrieve + |
| streaming | boolean Default: false Enable streaming response delivery + |
{- "query": "What are the key findings in the research papers?",
- "user": "alice",
- "collection": "research"
}{- "response": "The research papers present three key findings:\n1. Quantum entanglement exhibits non-local correlations\n2. Bell's inequality is violated in experimental tests\n3. Applications in quantum cryptography are promising\n",
- "end-of-stream": false
}Retrieval-Augmented Generation over knowledge graph.
+Graph RAG combines:
+This provides graph-aware answers that leverage relationships and structure.
+Enable streaming: true to receive the answer as it's generated:
response contentend-of-stream: trueWithout streaming, returns complete answer in single response.
+Control retrieval scope with multiple knobs:
+Higher limits = more context but:
+Best for queries requiring:
+| flow required | string Example: my-flow Flow instance ID + |
| query required | string User query or question + |
| user | string Default: "trustgraph" User identifier for multi-tenancy + |
| collection | string Default: "default" Collection to search within + |
| entity-limit | integer [ 1 .. 200 ] Default: 50 Maximum number of entities to retrieve + |
| triple-limit | integer [ 1 .. 100 ] Default: 30 Maximum number of triples to retrieve per entity + |
| max-subgraph-size | integer [ 10 .. 5000 ] Default: 1000 Maximum total subgraph size (triples) + |
| max-path-length | integer [ 1 .. 5 ] Default: 2 Maximum path length for graph traversal + |
| streaming | boolean Default: false Enable streaming response delivery + |
{- "query": "What connections exist between quantum physics and computer science?",
- "user": "alice",
- "collection": "research"
}{- "response": "Quantum physics and computer science intersect primarily through quantum computing.\nThe knowledge graph shows connections through:\n- Quantum algorithms (Shor's algorithm, Grover's algorithm)\n- Quantum information theory\n- Computational complexity theory\n",
- "end-of-stream": false
}Direct text completion using LLM without retrieval augmentation.
+Pure LLM generation for:
+Enable streaming: true to receive tokens as generated:
responseend-of-stream: trueWithout streaming, returns complete response in single message.
+Response includes token usage:
+in-token: Input tokens (system + prompt)out-token: Generated tokensUse text-completion when:
+Use document-rag/graph-rag when:
+| flow required | string Example: my-flow Flow instance ID + |
| system required | string System prompt that sets behavior and context for the LLM + |
| prompt required | string User prompt or question + |
| streaming | boolean Default: false Enable streaming response delivery + |
{- "system": "You are a helpful assistant that provides concise answers.",
- "prompt": "Explain the concept of recursion in programming."
}{- "response": "Recursion is a programming technique where a function calls itself\nto solve a problem by breaking it down into smaller, similar subproblems.\nEach recursive call works on a simpler version until reaching a base case.\n",
- "in-token": 45,
- "out-token": 128,
- "model": "gpt-4",
- "end-of-stream": false
}Execute stored prompt templates with variable substitution.
+The prompt service enables:
+Prompts are stored via config service (/api/v1/config) with:
{variable} placeholdersExample template:
+Summarize the following document in {max_length} words:
+
+{document}
+
+Two ways to pass variables:
+terms (explicit JSON strings):
+{
+ "terms": {
+ "document": "\"Text here...\"",
+ "max_length": "\"200\""
+ }
+}
+
+variables (auto-converted):
+{
+ "variables": {
+ "document": "Text here...",
+ "max_length": 200
+ }
+}
+
+text fieldobject field (as string)Enable streaming: true to receive response incrementally.
| flow required | string Example: my-flow Flow instance ID + |
| id required | string Prompt template ID (stored in config) + |
object Template variables as key-value pairs (values are JSON strings) + | |
object Alternative to terms - variables as native JSON values (auto-converted) + | |
| streaming | boolean Default: false Enable streaming response delivery + |
{- "id": "summarize-document",
- "terms": {
- "document": "\"This document discusses quantum computing, covering qubits, superposition, and entanglement. Applications include cryptography and optimization.\"",
- "max_length": "\"50\""
}
}{- "text": "This document provides an overview of quantum computing fundamentals and cryptographic applications.",
- "end-of-stream": false
}Convert text to embedding vectors for semantic similarity search.
+Embeddings transform text into dense vector representations that:
+Dimension count depends on embedding model:
+Unlike batch embedding APIs, this endpoint processes one text at a time. +For bulk operations, use document-load or text-load services.
+| flow required | string Example: my-flow Flow instance ID + |
| text required | string Text to convert to embedding vector + |
{- "text": "Machine learning"
}{- "vectors": [
- 0.023,
- -0.142,
- 0.089,
- 0.234,
- -0.067,
- 0.156,
- 0.201,
- -0.178,
- 0.045,
- 0.312
]
}Execute MCP (Model Context Protocol) tools for agent capabilities.
+MCP tools provide agent capabilities through standardized protocol:
+Tools extend agent capabilities beyond pure LLM generation.
+Tools are:
+Tools can return:
+Tools are registered via MCP server configuration:
+| flow required | string Example: my-flow Flow instance ID + |
| name required | string Tool name to execute + |
object Tool parameters (JSON object, auto-converted to string internally) + |
{- "name": "search",
- "parameters": {
- "query": "quantum computing",
- "limit": 10
}
}{- "text": "The result is 309"
}Query knowledge graph using subject-predicate-object patterns.
+Query RDF triples with flexible pattern matching:
+Pattern syntax supports:
+s only (all triples about that subject)p only (all uses of that property)o only (all triples with that value)Each component (s/p/o) uses RdfValue format:
+{"v": "https://example.com/entity", "e": true}{"v": "Some text", "e": false}Find all properties of an entity:
+{"s": {"v": "https://example.com/person/alice", "e": true}}
+
+Find all instances of a type:
+{
+ "p": {"v": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type", "e": true},
+ "o": {"v": "https://example.com/type/Person", "e": true}
+}
+
+Find specific relationship:
+{
+ "s": {"v": "https://example.com/person/alice", "e": true},
+ "p": {"v": "https://example.com/knows", "e": true}
+}
+
+| flow required | string Example: my-flow Flow instance ID + |
object (RdfValue) Subject filter (optional) + | |
object (RdfValue) Predicate filter (optional) + | |
object (RdfValue) Object filter (optional) + | |
| limit | integer [ 1 .. 100000 ] Default: 10000 Maximum number of triples to return + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection to query + |
{- "user": "alice",
- "collection": "research",
- "limit": 100
}{- "response": [
- {
}, - {
- "o": {
- "v": "Alice",
- "e": false
}
}, - {
}
]
}Query knowledge graph using GraphQL for object-oriented data access.
+GraphQL interface to knowledge graph:
+Abstracts RDF triples into familiar object model.
+Compared to triples query:
+Standard GraphQL query format:
+query OperationName($var: Type!) {
+ fieldName(arg: $var) {
+ subField1
+ subField2
+ nestedObject {
+ nestedField
+ }
+ }
+}
+
+Pass variables for parameterized queries:
+{
+ "query": "query GetPerson($id: ID!) { person(id: $id) { name } }",
+ "variables": {"id": "https://example.com/person/alice"}
+}
+
+GraphQL distinguishes:
+errors array)error object)Partial data may be returned with field errors.
+Schema defines available types via config service. +Use introspection query to discover schema.
+| flow required | string Example: my-flow Flow instance ID + |
| query required | string GraphQL query string + |
object GraphQL query variables + | |
| operation-name | string Operation name (for multi-operation documents) + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection to query + |
{- "query": "{\n person(id: \"https://example.com/person/alice\") {\n name\n email\n }\n}\n",
- "user": "alice",
- "collection": "research"
}{- "data": {
- "person": {
- "name": "Alice",
- "email": "alice@example.com",
- "knows": [
- {
- "name": "Bob"
}, - {
- "name": "Carol"
}
]
}
}, - "extensions": {
- "execution_time_ms": "42"
}
}Convert natural language questions to structured GraphQL queries.
+Transforms user questions into executable GraphQL:
+Enables non-technical users to query knowledge graph.
+Generated query can be:
+Example workflow:
+1. User asks: "Who does Alice know?"
+2. NLP Query generates GraphQL
+3. Execute via /api/v1/flow/{flow}/service/objects
+4. Return results to user
+
+Response includes detected-schemas array showing:
Helps understand query scope.
+Low scores suggest:
+| flow required | string Example: my-flow Flow instance ID + |
| question required | string Natural language question + |
| max-results | integer [ 1 .. 10000 ] Default: 100 Maximum results to return when query is executed + |
{- "question": "Who does Alice know?",
- "max-results": 50
}{- "graphql-query": "query GetConnections($person: ID!) {\n person(id: $person) {\n knows { name email }\n }\n}\n",
- "detected-schemas": [
- "Person"
], - "confidence": 0.92
}Ask natural language questions and get results directly.
+Combines two operations in one call:
+Simplest way to query knowledge graph with natural language.
+Returns standard GraphQL response:
+Three types of errors:
+error objecterrors arrayerror objectConvenience vs control trade-off:
+| flow required | string Example: my-flow Flow instance ID + |
| question required | string Natural language question + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection to query + |
{- "question": "Who does Alice know?",
- "user": "alice",
- "collection": "research"
}{- "data": {
- "person": {
- "name": "Alice",
- "knows": [
- {
- "name": "Bob",
- "email": "bob@example.com"
}, - {
- "name": "Carol",
- "email": "carol@example.com"
}
]
}
}, - "errors": [ ]
}Analyze and understand structured data (CSV, JSON, XML).
+Helps process unknown structured data:
+Essential for data ingestion pipelines.
+Identify data format from sample:
+Create schema descriptor:
+Combined analysis:
+Find matching schemas:
+Supported formats:
+Format-specific options:
+| flow required | string Example: my-flow Flow instance ID + |
| operation required | string Enum: "detect-type" "generate-descriptor" "diagnose" "schema-selection" Diagnosis operation: +
|
| sample required | string Data sample to analyze (text content) + |
| type | string Enum: "csv" "json" "xml" Data type (required for generate-descriptor) + |
| schema-name | string Target schema name for descriptor generation (optional) + |
object Format-specific options (e.g., CSV delimiter) + |
{- "operation": "detect-type",
- "sample": "name,age,email\nAlice,30,alice@example.com\nBob,25,bob@example.com\n"
}{- "operation": "detect-type",
- "detected-type": "csv",
- "confidence": 0.95
}Query graph embeddings to find similar entities by vector similarity.
+Find entities semantically similar to a query vector:
+Core component of graph RAG retrieval.
+Uses cosine similarity between vectors:
+Returns RDF values (entities):
+{v: "https://...", e: true}| flow required | string Example: my-flow Flow instance ID + |
| vectors required | Array of numbers Query embedding vector + |
| limit | integer [ 1 .. 1000 ] Default: 10 Maximum number of entities to return + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection to search + |
{- "vectors": [
- 0.023,
- -0.142,
- 0.089,
- 0.234,
- -0.067,
- 0.156,
- 0.201,
- -0.178
], - "limit": 10,
- "user": "alice",
- "collection": "research"
}{- "entities": [
]
}Query document embeddings to find similar text chunks by vector similarity.
+Find document chunks semantically similar to a query vector:
+Core component of document RAG retrieval.
+Documents are split into chunks during indexing:
+Queries return individual chunks, not full documents.
+Uses cosine similarity:
+Returns text chunks as strings:
+| flow required | string Example: my-flow Flow instance ID + |
| vectors required | Array of numbers Query embedding vector + |
| limit | integer [ 1 .. 1000 ] Default: 10 Maximum number of document chunks to return + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection to search + |
{- "vectors": [
- 0.023,
- -0.142,
- 0.089,
- 0.234,
- -0.067,
- 0.156,
- 0.201,
- -0.178
], - "limit": 10,
- "user": "alice",
- "collection": "research"
}{- "chunks": [
- "Quantum computing uses quantum mechanics principles like superposition and entanglement for computation. Unlike classical bits, quantum bits (qubits) can exist in multiple states simultaneously.",
- "Neural networks are computing systems inspired by biological neural networks. They consist of interconnected nodes organized in layers that process information through weighted connections.",
- "Machine learning algorithms learn patterns from data without being explicitly programmed. They improve their performance through experience and exposure to training data."
]
}Load text documents into processing pipeline for indexing and embedding.
+Fire-and-forget document loading:
+Asynchronous processing - document queued for background processing.
+Text documents go through:
+Pipeline runs asynchronously after request returns.
+Text must be base64 encoded:
+text_content = "This is the document..."
+encoded = base64.b64encode(text_content.encode('utf-8'))
+
+Default charset is UTF-8, specify charset if different.
Optional RDF triples describing document:
+Returns 202 Accepted immediately:
+| flow required | string Example: my-flow Flow instance ID + |
| text required | string <byte> Text content (base64 encoded) + |
| id | string Document identifier + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection for document + |
| charset | string Default: "utf-8" Text character encoding + |
Array of objects (Triple) Document metadata as RDF triples + |
{- "text": "VGhpcyBpcyB0aGUgZG9jdW1lbnQgdGV4dC4uLg==",
- "id": "doc-123",
- "user": "alice",
- "collection": "research"
}{ }Load binary documents (PDF, Word, etc.) into processing pipeline.
+Fire-and-forget binary document loading:
+Asynchronous processing for PDF and other binary formats.
+Documents go through:
+Pipeline runs asynchronously.
+Format detected from content, not extension.
+Documents must be base64 encoded:
+with open('document.pdf', 'rb') as f:
+ doc_bytes = f.read()
+encoded = base64.b64encode(doc_bytes).decode('utf-8')
+
+Optional RDF triples:
+Returns 202 Accepted immediately:
+| flow required | string Example: my-flow Flow instance ID + |
| data required | string <byte> Document data (base64 encoded) + |
| id | string Document identifier + |
| user | string Default: "trustgraph" User identifier + |
| collection | string Default: "default" Collection for document + |
Array of objects (Triple) Document metadata as RDF triples + |
{- "data": "JVBERi0xLjQKJeLjz9MKMSAwIG9iago8PC9UeXBlL0NhdGFsb2cvUGFnZXMgMiAwIFI+PmVuZG9iagoyIDAgb2JqCjw8L1R5cGUvUGFnZXMvS2lkc1szIDAgUl0vQ291bnQgMT4+ZW5kb2JqCg==",
- "id": "doc-789",
- "user": "alice",
- "collection": "research"
}{ }Import knowledge cores in bulk using streaming MessagePack format.
+Bulk data import for knowledge graph:
+Request body is MessagePack stream with message tuples:
+("t", {
+ "m": { // Metadata
+ "i": "core-id", // Knowledge core ID
+ "m": [...], // Metadata triples array
+ "u": "user", // User
+ "c": "collection" // Collection
+ },
+ "t": [...] // Triples array
+})
+
+("ge", {
+ "m": { // Metadata
+ "i": "core-id",
+ "m": [...],
+ "u": "user",
+ "c": "collection"
+ },
+ "e": [ // Entities array
+ {
+ "e": {"v": "uri", "e": true}, // Entity RdfValue
+ "v": [0.1, 0.2, ...] // Vectors
+ }
+ ]
+})
+
+Multiple messages can be sent in stream. +Each message processed as received. +No response body - returns 202 Accepted.
+| id required | string Example: id=core-123 Knowledge core ID to import + |
| user required | string Example: user=alice User identifier + |
MessagePack stream of knowledge data
+{ }Export knowledge cores in bulk using streaming MessagePack format.
+Bulk data export for knowledge graph:
+Response body is MessagePack stream with message tuples:
+("t", {
+ "m": { // Metadata
+ "i": "core-id", // Knowledge core ID
+ "m": [...], // Metadata triples array
+ "u": "user", // User
+ "c": "collection" // Collection
+ },
+ "t": [...] // Triples array
+})
+
+("ge", {
+ "m": { // Metadata
+ "i": "core-id",
+ "m": [...],
+ "u": "user",
+ "c": "collection"
+ },
+ "e": [ // Entities array
+ {
+ "e": {"v": "uri", "e": true}, // Entity RdfValue
+ "v": [0.1, 0.2, ...] // Vectors
+ }
+ ]
+})
+
+("eos", {})
+
+Data streamed incrementally:
+Client should process messages as received.
+| id required | string Example: id=core-123 Knowledge core ID to export + |
| user required | string Example: user=alice User identifier + |
{- "error": "Unauthorized"
}WebSocket interface providing multiplexed access to all TrustGraph services over a single persistent connection.
+The WebSocket API provides access to the same services as the REST API but with:
+Establish WebSocket connection to:
+ws://localhost:8088/api/v1/socket
+
+All messages are JSON objects with the following structure:
+Global Service Request (no flow parameter):
+{
+ "id": "req-123",
+ "service": "config",
+ "request": {
+ "operation": "list",
+ "type": "flow"
+ }
+}
+
+Flow-Hosted Service Request (with flow parameter):
+{
+ "id": "req-456",
+ "service": "agent",
+ "flow": "my-flow",
+ "request": {
+ "question": "What is quantum computing?",
+ "streaming": true
+ }
+}
+
+Request Fields:
+id (string, required): Client-generated unique identifier for this request within the session. Used to match responses to requests.service (string, required): Service identifier (e.g., "config", "agent", "document-rag"). Same as {kind} in REST URLs.flow (string, optional): Flow ID for flow-hosted services. Omit for global services.request (object, required): Service-specific request payload. Same structure as REST API request body.Success Response:
+{
+ "id": "req-123",
+ "response": {
+ "chunk-type": "answer",
+ "content": "Quantum computing uses...",
+ "end-of-stream": false
+ }
+}
+
+Error Response:
+{
+ "id": "req-123",
+ "error": {
+ "type": "gateway-error",
+ "message": "Flow not found"
+ }
+}
+
+Response Fields:
+id (string, required): Matches the id from the request. Client uses this to correlate responses.response (object, conditional): Service-specific response payload. Same structure as REST API response. Present on success.error (object, conditional): Error information with type and message fields. Present on failure.The WebSocket protocol routes to services using message parameters instead of URL paths:
+| REST Endpoint | +WebSocket Message | +
|---|---|
POST /api/v1/config |
+{"service": "config"} |
+
POST /api/v1/flow/{flow}/service/agent |
+{"service": "agent", "flow": "my-flow"} |
+
Global Services (no flow parameter):
config - Configuration managementflow - Flow lifecycle and blueprintslibrarian - Document library managementknowledge - Knowledge graph core managementcollection-management - Collection metadataFlow-Hosted Services (require flow parameter):
agent, text-completion, prompt, document-rag, graph-ragembeddings, graph-embeddings, document-embeddingstriples, objects, nlp-query, structured-querytext-load, document-loadmcp-tool, structured-diagThe request and response fields use identical schemas to the REST API for each service.
+See individual service documentation for detailed request/response formats.
Multiple requests can be in flight simultaneously:
+id valuesid fieldExample concurrent requests:
+{"id": "req-1", "service": "config", "request": {...}}
+{"id": "req-2", "service": "agent", "flow": "f1", "request": {...}}
+{"id": "req-3", "service": "document-rag", "flow": "f2", "request": {...}}
+
+Responses may arrive in any order: req-2, req-1, req-3
Services that support streaming (e.g., agent, RAG) send multiple response messages with the same id:
{"id": "req-1", "response": {"chunk-type": "thought", "content": "...", "end-of-stream": false}}
+{"id": "req-1", "response": {"chunk-type": "answer", "content": "...", "end-of-stream": false}}
+{"id": "req-1", "response": {"chunk-type": "answer", "content": "...", "end-of-stream": true}}
+
+The end-of-stream flag (or service-specific completion flag) indicates the final message.
When GATEWAY_SECRET is set, include bearer token:
ws://localhost:8088/api/v1/socket?token=<token>| Upgrade required | string Value: "websocket" WebSocket upgrade header + |
| Connection required | string Value: "Upgrade" Connection upgrade header + |
{- "error": "Unauthorized"
}Proxy to Prometheus metrics for system monitoring.
+Exposes system metrics via Prometheus format:
+Returns metrics in Prometheus text exposition format:
+# HELP metric_name Description
+# TYPE metric_name counter
+metric_name{label="value"} 123.45
+
+Common metrics include:
+Standard Prometheus scraping:
+/api/metricsThe {path} parameter allows querying specific Prometheus endpoints
+or metrics if the backend Prometheus supports it.
{- "error": "Unauthorized"
}WebSocket API for TrustGraph - providing multiplexed, asynchronous access to all services.
+The WebSocket API provides access to all TrustGraph services over a single persistent connection:
+All messages are JSON with:
+id: Client-generated unique identifier for request/response correlationservice: Service identifier (e.g., "config", "agent", "document-rag")flow: Optional flow ID for flow-hosted servicesrequest/response: Service-specific payload (identical to REST API schemas)error: Error information on failureGlobal Services (no flow parameter):
Flow-Hosted Services (require flow parameter):
Request and response payloads use identical schemas to the REST API. +See OpenAPI specification for detailed schema documentation.
+Local development WebSocket server
+Bearer token authentication when GATEWAY_SECRET is configured. +Include as query parameter: ws://localhost:8088/api/v1/socket?token=
+Request message for any TrustGraph service
Generic request message that can invoke any TrustGraph service.
+The request field payload varies by service and matches the REST API request body schema.
Service request envelope with id, service, optional flow, and service-specific request payload
+Successful response from any TrustGraph service
Generic response message from any TrustGraph service.
+The response field payload varies by service and matches the REST API response body schema.
For streaming services, multiple messages with the same id may be sent.
WebSocket response message envelope for successful responses.
+Contains the request ID for correlation and the service-specific response payload.
+Error response from any TrustGraph service
Error message sent when a service request fails.
+Contains the request ID and error details.
+WebSocket error message envelope.
+Sent when a request fails. Contains the request ID and error details.
+WebSocket request message envelope.
+Wraps service-specific request payloads with routing and correlation metadata.
+WebSocket response message envelope for successful responses.
+Contains the request ID for correlation and the service-specific response payload.
+WebSocket error message envelope.
+Sent when a request fails. Contains the request ID and error details.
+