Some CLI docs

2026-07-22 03:31:02 +02:00 · 2025-07-03 10:33:27 +01:00 · 2025-07-03 10:33:27 +01:00 · 1e5352e8b7
commit 1e5352e8b7
parent c9e5c3f187
4 changed files with 710 additions and 0 deletions
--- a/docs/cli/README.md
+++ b/docs/cli/README.md
@ -0,0 +1,166 @@
+# TrustGraph CLI Documentation
+
+The TrustGraph Command Line Interface (CLI) provides comprehensive command-line access to all TrustGraph services. These tools wrap the REST and WebSocket APIs to provide convenient, scriptable access to TrustGraph functionality.
+
+## Installation
+
+The CLI tools are installed as part of the `trustgraph-cli` package:
+
+```bash
+pip install trustgraph-cli
+```
+
+## Global Options
+
+Most CLI commands support these common options:
+
+- `-u, --api-url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `http://localhost:8088/`)
+- `-U, --user USER`: User identifier (default: `trustgraph`)
+- `-C, --collection COLLECTION`: Collection identifier (default: `default`)
+- `-f, --flow-id FLOW`: Flow identifier (default: `default`)
+
+## Command Categories
+
+### System Administration & Configuration
+
+**System Setup:**
+- [`tg-init-trustgraph`](tg-init-trustgraph.md) - Initialize Pulsar with TrustGraph configuration
+- [`tg-init-pulsar-manager`](tg-init-pulsar-manager.md) - Initialize Pulsar manager setup
+- [`tg-show-config`](tg-show-config.md) - Display current system configuration
+
+**Token Management:**
+- [`tg-set-token-costs`](tg-set-token-costs.md) - Configure model token costs
+- [`tg-show-token-costs`](tg-show-token-costs.md) - Display token cost configuration
+- [`tg-show-token-rate`](tg-show-token-rate.md) - Show token usage rates
+
+### Flow Management
+
+**Flow Operations:**
+- [`tg-start-flow`](tg-start-flow.md) - Start a processing flow
+- [`tg-stop-flow`](tg-stop-flow.md) - Stop a running flow
+- [`tg-show-flows`](tg-show-flows.md) - List all configured flows
+- [`tg-show-flow-state`](tg-show-flow-state.md) - Show current flow states
+
+**Flow Class Management:**
+- [`tg-put-flow-class`](tg-put-flow-class.md) - Upload/update flow class definition
+- [`tg-get-flow-class`](tg-get-flow-class.md) - Retrieve flow class definition
+- [`tg-delete-flow-class`](tg-delete-flow-class.md) - Remove flow class definition
+- [`tg-show-flow-classes`](tg-show-flow-classes.md) - List available flow classes
+
+### Knowledge Graph Management
+
+**Knowledge Core Operations:**
+- [`tg-load-kg-core`](tg-load-kg-core.md) - Load knowledge core into processing
+- [`tg-put-kg-core`](tg-put-kg-core.md) - Store knowledge core in system
+- [`tg-get-kg-core`](tg-get-kg-core.md) - Retrieve knowledge core
+- [`tg-delete-kg-core`](tg-delete-kg-core.md) - Remove knowledge core
+- [`tg-unload-kg-core`](tg-unload-kg-core.md) - Unload knowledge core from processing
+- [`tg-show-kg-cores`](tg-show-kg-cores.md) - List available knowledge cores
+
+**Graph Data Operations:**
+- [`tg-show-graph`](tg-show-graph.md) - Display graph triples/edges
+- [`tg-graph-to-turtle`](tg-graph-to-turtle.md) - Export graph to Turtle format
+- [`tg-load-turtle`](tg-load-turtle.md) - Import RDF triples from Turtle files
+
+### Document Processing & Library Management
+
+**Document Loading:**
+- [`tg-load-pdf`](tg-load-pdf.md) - Load PDF documents into processing
+- [`tg-load-text`](tg-load-text.md) - Load text documents into processing
+- [`tg-load-sample-documents`](tg-load-sample-documents.md) - Load sample documents for testing
+
+**Library Management:**
+- [`tg-add-library-document`](tg-add-library-document.md) - Add documents to library
+- [`tg-remove-library-document`](tg-remove-library-document.md) - Remove documents from library
+- [`tg-show-library-documents`](tg-show-library-documents.md) - List library documents
+- [`tg-start-library-processing`](tg-start-library-processing.md) - Start library document processing
+- [`tg-stop-library-processing`](tg-stop-library-processing.md) - Stop library document processing
+- [`tg-show-library-processing`](tg-show-library-processing.md) - Show library processing status
+
+**Document Embeddings:**
+- [`tg-load-doc-embeds`](tg-load-doc-embeds.md) - Load document embeddings
+- [`tg-save-doc-embeds`](tg-save-doc-embeds.md) - Save document embeddings
+
+### AI Services & Agent Interaction
+
+**Query & Interaction:**
+- [`tg-invoke-agent`](tg-invoke-agent.md) - Interactive agent Q&A via WebSocket
+- [`tg-invoke-llm`](tg-invoke-llm.md) - Direct LLM text completion
+- [`tg-invoke-prompt`](tg-invoke-prompt.md) - Use configured prompt templates
+- [`tg-invoke-document-rag`](tg-invoke-document-rag.md) - Document-based RAG queries
+- [`tg-invoke-graph-rag`](tg-invoke-graph-rag.md) - Graph-based RAG queries
+
+**Tool & Prompt Management:**
+- [`tg-show-tools`](tg-show-tools.md) - List available agent tools
+- [`tg-set-prompt`](tg-set-prompt.md) - Configure prompt templates
+- [`tg-show-prompts`](tg-show-prompts.md) - List configured prompts
+
+### System Monitoring & Debugging
+
+**System Status:**
+- [`tg-show-processor-state`](tg-show-processor-state.md) - Show processing component states
+
+**Debugging:**
+- [`tg-dump-msgpack`](tg-dump-msgpack.md) - Dump MessagePack data for debugging
+
+## Quick Start Examples
+
+### Basic Document Processing
+```bash
+# Start a flow
+tg-start-flow --flow-id my-flow --class-name document-processing
+
+# Load a document
+tg-load-text --flow-id my-flow --text "Your document content" --title "Test Document"
+
+# Query the knowledge
+tg-invoke-graph-rag --flow-id my-flow --query "What is the document about?"
+```
+
+### Knowledge Management
+```bash
+# List available knowledge cores
+tg-show-kg-cores
+
+# Load a knowledge core into a flow
+tg-load-kg-core --flow-id my-flow --kg-core-id my-knowledge
+
+# Query the knowledge graph
+tg-show-graph --limit 100
+```
+
+### Flow Management
+```bash
+# Show available flow classes
+tg-show-flow-classes
+
+# Show running flows
+tg-show-flows
+
+# Stop a flow
+tg-stop-flow --flow-id my-flow
+```
+
+## Environment Variables
+
+- `TRUSTGRAPH_URL`: Default API URL for all commands
+- `TRUSTGRAPH_USER`: Default user identifier
+- `TRUSTGRAPH_COLLECTION`: Default collection identifier
+
+## Authentication
+
+CLI commands inherit authentication from the environment or API configuration. See the main TrustGraph documentation for authentication setup.
+
+## Error Handling
+
+All CLI commands provide:
+- Consistent error reporting
+- Exit codes (0 for success, non-zero for errors)
+- Detailed error messages for troubleshooting
+- Retry logic for network operations where appropriate
+
+## Related Documentation
+
+- [TrustGraph API Documentation](../apis/README.md)
+- [TrustGraph WebSocket Guide](../apis/websocket.md)
+- [TrustGraph Pulsar Guide](../apis/pulsar.md)
--- a/docs/cli/tg-invoke-agent.md
+++ b/docs/cli/tg-invoke-agent.md
@ -0,0 +1,163 @@
+# tg-invoke-agent
+
+Uses the agent service to answer a question via interactive WebSocket connection.
+
+## Synopsis
+
+```bash
+tg-invoke-agent -q "your question" [options]
+```
+
+## Description
+
+The `tg-invoke-agent` command provides an interactive interface to TrustGraph's agent service. It connects via WebSocket to submit questions and receive real-time responses, including the agent's thinking process and observations when verbose mode is enabled.
+
+The agent uses available tools and knowledge sources to answer questions, providing a conversational AI interface to your TrustGraph knowledge base.
+
+## Options
+
+### Required Arguments
+
+- `-q, --question QUESTION`: The question to ask the agent
+
+### Optional Arguments
+
+- `-u, --url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `ws://localhost:8088/`)
+- `-f, --flow-id FLOW`: Flow ID to use (default: `default`)
+- `-U, --user USER`: User identifier (default: `trustgraph`)
+- `-C, --collection COLLECTION`: Collection identifier (default: `default`)
+- `-l, --plan PLAN`: Agent plan specification (optional)
+- `-s, --state STATE`: Agent initial state (optional)
+- `-v, --verbose`: Output agent's thinking process and observations
+
+## Examples
+
+### Basic Question
+```bash
+tg-invoke-agent -q "What is machine learning?"
+```
+
+### Verbose Output with Thinking Process
+```bash
+tg-invoke-agent -q "Explain the benefits of neural networks" -v
+```
+
+### Using Specific Flow
+```bash
+tg-invoke-agent -q "What documents are available?" -f research-flow
+```
+
+### With Custom User and Collection
+```bash
+tg-invoke-agent -q "Show me recent papers" -U alice -C research-papers
+```
+
+### Using Custom API URL
+```bash
+tg-invoke-agent -q "What is AI?" -u ws://production:8088/
+```
+
+## Output Format
+
+### Standard Output
+The agent provides direct answers to your questions:
+
+```
+AI stands for Artificial Intelligence, which refers to computer systems that can perform tasks typically requiring human intelligence.
+```
+
+### Verbose Output
+With `-v` flag, you see the agent's thinking process:
+
+```
+❓ What is machine learning?
+
+🤔 I need to provide a comprehensive explanation of machine learning, including its definition, key concepts, and applications.
+
+💡 Let me search for information about machine learning in the knowledge base.
+
+Machine learning is a subset of artificial intelligence that enables computers to learn and improve automatically from experience without being explicitly programmed...
+```
+
+The emoji indicators represent:
+- ❓ Your question
+- 🤔 Agent's thinking/reasoning
+- 💡 Agent's observations from tools/searches
+
+## Error Handling
+
+Common errors and solutions:
+
+### Connection Errors
+```bash
+Exception: Connection refused
+```
+**Solution**: Verify the API URL and ensure TrustGraph is running.
+
+### Flow Not Found
+```bash
+Exception: Invalid flow
+```
+**Solution**: Check that the specified flow exists and is running using `tg-show-flows`.
+
+### Authentication Errors
+```bash
+Exception: Unauthorized
+```
+**Solution**: Verify your authentication credentials and permissions.
+
+## Environment Variables
+
+- `TRUSTGRAPH_URL`: Default API URL (converted to WebSocket URL automatically)
+
+## Related Commands
+
+- [`tg-invoke-graph-rag`](tg-invoke-graph-rag.md) - Graph-based retrieval augmented generation
+- [`tg-invoke-document-rag`](tg-invoke-document-rag.md) - Document-based retrieval augmented generation
+- [`tg-invoke-llm`](tg-invoke-llm.md) - Direct LLM text completion
+- [`tg-show-tools`](tg-show-tools.md) - List available agent tools
+- [`tg-show-flows`](tg-show-flows.md) - List available flows
+
+## Technical Details
+
+### WebSocket Communication
+The command uses WebSocket protocol for real-time communication with the agent service. The URL is automatically converted from HTTP to WebSocket format.
+
+### Message Format
+Messages are exchanged in JSON format:
+
+**Request:**
+```json
+{
+    "id": "unique-message-id",
+    "service": "agent",
+    "flow": "flow-id",
+    "request": {
+        "question": "your question"
+    }
+}
+```
+
+**Response:**
+```json
+{
+    "id": "unique-message-id",
+    "response": {
+        "thought": "agent thinking",
+        "observation": "agent observation",
+        "answer": "final answer"
+    },
+    "complete": true
+}
+```
+
+### API Integration
+This command uses the [Agent API](../apis/api-agent.md) via WebSocket connection for real-time interaction.
+
+## Use Cases
+
+- **Interactive Q&A**: Ask questions about your knowledge base
+- **Research Assistance**: Get help analyzing documents and data
+- **Knowledge Discovery**: Explore connections in your data
+- **Troubleshooting**: Get help with technical issues using verbose mode
+- **Educational**: Learn about topics in your knowledge base
--- a/docs/cli/tg-load-text.md
+++ b/docs/cli/tg-load-text.md
@ -0,0 +1,211 @@
+# tg-load-text
+
+Loads text documents into TrustGraph processing pipelines with rich metadata support.
+
+## Synopsis
+
+```bash
+tg-load-text [options] file1 [file2 ...]
+```
+
+## Description
+
+The `tg-load-text` command loads text documents into TrustGraph for processing. It creates a SHA256 hash-based document ID and supports comprehensive metadata including copyright information, publication details, and keywords.
+
+**Note**: Consider using `tg-add-library-document` followed by `tg-start-library-processing` for better document management and processing control.
+
+## Options
+
+### Connection & Flow
+- `-u, --url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `http://localhost:8088/`)
+- `-f, --flow-id FLOW`: Flow ID for processing (default: `default`)
+- `-U, --user USER`: User identifier (default: `trustgraph`)
+- `-C, --collection COLLECTION`: Collection identifier (default: `default`)
+
+### Document Metadata
+- `--name NAME`: Document name/title
+- `--description DESCRIPTION`: Document description
+- `--document-url URL`: Document source URL
+
+### Copyright Information
+- `--copyright-notice NOTICE`: Copyright notice text
+- `--copyright-holder HOLDER`: Copyright holder name
+- `--copyright-year YEAR`: Copyright year
+- `--license LICENSE`: Copyright license
+
+### Publication Information
+- `--publication-organization ORG`: Publishing organization
+- `--publication-description DESC`: Publication description
+- `--publication-date DATE`: Publication date
+
+### Keywords
+- `--keyword KEYWORD [KEYWORD ...]`: Document keywords (can specify multiple)
+
+## Arguments
+
+- `file1 [file2 ...]`: One or more text files to load
+
+## Examples
+
+### Basic Document Loading
+```bash
+tg-load-text document.txt
+```
+
+### Loading with Metadata
+```bash
+tg-load-text \
+  --name "Research Paper on AI" \
+  --description "Comprehensive study of machine learning algorithms" \
+  --keyword "AI" "machine learning" "research" \
+  research-paper.txt
+```
+
+### Complete Metadata Example
+```bash
+tg-load-text \
+  --name "TrustGraph Documentation" \
+  --description "Complete user guide for TrustGraph system" \
+  --copyright-holder "TrustGraph Project" \
+  --copyright-year "2024" \
+  --license "MIT" \
+  --publication-organization "TrustGraph Foundation" \
+  --publication-date "2024-01-15" \
+  --keyword "documentation" "guide" "tutorial" \
+  --flow-id research-flow \
+  trustgraph-guide.txt
+```
+
+### Multiple Files
+```bash
+tg-load-text chapter1.txt chapter2.txt chapter3.txt
+```
+
+### Custom Flow and Collection
+```bash
+tg-load-text \
+  --flow-id medical-research \
+  --user researcher \
+  --collection medical-papers \
+  medical-study.txt
+```
+
+## Output
+
+For each file processed, the command outputs:
+
+### Success
+```
+document.txt: Loaded successfully.
+```
+
+### Failure
+```
+document.txt: Failed: Connection refused
+```
+
+## Document Processing
+
+1. **File Reading**: Reads the text file content
+2. **Hash Generation**: Creates SHA256 hash for unique document ID
+3. **URI Creation**: Converts hash to document URI format
+4. **Metadata Assembly**: Combines all metadata into RDF triples
+5. **API Submission**: Sends to TrustGraph via Text Load API
+
+## Document ID Generation
+
+Documents are assigned IDs based on their content hash:
+- SHA256 hash of file content
+- Converted to TrustGraph document URI format
+- Example: `http://trustgraph.ai/d/abc123...`
+
+## Metadata Format
+
+The metadata is stored as RDF triples including:
+
+### Standard Properties
+- `dc:title`: Document name
+- `dc:description`: Document description  
+- `dc:creator`: Copyright holder
+- `dc:date`: Publication date
+- `dc:rights`: Copyright notice
+- `dc:license`: License information
+
+### Keywords
+- `dc:subject`: Each keyword as separate triple
+
+### Organization Information
+- `foaf:Organization`: Publication organization details
+
+## Error Handling
+
+### File Errors
+```bash
+document.txt: Failed: No such file or directory
+```
+**Solution**: Verify the file path exists and is readable.
+
+### Connection Errors
+```bash
+document.txt: Failed: Connection refused
+```
+**Solution**: Check the API URL and ensure TrustGraph is running.
+
+### Flow Errors
+```bash
+document.txt: Failed: Invalid flow
+```
+**Solution**: Verify the flow exists and is running using `tg-show-flows`.
+
+## Environment Variables
+
+- `TRUSTGRAPH_URL`: Default API URL
+
+## Related Commands
+
+- [`tg-add-library-document`](tg-add-library-document.md) - Add documents to library (recommended)
+- [`tg-load-pdf`](tg-load-pdf.md) - Load PDF documents
+- [`tg-show-library-documents`](tg-show-library-documents.md) - List loaded documents
+- [`tg-start-library-processing`](tg-start-library-processing.md) - Start document processing
+
+## API Integration
+
+This command uses the [Text Load API](../apis/api-text-load.md) to submit documents for processing. The text content is base64-encoded for transmission.
+
+## Use Cases
+
+### Academic Research
+```bash
+tg-load-text \
+  --name "Climate Change Impact Study" \
+  --publication-organization "University Research Center" \
+  --keyword "climate" "research" "environment" \
+  climate-study.txt
+```
+
+### Corporate Documentation
+```bash
+tg-load-text \
+  --name "Product Manual" \
+  --copyright-holder "Acme Corp" \
+  --license "Proprietary" \
+  --keyword "manual" "product" "guide" \
+  product-manual.txt
+```
+
+### Technical Documentation
+```bash
+tg-load-text \
+  --name "API Reference" \
+  --description "Complete API documentation" \
+  --keyword "API" "reference" "technical" \
+  api-docs.txt
+```
+
+## Best Practices
+
+1. **Use Descriptive Names**: Provide clear document names and descriptions
+2. **Add Keywords**: Include relevant keywords for better searchability
+3. **Complete Metadata**: Fill in copyright and publication information
+4. **Batch Processing**: Load multiple related files together
+5. **Use Collections**: Organize documents by topic or project using collections
--- a/docs/cli/tg-show-config.md
+++ b/docs/cli/tg-show-config.md
@ -0,0 +1,170 @@
+# tg-show-config
+
+Displays the current TrustGraph system configuration.
+
+## Synopsis
+
+```bash
+tg-show-config [options]
+```
+
+## Description
+
+The `tg-show-config` command retrieves and displays the complete TrustGraph system configuration in JSON format. This includes flow definitions, service configurations, and other system settings stored in the configuration service.
+
+This is particularly useful for:
+- Understanding the current system setup
+- Debugging configuration issues
+- Finding queue names for Pulsar integration
+- Verifying flow definitions and interfaces
+
+## Options
+
+- `-u, --api-url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `http://localhost:8088/`)
+
+## Examples
+
+### Display Complete Configuration
+```bash
+tg-show-config
+```
+
+### Using Custom API URL
+```bash
+tg-show-config -u http://production:8088/
+```
+
+## Output Format
+
+The command outputs the configuration version followed by the complete configuration in JSON format:
+
+```
+Version: 42
+{
+    "flows": {
+        "default": {
+            "class-name": "document-rag+graph-rag",
+            "description": "Default processing flow",
+            "interfaces": {
+                "agent": {
+                    "request": "non-persistent://tg/request/agent:default",
+                    "response": "non-persistent://tg/response/agent:default"
+                },
+                "graph-rag": {
+                    "request": "non-persistent://tg/request/graph-rag:document-rag+graph-rag",
+                    "response": "non-persistent://tg/response/graph-rag:document-rag+graph-rag"
+                },
+                "text-load": "persistent://tg/flow/text-document-load:default",
+                ...
+            }
+        }
+    },
+    "prompts": {
+        "system": "You are a helpful AI assistant...",
+        "graph-rag": "Answer the question using the provided context..."
+    },
+    "token-costs": {
+        "gpt-4": {
+            "prompt": 0.03,
+            "completion": 0.06
+        }
+    },
+    ...
+}
+```
+
+## Configuration Sections
+
+### Flow Definitions
+Flow configurations showing:
+- **class-name**: The flow class being used
+- **description**: Human-readable flow description  
+- **interfaces**: Pulsar queue names for each service
+
+### Prompt Templates
+System and service-specific prompt templates used by AI services.
+
+### Token Costs
+Model pricing information for cost tracking and billing.
+
+### Service Settings
+Various service-specific configuration parameters.
+
+## Finding Queue Names
+
+The configuration output is essential for discovering Pulsar queue names:
+
+### Flow-Hosted Services
+Look in the `flows` section under `interfaces`:
+
+```json
+"graph-rag": {
+    "request": "non-persistent://tg/request/graph-rag:document-rag+graph-rag",
+    "response": "non-persistent://tg/response/graph-rag:document-rag+graph-rag"
+}
+```
+
+### Fire-and-Forget Services
+Some services only have input queues:
+
+```json
+"text-load": "persistent://tg/flow/text-document-load:default"
+```
+
+## Error Handling
+
+### Connection Errors
+```bash
+Exception: Connection refused
+```
+**Solution**: Verify the API URL and ensure TrustGraph is running.
+
+### Authentication Errors
+```bash
+Exception: Unauthorized
+```
+**Solution**: Check authentication credentials and permissions.
+
+## Environment Variables
+
+- `TRUSTGRAPH_URL`: Default API URL
+
+## Related Commands
+
+- [`tg-put-flow-class`](tg-put-flow-class.md) - Update flow class definitions
+- [`tg-show-flows`](tg-show-flows.md) - List active flows
+- [`tg-set-prompt`](tg-set-prompt.md) - Configure prompt templates
+- [`tg-set-token-costs`](tg-set-token-costs.md) - Configure token costs
+
+## API Integration
+
+This command uses the [Config API](../apis/api-config.md) with the `config` operation to retrieve the complete system configuration.
+
+**API Call:**
+```json
+{
+    "operation": "config"
+}
+```
+
+## Use Cases
+
+### Development and Debugging
+- Verify flow configurations are correct
+- Check that services have proper queue assignments
+- Debug configuration-related issues
+
+### System Administration
+- Monitor configuration changes over time
+- Document current system setup
+- Prepare for system migrations
+
+### Integration Development
+- Discover Pulsar queue names for direct integration
+- Understand service interfaces and capabilities
+- Verify API endpoint configurations
+
+### Troubleshooting
+- Check if flows are properly configured
+- Verify prompt templates are set correctly
+- Confirm token cost configurations