diff --git a/docs/cli/tg-load-kg-core.md b/docs/cli/tg-load-kg-core.md new file mode 100644 index 00000000..d83c8dd6 --- /dev/null +++ b/docs/cli/tg-load-kg-core.md @@ -0,0 +1,313 @@ +# tg-load-kg-core + +Loads a stored knowledge core into a processing flow for active use. + +## Synopsis + +```bash +tg-load-kg-core --id CORE_ID [options] +``` + +## Description + +The `tg-load-kg-core` command loads a previously stored knowledge core into an active processing flow, making the knowledge available for queries, reasoning, and other AI operations. This is different from storing knowledge cores - this command makes stored knowledge active and accessible within a specific flow context. + +Once loaded, the knowledge core's RDF triples and graph embeddings become available for Graph RAG queries, agent reasoning, and other knowledge-based operations within the specified flow. + +## Options + +### Required Arguments + +- `--id, --identifier CORE_ID`: Identifier of the knowledge core to load + +### Optional Arguments + +- `-u, --api-url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `http://localhost:8088/`) +- `-U, --user USER`: User identifier (default: `trustgraph`) +- `-f, --flow-id FLOW`: Flow ID to load knowledge into (default: `default`) +- `-c, --collection COLLECTION`: Collection identifier (default: `default`) + +## Examples + +### Load Knowledge Core into Default Flow +```bash +tg-load-kg-core --id "research-knowledge-v1" +``` + +### Load into Specific Flow +```bash +tg-load-kg-core \ + --id "medical-knowledge" \ + --flow-id "medical-analysis" \ + --user researcher +``` + +### Load with Custom Collection +```bash +tg-load-kg-core \ + --id "legal-documents" \ + --flow-id "legal-flow" \ + --collection "law-firm-data" +``` + +### Using Custom API URL +```bash +tg-load-kg-core \ + --id "production-knowledge" \ + --flow-id "prod-flow" \ + -u http://production:8088/ +``` + +## Prerequisites + +### Knowledge Core Must Exist +The knowledge core must be stored in the system: + +```bash +# Check available knowledge cores +tg-show-kg-cores + +# Store knowledge core if needed +tg-put-kg-core --id "my-knowledge" -i knowledge.msgpack +``` + +### Flow Must Be Running +The target flow must be active: + +```bash +# Check running flows +tg-show-flows + +# Start flow if needed +tg-start-flow -n "my-class" -i "my-flow" -d "Knowledge processing flow" +``` + +## Loading Process + +1. **Validation**: Verifies knowledge core exists and flow is running +2. **Knowledge Retrieval**: Retrieves RDF triples and graph embeddings +3. **Flow Integration**: Makes knowledge available within flow context +4. **Index Building**: Creates searchable indexes for efficient querying +5. **Service Activation**: Enables knowledge-based services in the flow + +## What Gets Loaded + +### RDF Triples +- Subject-predicate-object relationships +- Entity definitions and properties +- Factual knowledge and assertions +- Metadata and provenance information + +### Graph Embeddings +- Vector representations of entities +- Semantic similarity data +- Neural network-compatible formats +- Machine learning-ready representations + +## Knowledge Availability + +Once loaded, knowledge becomes available through: + +### Graph RAG Queries +```bash +tg-invoke-graph-rag \ + -q "What information is available about AI research?" \ + -f my-flow +``` + +### Agent Interactions +```bash +tg-invoke-agent \ + -q "Tell me about the loaded knowledge" \ + -f my-flow +``` + +### Direct Triple Queries +```bash +tg-show-graph -f my-flow +``` + +## Output + +Successful loading typically produces no output, but knowledge becomes queryable: + +```bash +# Load knowledge (no output expected) +tg-load-kg-core --id "research-knowledge" + +# Verify loading by querying +tg-show-graph | head -10 +``` + +## Error Handling + +### Knowledge Core Not Found +```bash +Exception: Knowledge core 'invalid-core' not found +``` +**Solution**: Check available cores with `tg-show-kg-cores` and verify the core ID. + +### Flow Not Found +```bash +Exception: Flow 'invalid-flow' not found +``` +**Solution**: Verify the flow exists and is running with `tg-show-flows`. + +### Permission Errors +```bash +Exception: Access denied to knowledge core +``` +**Solution**: Verify user permissions for the specified knowledge core. + +### Connection Errors +```bash +Exception: Connection refused +``` +**Solution**: Check the API URL and ensure TrustGraph is running. + +### Resource Errors +```bash +Exception: Insufficient memory to load knowledge core +``` +**Solution**: Check system resources or try loading smaller knowledge cores. + +## Knowledge Core Management + +### Loading Workflow +```bash +# 1. Check available knowledge +tg-show-kg-cores + +# 2. Ensure flow is running +tg-show-flows + +# 3. Load knowledge into flow +tg-load-kg-core --id "my-knowledge" --flow-id "my-flow" + +# 4. Verify knowledge is accessible +tg-invoke-graph-rag -q "What knowledge is loaded?" -f my-flow +``` + +### Multiple Knowledge Cores +```bash +# Load multiple cores for comprehensive knowledge +tg-load-kg-core --id "core-1" --flow-id "research-flow" +tg-load-kg-core --id "core-2" --flow-id "research-flow" +tg-load-kg-core --id "core-3" --flow-id "research-flow" +``` + +## Environment Variables + +- `TRUSTGRAPH_URL`: Default API URL + +## Related Commands + +- [`tg-show-kg-cores`](tg-show-kg-cores.md) - List available knowledge cores +- [`tg-put-kg-core`](tg-put-kg-core.md) - Store knowledge core in system +- [`tg-unload-kg-core`](tg-unload-kg-core.md) - Remove knowledge from flow +- [`tg-show-graph`](tg-show-graph.md) - View loaded knowledge triples +- [`tg-invoke-graph-rag`](tg-invoke-graph-rag.md) - Query loaded knowledge + +## API Integration + +This command uses the [Knowledge API](../apis/api-knowledge.md) with the `load-kg-core` operation to make stored knowledge active within flows. + +## Use Cases + +### Research Analysis +```bash +# Load research knowledge for analysis +tg-load-kg-core \ + --id "research-papers-2024" \ + --flow-id "research-analysis" \ + --collection "academic-research" + +# Query the research knowledge +tg-invoke-graph-rag \ + -q "What are the main research trends in AI?" \ + -f research-analysis +``` + +### Domain-Specific Processing +```bash +# Load medical knowledge for healthcare analysis +tg-load-kg-core \ + --id "medical-terminology" \ + --flow-id "healthcare-nlp" \ + --user medical-team +``` + +### Multi-Domain Knowledge +```bash +# Load knowledge from multiple domains +tg-load-kg-core --id "technical-specs" --flow-id "analysis-flow" +tg-load-kg-core --id "business-data" --flow-id "analysis-flow" +tg-load-kg-core --id "market-research" --flow-id "analysis-flow" +``` + +### Development and Testing +```bash +# Load test knowledge for development +tg-load-kg-core \ + --id "test-knowledge" \ + --flow-id "dev-flow" \ + --user developer +``` + +### Production Processing +```bash +# Load production knowledge +tg-load-kg-core \ + --id "production-kb-v2.1" \ + --flow-id "production-flow" \ + --collection "live-data" +``` + +## Performance Considerations + +### Loading Time +- Large knowledge cores may take time to load +- Loading includes indexing for efficient querying +- Multiple cores can be loaded incrementally + +### Memory Usage +- Knowledge cores consume memory proportional to their size +- Monitor system resources when loading large cores +- Consider flow capacity when loading multiple cores + +### Query Performance +- Loaded knowledge enables faster query responses +- Pre-built indexes improve search performance +- Multiple cores may impact query speed + +## Best Practices + +1. **Pre-Loading**: Load knowledge cores before intensive querying +2. **Resource Planning**: Monitor memory usage with large knowledge cores +3. **Flow Management**: Use dedicated flows for specific knowledge domains +4. **Version Control**: Load specific knowledge core versions for reproducibility +5. **Testing**: Verify knowledge loading with simple queries +6. **Documentation**: Document which knowledge cores are loaded in which flows + +## Knowledge Loading Strategy + +### Single Domain +```bash +# Load focused knowledge for specific tasks +tg-load-kg-core --id "specialized-domain" --flow-id "domain-flow" +``` + +### Multi-Domain +```bash +# Load comprehensive knowledge for broad analysis +tg-load-kg-core --id "general-knowledge" --flow-id "general-flow" +tg-load-kg-core --id "domain-specific" --flow-id "general-flow" +``` + +### Incremental Loading +```bash +# Load knowledge incrementally as needed +tg-load-kg-core --id "base-knowledge" --flow-id "analysis-flow" +# ... perform some analysis ... +tg-load-kg-core --id "additional-knowledge" --flow-id "analysis-flow" +``` \ No newline at end of file diff --git a/docs/cli/tg-show-graph.md b/docs/cli/tg-show-graph.md new file mode 100644 index 00000000..1da66dd5 --- /dev/null +++ b/docs/cli/tg-show-graph.md @@ -0,0 +1,286 @@ +# tg-show-graph + +Displays knowledge graph triples (edges) from the TrustGraph system. + +## Synopsis + +```bash +tg-show-graph [options] +``` + +## Description + +The `tg-show-graph` command queries the knowledge graph and displays up to 10,000 triples (subject-predicate-object relationships) in a human-readable format. This is useful for exploring knowledge graph contents, debugging knowledge loading, and understanding the structure of stored knowledge. + +Each triple represents a fact or relationship in the knowledge graph, showing how entities are connected through various predicates. + +## Options + +- `-u, --api-url URL`: TrustGraph API URL (default: `$TRUSTGRAPH_URL` or `http://localhost:8088/`) +- `-f, --flow-id FLOW`: Flow ID to query (default: `default`) +- `-U, --user USER`: User identifier (default: `trustgraph`) +- `-C, --collection COLLECTION`: Collection identifier (default: `default`) + +## Examples + +### Display All Graph Triples +```bash +tg-show-graph +``` + +### Query Specific Flow +```bash +tg-show-graph -f research-flow +``` + +### Query User's Collection +```bash +tg-show-graph -U researcher -C medical-papers +``` + +### Using Custom API URL +```bash +tg-show-graph -u http://production:8088/ +``` + +## Output Format + +The command displays triples in subject-predicate-object format: + +``` + "John Doe" + + "Acme Corporation" + + "New York" + + "Research Report" + "2024" +``` + +### Triple Components + +- **Subject**: The entity the statement is about (usually a URI) +- **Predicate**: The relationship or property (usually a URI) +- **Object**: The value or target entity (can be URI or literal) + +### URI vs Literal Values + +- **URIs**: Enclosed in angle brackets `` +- **Literals**: Enclosed in quotes `"Literal Value"` + +### Common Predicates + +- ``: Entity names +- ``: Document titles +- ``: Authorship relationships +- ``: Employment relationships +- ``: Location relationships +- ``: Publication information +- ``: Dublin Core creator +- ``: Friend of a Friend name + +## Data Limitations + +### 10,000 Triple Limit +The command displays up to 10,000 triples to prevent overwhelming output. For larger graphs: + +```bash +# Use graph export for complete data +tg-graph-to-turtle -o complete-graph.ttl + +# Use targeted queries for specific data +tg-invoke-graph-rag -q "Show me information about specific entities" +``` + +### Collection Scope +Results are limited to the specified user and collection. To see all data: + +```bash +# Query different collections +tg-show-graph -C collection1 +tg-show-graph -C collection2 +``` + +## Knowledge Graph Structure + +### Entity Types +Common entity types in the output: +- **Documents**: Research papers, reports, manuals +- **People**: Authors, researchers, employees +- **Organizations**: Companies, institutions, publishers +- **Concepts**: Technical terms, topics, categories +- **Events**: Publications, meetings, processes + +### Relationship Types +Common relationship types: +- **Authorship**: Who created what +- **Membership**: Who belongs to what organization +- **Hierarchical**: Parent-child relationships +- **Temporal**: When things happened +- **Topical**: What topics are related + +## Error Handling + +### Flow Not Available +```bash +Exception: Invalid flow +``` +**Solution**: Verify the flow exists and is running with `tg-show-flows`. + +### No Data Available +```bash +# Empty output (no triples displayed) +``` +**Solution**: Check if knowledge has been loaded using `tg-show-kg-cores` and `tg-load-kg-core`. + +### Connection Errors +```bash +Exception: Connection refused +``` +**Solution**: Check the API URL and ensure TrustGraph is running. + +### Permission Errors +```bash +Exception: Access denied +``` +**Solution**: Verify user permissions for the specified collection. + +## Environment Variables + +- `TRUSTGRAPH_URL`: Default API URL + +## Related Commands + +- [`tg-graph-to-turtle`](tg-graph-to-turtle.md) - Export graph to Turtle format +- [`tg-load-kg-core`](tg-load-kg-core.md) - Load knowledge into graph +- [`tg-show-kg-cores`](tg-show-kg-cores.md) - List available knowledge cores +- [`tg-invoke-graph-rag`](tg-invoke-graph-rag.md) - Query graph with natural language +- [`tg-load-turtle`](tg-load-turtle.md) - Import RDF data from Turtle files + +## API Integration + +This command uses the [Triples Query API](../apis/api-triples-query.md) to retrieve knowledge graph triples with no filtering constraints. + +## Use Cases + +### Knowledge Exploration +```bash +# Explore what knowledge is available +tg-show-graph | head -50 + +# Look for specific entities +tg-show-graph | grep "Einstein" +``` + +### Data Verification +```bash +# Verify knowledge loading worked correctly +tg-load-kg-core --kg-core-id "research-data" --flow-id "research-flow" +tg-show-graph -f research-flow | wc -l +``` + +### Debugging Knowledge Issues +```bash +# Check if specific relationships exist +tg-show-graph | grep "hasName" +tg-show-graph | grep "createdBy" +``` + +### Graph Analysis +```bash +# Count different relationship types +tg-show-graph | awk '{print $2}' | sort | uniq -c + +# Find most connected entities +tg-show-graph | awk '{print $1}' | sort | uniq -c | sort -nr +``` + +### Data Quality Assessment +```bash +# Check for malformed triples +tg-show-graph | grep -v "^<.*> <.*>" + +# Verify URI patterns +tg-show-graph | grep "http://" | head -20 +``` + +## Output Processing + +### Filter by Predicate +```bash +# Show only name relationships +tg-show-graph | grep "hasName" + +# Show only authorship +tg-show-graph | grep "createdBy" +``` + +### Extract Entities +```bash +# List all subjects (entities) +tg-show-graph | awk '{print $1}' | sort | uniq + +# List all predicates (relationships) +tg-show-graph | awk '{print $2}' | sort | uniq +``` + +### Export Subsets +```bash +# Save specific relationships +tg-show-graph | grep "Organization" > organization-data.txt + +# Save person-related triples +tg-show-graph | grep "Person" > person-data.txt +``` + +## Performance Considerations + +### Large Graphs +For graphs with many triples: +- Command may take time to retrieve 10,000 triples +- Consider using filtered queries for specific data +- Use `tg-graph-to-turtle` for complete export + +### Memory Usage +- Output is streamed, so memory usage is manageable +- Piping to other commands processes data incrementally + +## Best Practices + +1. **Start Small**: Begin with small collections to understand structure +2. **Use Filters**: Pipe output through grep/awk for specific data +3. **Regular Inspection**: Periodically check graph contents +4. **Data Validation**: Verify expected relationships exist +5. **Performance Monitoring**: Monitor query time for large graphs +6. **Collection Organization**: Use collections to organize different domains + +## Integration Examples + +### With Other Tools +```bash +# Convert to different formats +tg-show-graph | sed 's/[<>"]//g' > simple-triples.txt + +# Create entity lists +tg-show-graph | awk '{print $1}' | sort | uniq > entities.txt + +# Generate statistics +tg-show-graph | wc -l +echo "Total triples in graph" +``` + +### Graph Exploration Workflow +```bash +# 1. Check available knowledge +tg-show-kg-cores + +# 2. Load knowledge into flow +tg-load-kg-core --kg-core-id "my-knowledge" --flow-id "my-flow" + +# 3. Explore the graph +tg-show-graph -f my-flow + +# 4. Query specific information +tg-invoke-graph-rag -q "What entities are in the graph?" -f my-flow +``` \ No newline at end of file