Release/v1.2 (#457)

* Bump setup.py versions for 1.1

* PoC MCP server (#419)

* Very initial MCP server PoC for TrustGraph

* Put service on port 8000

* Add MCP container and packages to buildout

* Update docs for API/CLI changes in 1.0 (#421)

* Update some API basics for the 0.23/1.0 API change

* Add MCP container push (#425)

* Add command args to the MCP server (#426)

* Host and port parameters

* Added websocket arg

* More docs

* MCP client support (#427)

- MCP client service
- Tool request/response schema
- API gateway support for mcp-tool
- Message translation for tool request & response
- Make mcp-tool using configuration service for information
  about where the MCP services are.

* Feature/react call mcp (#428)

Key Features

  - MCP Tool Integration: Added core MCP tool support with ToolClientSpec and ToolClient classes
  - API Enhancement: New mcp_tool method for flow-specific tool invocation
  - CLI Tooling: New tg-invoke-mcp-tool command for testing MCP integration
  - React Agent Enhancement: Fixed and improved multi-tool invocation capabilities
  - Tool Management: Enhanced CLI for tool configuration and management

Changes

  - Added MCP tool invocation to API with flow-specific integration
  - Implemented ToolClientSpec and ToolClient for tool call handling
  - Updated agent-manager-react to invoke MCP tools with configurable types
  - Enhanced CLI with new commands and improved help text
  - Added comprehensive documentation for new CLI commands
  - Improved tool configuration management

Testing

  - Added tg-invoke-mcp-tool CLI command for isolated MCP integration testing
  - Enhanced agent capability to invoke multiple tools simultaneously

* Test suite executed from CI pipeline (#433)

* Test strategy & test cases

* Unit tests

* Integration tests

* Extending test coverage (#434)

* Contract tests

* Testing embeedings

* Agent unit tests

* Knowledge pipeline tests

* Turn on contract tests

* Increase storage test coverage (#435)

* Fixing storage and adding tests

* PR pipeline only runs quick tests

* Empty configuration is returned as empty list, previously was not in response (#436)

* Update config util to take files as well as command-line text (#437)

* Updated CLI invocation and config model for tools and mcp (#438)

* Updated CLI invocation and config model for tools and mcp

* CLI anomalies

* Tweaked the MCP tool implementation for new model

* Update agent implementation to match the new model

* Fix agent tools, now all tested

* Fixed integration tests

* Fix MCP delete tool params

* Update Python deps to 1.2

* Update to enable knowledge extraction using the agent framework (#439)

* Implement KG extraction agent (kg-extract-agent)

* Using ReAct framework (agent-manager-react)
 
* ReAct manager had an issue when emitting JSON, which conflicts which ReAct manager's own JSON messages, so refactored ReAct manager to use traditional ReAct messages, non-JSON structure.
 
* Minor refactor to take the prompt template client out of prompt-template so it can be more readily used by other modules. kg-extract-agent uses this framework.

* Migrate from setup.py to pyproject.toml (#440)

* Converted setup.py to pyproject.toml

* Modern package infrastructure as recommended by py docs

* Install missing build deps (#441)

* Install missing build deps (#442)

* Implement logging strategy (#444)

* Logging strategy and convert all prints() to logging invocations

* Fix/startup failure (#445)

* Fix loggin startup problems

* Fix logging startup problems (#446)

* Fix logging startup problems (#447)

* Fixed Mistral OCR to use current API (#448)

* Fixed Mistral OCR to use current API

* Added PDF decoder tests

* Fix Mistral OCR ident to be standard pdf-decoder (#450)

* Fix Mistral OCR ident to be standard pdf-decoder

* Correct test

* Schema structure refactor (#451)

* Write schema refactor spec

* Implemented schema refactor spec

* Structure data mvp (#452)

* Structured data tech spec

* Architecture principles

* New schemas

* Updated schemas and specs

* Object extractor

* Add .coveragerc

* New tests

* Cassandra object storage

* Trying to object extraction working, issues exist

* Validate librarian collection (#453)

* Fix token chunker, broken API invocation (#454)

* Fix token chunker, broken API invocation (#455)

* Knowledge load utility CLI (#456)

* Knowledge loader

* More tests
This commit is contained in:
cybermaggedon 2025-08-18 20:56:09 +01:00 committed by GitHub
parent c85ba197be
commit 89be656990
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
509 changed files with 49632 additions and 5159 deletions

View file

@ -0,0 +1,86 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "trustgraph-cli"
dynamic = ["version"]
authors = [{name = "trustgraph.ai", email = "security@trustgraph.ai"}]
description = "TrustGraph provides a means to run a pipeline of flexible AI processing components in a flexible means to achieve a processing pipeline."
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
"trustgraph-base>=1.2,<1.3",
"requests",
"pulsar-client",
"aiohttp",
"rdflib",
"tabulate",
"msgpack",
"websockets",
]
classifiers = [
"Programming Language :: Python :: 3",
"Operating System :: OS Independent",
]
[project.urls]
Homepage = "https://github.com/trustgraph-ai/trustgraph"
[project.scripts]
tg-add-library-document = "trustgraph.cli.add_library_document:main"
tg-delete-flow-class = "trustgraph.cli.delete_flow_class:main"
tg-delete-mcp-tool = "trustgraph.cli.delete_mcp_tool:main"
tg-delete-kg-core = "trustgraph.cli.delete_kg_core:main"
tg-delete-tool = "trustgraph.cli.delete_tool:main"
tg-dump-msgpack = "trustgraph.cli.dump_msgpack:main"
tg-get-flow-class = "trustgraph.cli.get_flow_class:main"
tg-get-kg-core = "trustgraph.cli.get_kg_core:main"
tg-graph-to-turtle = "trustgraph.cli.graph_to_turtle:main"
tg-init-trustgraph = "trustgraph.cli.init_trustgraph:main"
tg-invoke-agent = "trustgraph.cli.invoke_agent:main"
tg-invoke-document-rag = "trustgraph.cli.invoke_document_rag:main"
tg-invoke-graph-rag = "trustgraph.cli.invoke_graph_rag:main"
tg-invoke-llm = "trustgraph.cli.invoke_llm:main"
tg-invoke-mcp-tool = "trustgraph.cli.invoke_mcp_tool:main"
tg-invoke-prompt = "trustgraph.cli.invoke_prompt:main"
tg-load-doc-embeds = "trustgraph.cli.load_doc_embeds:main"
tg-load-kg-core = "trustgraph.cli.load_kg_core:main"
tg-load-pdf = "trustgraph.cli.load_pdf:main"
tg-load-sample-documents = "trustgraph.cli.load_sample_documents:main"
tg-load-text = "trustgraph.cli.load_text:main"
tg-load-turtle = "trustgraph.cli.load_turtle:main"
tg-load-knowledge = "trustgraph.cli.load_knowledge:main"
tg-put-flow-class = "trustgraph.cli.put_flow_class:main"
tg-put-kg-core = "trustgraph.cli.put_kg_core:main"
tg-remove-library-document = "trustgraph.cli.remove_library_document:main"
tg-save-doc-embeds = "trustgraph.cli.save_doc_embeds:main"
tg-set-mcp-tool = "trustgraph.cli.set_mcp_tool:main"
tg-set-prompt = "trustgraph.cli.set_prompt:main"
tg-set-token-costs = "trustgraph.cli.set_token_costs:main"
tg-set-tool = "trustgraph.cli.set_tool:main"
tg-show-config = "trustgraph.cli.show_config:main"
tg-show-flow-classes = "trustgraph.cli.show_flow_classes:main"
tg-show-flow-state = "trustgraph.cli.show_flow_state:main"
tg-show-flows = "trustgraph.cli.show_flows:main"
tg-show-graph = "trustgraph.cli.show_graph:main"
tg-show-kg-cores = "trustgraph.cli.show_kg_cores:main"
tg-show-library-documents = "trustgraph.cli.show_library_documents:main"
tg-show-library-processing = "trustgraph.cli.show_library_processing:main"
tg-show-mcp-tools = "trustgraph.cli.show_mcp_tools:main"
tg-show-processor-state = "trustgraph.cli.show_processor_state:main"
tg-show-prompts = "trustgraph.cli.show_prompts:main"
tg-show-token-costs = "trustgraph.cli.show_token_costs:main"
tg-show-token-rate = "trustgraph.cli.show_token_rate:main"
tg-show-tools = "trustgraph.cli.show_tools:main"
tg-start-flow = "trustgraph.cli.start_flow:main"
tg-unload-kg-core = "trustgraph.cli.unload_kg_core:main"
tg-start-library-processing = "trustgraph.cli.start_library_processing:main"
tg-stop-flow = "trustgraph.cli.stop_flow:main"
tg-stop-library-processing = "trustgraph.cli.stop_library_processing:main"
[tool.setuptools.packages.find]
include = ["trustgraph*"]
[tool.setuptools.dynamic]
version = {attr = "trustgraph.cli_version.__version__"}

View file

@ -1,91 +0,0 @@
import setuptools
import os
import importlib
with open("README.md", "r") as fh:
long_description = fh.read()
# Load a version number module
spec = importlib.util.spec_from_file_location(
'version', 'trustgraph/cli_version.py'
)
version_module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(version_module)
version = version_module.__version__
setuptools.setup(
name="trustgraph-cli",
version=version,
author="trustgraph.ai",
author_email="security@trustgraph.ai",
description="TrustGraph provides a means to run a pipeline of flexible AI processing components in a flexible means to achieve a processing pipeline.",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/trustgraph-ai/trustgraph",
packages=setuptools.find_namespace_packages(
where='./',
),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
"Operating System :: OS Independent",
],
python_requires='>=3.8',
download_url = "https://github.com/trustgraph-ai/trustgraph/archive/refs/tags/v" + version + ".tar.gz",
install_requires=[
"trustgraph-base>=1.0,<1.1",
"requests",
"pulsar-client",
"aiohttp",
"rdflib",
"tabulate",
"msgpack",
"websockets",
],
scripts=[
"scripts/tg-add-library-document",
"scripts/tg-delete-flow-class",
"scripts/tg-delete-kg-core",
"scripts/tg-dump-msgpack",
"scripts/tg-get-flow-class",
"scripts/tg-get-kg-core",
"scripts/tg-graph-to-turtle",
"scripts/tg-init-trustgraph",
"scripts/tg-invoke-agent",
"scripts/tg-invoke-document-rag",
"scripts/tg-invoke-graph-rag",
"scripts/tg-invoke-llm",
"scripts/tg-invoke-prompt",
"scripts/tg-load-doc-embeds",
"scripts/tg-load-kg-core",
"scripts/tg-load-pdf",
"scripts/tg-load-sample-documents",
"scripts/tg-load-text",
"scripts/tg-load-turtle",
"scripts/tg-put-flow-class",
"scripts/tg-put-kg-core",
"scripts/tg-remove-library-document",
"scripts/tg-save-doc-embeds",
"scripts/tg-set-prompt",
"scripts/tg-set-token-costs",
"scripts/tg-show-config",
"scripts/tg-show-flow-classes",
"scripts/tg-show-flow-state",
"scripts/tg-show-flows",
"scripts/tg-show-graph",
"scripts/tg-show-kg-cores",
"scripts/tg-show-library-documents",
"scripts/tg-show-library-processing",
"scripts/tg-show-processor-state",
"scripts/tg-show-prompts",
"scripts/tg-show-token-costs",
"scripts/tg-show-token-rate",
"scripts/tg-show-tools",
"scripts/tg-start-flow",
"scripts/tg-unload-kg-core",
"scripts/tg-start-library-processing",
"scripts/tg-stop-flow",
"scripts/tg-stop-library-processing",
]
)

View file

@ -0,0 +1 @@
# TrustGraph CLI modules

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Loads a document into the library
"""
@ -202,5 +200,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Deletes a flow class
"""
@ -49,5 +47,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Deletes a flow class
"""
@ -57,5 +55,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,93 @@
"""
Deletes MCP (Model Control Protocol) tools from the TrustGraph system.
Removes MCP tool configurations by ID from the 'mcp' configuration group.
"""
import argparse
import os
from trustgraph.api import Api, ConfigKey
import textwrap
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
def delete_mcp_tool(
url : str,
id : str,
):
api = Api(url).config()
# Check if the tool exists first
try:
values = api.get([
ConfigKey(type="mcp", key=id)
])
if not values or not values[0].value:
print(f"MCP tool '{id}' not found.")
return False
except Exception as e:
print(f"MCP tool '{id}' not found.")
return False
# Delete the MCP tool configuration from the 'mcp' group
try:
api.delete([
ConfigKey(type="mcp", key=id)
])
print(f"MCP tool '{id}' deleted successfully.")
return True
except Exception as e:
print(f"Error deleting MCP tool '{id}': {e}")
return False
def main():
parser = argparse.ArgumentParser(
prog='tg-delete-mcp-tool',
description=__doc__,
epilog=textwrap.dedent('''
This utility removes MCP tool configurations from the TrustGraph system.
Once deleted, the tool will no longer be available for use.
Examples:
%(prog)s --id weather
%(prog)s --id calculator
%(prog)s --api-url http://localhost:9000/ --id file-reader
''').strip(),
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'--id',
required=True,
help='MCP tool ID to delete',
)
args = parser.parse_args()
try:
if not args.id:
raise RuntimeError("Must specify --id for MCP tool to delete")
delete_mcp_tool(
url=args.api_url,
id=args.id
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -0,0 +1,98 @@
"""
Deletes tools from the TrustGraph system.
Removes tool configurations by ID from the agent configuration
and updates the tool index accordingly.
"""
import argparse
import os
from trustgraph.api import Api, ConfigKey, ConfigValue
import json
import textwrap
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
def delete_tool(
url : str,
id : str,
):
api = Api(url).config()
# Check if the tool configuration exists
try:
tool_values = api.get([
ConfigKey(type="tool", key=id)
])
if not tool_values or not tool_values[0].value:
print(f"Tool configuration for '{id}' not found.")
return False
except Exception as e:
print(f"Tool configuration for '{id}' not found.")
return False
# Delete the tool configuration and update the index
try:
# Delete the tool configuration
api.delete([
ConfigKey(type="tool", key=id)
])
print(f"Tool '{id}' deleted successfully.")
return True
except Exception as e:
print(f"Error deleting tool '{id}': {e}")
return False
def main():
parser = argparse.ArgumentParser(
prog='tg-delete-tool',
description=__doc__,
epilog=textwrap.dedent('''
This utility removes tool configurations from the TrustGraph system.
It removes the tool from both the tool index and deletes the tool
configuration. Once deleted, the tool will no longer be available for use.
Examples:
%(prog)s --id weather
%(prog)s --id calculator
%(prog)s --api-url http://localhost:9000/ --id file-reader
''').strip(),
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'--id',
required=True,
help='Tool ID to delete',
)
args = parser.parse_args()
try:
if not args.id:
raise RuntimeError("Must specify --id for tool to delete")
delete_tool(
url=args.api_url,
id=args.id
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
This utility reads a knowledge core in msgpack format and outputs its
contents in JSON form to standard output. This is useful only as a
@ -89,5 +87,5 @@ def main():
else:
dump(**vars(args))
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Outputs a flow class definition in JSON format.
"""
@ -52,5 +50,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uses the knowledge service to fetch a knowledge core which is saved
to a local file in msgpack format.
@ -157,5 +155,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Connects to the graph query service and dumps all graph edges in Turtle
format.
@ -102,5 +100,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Initialises Pulsar with Trustgraph tenant / namespaces & policy.
"""
@ -118,7 +116,10 @@ def ensure_config(config, pulsar_host, pulsar_api_key):
print("Retrying...", flush=True)
continue
def init(pulsar_admin_url, pulsar_host, pulsar_api_key, config, tenant):
def init(
pulsar_admin_url, pulsar_host, pulsar_api_key, tenant,
config, config_file,
):
clusters = get_clusters(pulsar_admin_url)
@ -156,6 +157,18 @@ def init(pulsar_admin_url, pulsar_host, pulsar_api_key, config, tenant):
ensure_config(dec, pulsar_host, pulsar_api_key)
elif config_file is not None:
try:
print("Decoding config...", flush=True)
dec = json.load(open(config_file))
print("Decoded.", flush=True)
except Exception as e:
print("Exception:", e, flush=True)
raise e
ensure_config(dec, pulsar_host, pulsar_api_key)
else:
print("No config to update.", flush=True)
@ -188,6 +201,11 @@ def main():
help=f'Initial configuration to load',
)
parser.add_argument(
'-C', '--config-file',
help=f'Initial configuration to load from file',
)
parser.add_argument(
'-t', '--tenant',
default="tg",
@ -217,5 +235,5 @@ def main():
time.sleep(2)
print("Will retry...", flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uses the agent service to answer a question
"""
@ -169,5 +167,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uses the DocumentRAG service to answer a question
"""
@ -84,5 +82,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uses the GraphRAG service to answer a question
"""
@ -113,5 +111,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Invokes the text completion service by specifying an LLM system prompt
and user prompt. Both arguments are required.
@ -66,5 +64,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,78 @@
"""
Invokes MCP (Model Control Protocol) tools through the TrustGraph API.
Allows calling MCP tools by specifying the tool name and providing
parameters as a JSON-encoded dictionary. The tool is executed within
the context of a specified flow.
"""
import argparse
import os
import json
from trustgraph.api import Api
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
def query(url, flow_id, name, parameters):
api = Api(url).flow().id(flow_id)
resp = api.mcp_tool(name=name, parameters=parameters)
if isinstance(resp, str):
print(resp)
else:
print(json.dumps(resp, indent=4))
def main():
parser = argparse.ArgumentParser(
prog='tg-invoke-mcp-tool',
description=__doc__,
)
parser.add_argument(
'-u', '--url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'-f', '--flow-id',
default="default",
help=f'Flow ID (default: default)'
)
parser.add_argument(
'-n', '--name',
metavar='tool-name',
help=f'MCP tool name',
)
parser.add_argument(
'-P', '--parameters',
help='''Tool parameters, should be JSON-encoded dict.''',
)
args = parser.parse_args()
if args.parameters:
parameters = json.loads(args.parameters)
else:
parameters = {}
try:
query(
url = args.url,
flow_id = args.flow_id,
name = args.name,
parameters = parameters,
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Invokes the LLM prompt service by specifying the prompt template to use
and values for the variables in the prompt template. The
@ -86,5 +84,5 @@ specified multiple times''',
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
This utility takes a document embeddings core and loads it into a running
TrustGraph through the API. The document embeddings core should be in msgpack

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Starts a load operation on a knowledge core which is already stored by
the knowledge manager. You could load a core with tg-put-kg-core and then
@ -76,5 +74,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,202 @@
"""
Loads triples and entity contexts into the knowledge graph.
"""
import asyncio
import argparse
import os
import time
import rdflib
import json
from websockets.asyncio.client import connect
from typing import List, Dict, Any
from trustgraph.log_level import LogLevel
default_url = os.getenv("TRUSTGRAPH_URL", 'ws://localhost:8088/')
default_user = 'trustgraph'
default_collection = 'default'
class KnowledgeLoader:
def __init__(
self,
files,
flow,
user,
collection,
document_id,
url = default_url,
):
if not url.endswith("/"):
url += "/"
self.triples_url = url + f"api/v1/flow/{flow}/import/triples"
self.entity_contexts_url = url + f"api/v1/flow/{flow}/import/entity-contexts"
self.files = files
self.user = user
self.collection = collection
self.document_id = document_id
async def run(self):
try:
# Load triples first
async with connect(self.triples_url) as ws:
for file in self.files:
await self.load_triples(file, ws)
# Then load entity contexts
async with connect(self.entity_contexts_url) as ws:
for file in self.files:
await self.load_entity_contexts(file, ws)
except Exception as e:
print(e, flush=True)
async def load_triples(self, file, ws):
g = rdflib.Graph()
g.parse(file, format="turtle")
def Value(value, is_uri):
return { "v": value, "e": is_uri }
for e in g:
s = Value(value=str(e[0]), is_uri=True)
p = Value(value=str(e[1]), is_uri=True)
if type(e[2]) == rdflib.term.URIRef:
o = Value(value=str(e[2]), is_uri=True)
else:
o = Value(value=str(e[2]), is_uri=False)
req = {
"metadata": {
"id": self.document_id,
"metadata": [],
"user": self.user,
"collection": self.collection
},
"triples": [
{
"s": s,
"p": p,
"o": o,
}
]
}
await ws.send(json.dumps(req))
async def load_entity_contexts(self, file, ws):
"""
Load entity contexts by extracting entities from the RDF graph
and generating contextual descriptions based on their relationships.
"""
g = rdflib.Graph()
g.parse(file, format="turtle")
for s, p, o in g:
# If object is a URI, do nothing
if isinstance(o, rdflib.term.URIRef):
continue
# If object is a literal, create entity context for subject with literal as context
s_str = str(s)
o_str = str(o)
req = {
"metadata": {
"id": self.document_id,
"metadata": [],
"user": self.user,
"collection": self.collection
},
"entities": [
{
"entity": {
"v": s_str,
"e": True
},
"context": o_str
}
]
}
await ws.send(json.dumps(req))
def main():
parser = argparse.ArgumentParser(
prog='tg-load-knowledge',
description=__doc__,
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'-i', '--document-id',
required=True,
help=f'Document ID)',
)
parser.add_argument(
'-f', '--flow-id',
default="default",
help=f'Flow ID (default: default)'
)
parser.add_argument(
'-U', '--user',
default=default_user,
help=f'User ID (default: {default_user})'
)
parser.add_argument(
'-C', '--collection',
default=default_collection,
help=f'Collection ID (default: {default_collection})'
)
parser.add_argument(
'files', nargs='+',
help=f'Turtle files to load'
)
args = parser.parse_args()
while True:
try:
loader = KnowledgeLoader(
document_id = args.document_id,
url = args.api_url,
flow = args.flow_id,
files = args.files,
user = args.user,
collection = args.collection,
)
asyncio.run(loader.run())
print("Triples and entity contexts loaded.")
break
except Exception as e:
print("Exception:", e, flush=True)
print("Will retry...", flush=True)
time.sleep(10)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Loads a PDF document into TrustGraph processing by directing to
the pdf-decoder queue.
@ -198,5 +196,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Loads a PDF document into the library
"""
@ -737,5 +735,5 @@ def main():
print("Exception:", e, flush=True)
raise e
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Loads a text document into TrustGraph processing by directing to a text
loader queue.
@ -203,6 +201,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Loads triples into the knowledge graph.
"""
@ -157,5 +155,5 @@ def main():
time.sleep(10)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uploads a flow class definition. You can take the output of
tg-get-flow-class and load it back in using this utility.
@ -55,5 +53,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Uses the agent service to answer a question
"""
@ -179,5 +177,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Remove a document from the library
"""
@ -55,5 +53,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
This utility connects to a running TrustGraph through the API and creates
a document embeddings core from the data streaming through the processing

View file

@ -0,0 +1,109 @@
"""
Configures and registers MCP (Model Context Protocol) tools in the
TrustGraph system.
MCP tools are external services that follow the Model Context Protocol
specification. This script stores MCP tool configurations with:
- id: Unique identifier for the tool
- remote-name: Name used by the MCP server (defaults to id)
- url: MCP server endpoint URL
Configurations are stored in the 'mcp' configuration group and can be
referenced by agent tools using the 'mcp-tool' type.
"""
import argparse
import os
from trustgraph.api import Api, ConfigValue
import textwrap
import json
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
def set_mcp_tool(
url : str,
id : str,
remote_name : str,
tool_url : str,
):
api = Api(url).config()
# Store the MCP tool configuration in the 'mcp' group
values = api.put([
ConfigValue(
type="mcp", key=id, value=json.dumps({
"remote-name": remote_name,
"url": tool_url,
})
)
])
def main():
parser = argparse.ArgumentParser(
prog='tg-set-mcp-tool',
description=__doc__,
epilog=textwrap.dedent('''
MCP tools are configured with just a name and URL. The URL should point
to the MCP server endpoint that provides the tool functionality.
Examples:
%(prog)s --id weather --tool-url "http://localhost:3000/weather"
%(prog)s --id calculator --tool-url "http://mcp-tools.example.com/calc"
''').strip(),
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'-i', '--id',
required=True,
help='MCP tool identifier',
)
parser.add_argument(
'-r', '--remote-name',
required=False,
help='Remote MCP tool name (defaults to --id if not specified)',
)
parser.add_argument(
'--tool-url',
required=True,
help='MCP tool URL endpoint',
)
args = parser.parse_args()
try:
if not args.id:
raise RuntimeError("Must specify --id for MCP tool")
if not args.tool_url:
raise RuntimeError("Must specify --tool-url for MCP tool")
if args.remote_name:
remote_name = args.remote_name
else:
remote_name = args.id
set_mcp_tool(
url=args.api_url,
id=args.id,
remote_name=remote_name,
tool_url=args.tool_url
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Sets a prompt template.
"""
@ -139,5 +137,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Sets a model's token costs.
"""
@ -107,5 +105,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,224 @@
"""
Configures and registers tools in the TrustGraph system.
This script allows you to define agent tools with various types including:
- knowledge-query: Query knowledge bases
- text-completion: Text generation
- mcp-tool: Reference to MCP (Model Context Protocol) tools
- prompt: Prompt template execution
Tools are stored in the 'tool' configuration group and can include
argument specifications for parameterized execution.
"""
from typing import List
import argparse
import os
from trustgraph.api import Api, ConfigKey, ConfigValue
import json
import tabulate
import textwrap
import dataclasses
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
@dataclasses.dataclass
class Argument:
name : str
type : str
description : str
@staticmethod
def parse(s):
parts = s.split(":")
if len(parts) != 3:
raise RuntimeError(
"Arguments should be form name:type:description"
)
valid_types = [
"string", "number",
]
if parts[1] not in valid_types:
raise RuntimeError(
f"Type {parts[1]} invalid, use: " +
", ".join(valid_types)
)
return Argument(name=parts[0], type=parts[1], description=parts[2])
def set_tool(
url : str,
id : str,
name : str,
description : str,
type : str,
mcp_tool : str,
collection : str,
template : str,
arguments : List[Argument],
):
api = Api(url).config()
values = api.get([
ConfigKey(type="agent", key="tool-index")
])
object = {
"name": name,
"description": description,
"type": type,
}
if mcp_tool: object["mcp-tool"] = mcp_tool
if collection: object["collection"] = collection
if template: object["template"] = template
if arguments:
object["arguments"] = [
{
"name": a.name,
"type": a.type,
"description": a.description,
}
for a in arguments
]
values = api.put([
ConfigValue(
type="tool", key=f"{id}", value=json.dumps(object)
)
])
print("Tool set.")
def main():
parser = argparse.ArgumentParser(
prog='tg-set-tool',
description=__doc__,
epilog=textwrap.dedent('''
Valid tool types:
knowledge-query - Query knowledge bases
text-completion - Text completion/generation
mcp-tool - Model Control Protocol tool
prompt - Prompt template query
Valid argument types:
string - String/text parameter
number - Numeric parameter
Examples:
%(prog)s --id weather --name "Weather lookup" \\
--type knowledge-query \\
--description "Get weather information" \\
--argument location:string:"Location to query" \\
--argument units:string:"Temperature units (C/F)"
%(prog)s --id calculator --name "Calculator" --type mcp-tool \\
--description "Perform calculations" \\
--argument expression:string:"Mathematical expression"
''').strip(),
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
parser.add_argument(
'--id',
help=f'Unique tool identifier',
)
parser.add_argument(
'--name',
help=f'Human-readable tool name',
)
parser.add_argument(
'--description',
help=f'Detailed description of what the tool does',
)
parser.add_argument(
'--type',
help=f'Tool type, one of: knowledge-query, text-completion, mcp-tool, prompt',
)
parser.add_argument(
'--mcp-tool',
help=f'For MCP type: ID of MCP tool configuration (as defined by tg-set-mcp-tool)',
)
parser.add_argument(
'--collection',
help=f'For knowledge-query type: collection to query',
)
parser.add_argument(
'--template',
help=f'For prompt type: template ID to use',
)
parser.add_argument(
'--argument',
nargs="*",
help=f'Tool arguments in the form: name:type:description (can specify multiple)',
)
args = parser.parse_args()
try:
valid_types = [
"knowledge-query", "text-completion", "mcp-tool", "prompt"
]
if args.id is None:
raise RuntimeError("Must specify --id for tool")
if args.name is None:
raise RuntimeError("Must specify --name for tool")
if args.type:
if args.type not in valid_types:
raise RuntimeError(
"Type must be one of: " + ", ".join(valid_types)
)
mcp_tool = args.mcp_tool
if args.argument:
arguments = [
Argument.parse(a)
for a in args.argument
]
else:
arguments = []
set_tool(
url=args.api_url,
id=args.id,
name=args.name,
description=args.description,
type=args.type,
mcp_tool=mcp_tool,
collection=args.collection,
template=args.template,
arguments=arguments,
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dumps out the current configuration
"""
@ -45,5 +43,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Shows all defined flow classes.
"""
@ -65,5 +63,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dump out a flow's processor states
"""
@ -89,5 +87,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Shows configured flows.
"""
@ -110,5 +108,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Connects to the graph query service and dumps all graph edges.
"""
@ -70,5 +68,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Shows knowledge cores
"""
@ -55,5 +53,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Shows all loaded library documents
"""
@ -72,5 +70,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
"""
@ -71,5 +69,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,7 +1,5 @@
#!/usr/bin/env python3
"""
Dumps out the current agent tool configuration
Displays the current MCP (Model Context Protocol) tool configuration
"""
import argparse
@ -17,36 +15,19 @@ def show_config(url):
api = Api(url).config()
values = api.get([
ConfigKey(type="agent", key="tool-index")
])
values = api.get_values(type="mcp")
ix = json.loads(values[0].value)
for n, value in enumerate(values):
values = api.get([
ConfigKey(type="agent", key=f"tool.{v}")
for v in ix
])
for n, key in enumerate(ix):
data = json.loads(values[n].value)
data = json.loads(value.value)
table = []
table.append(("id", data["id"]))
table.append(("name", data["name"]))
table.append(("description", data["description"]))
for n, arg in enumerate(data["arguments"]):
table.append((
f"arg {n}",
f"{arg['name']}: {arg['type']}\n{arg['description']}"
))
table.append(("id", value.key))
table.append(("remote-name", data["remote-name"]))
table.append(("url", data["url"]))
print()
print(key + ":")
print(tabulate.tabulate(
table,
@ -60,7 +41,7 @@ def show_config(url):
def main():
parser = argparse.ArgumentParser(
prog='tg-show-tools',
prog='tg-show-mcp-tools',
description=__doc__,
)
@ -82,5 +63,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dump out TrustGraph processor states.
"""
@ -51,5 +49,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dumps out the current prompts
"""
@ -92,5 +90,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dumps out token cost configuration
"""
@ -75,5 +73,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Dump out a stream of token rates, input, output and total. This is averaged
across the time since tg-show-token-rate is started.
@ -105,5 +103,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,91 @@
"""
Displays the current agent tool configurations
Shows all configured tools including their types:
- knowledge-query: Tools that query knowledge bases
- text-completion: Tools for text generation
- mcp-tool: References to MCP (Model Context Protocol) tools
- prompt: Tools that execute prompt templates
"""
import argparse
import os
from trustgraph.api import Api, ConfigKey
import json
import tabulate
import textwrap
default_url = os.getenv("TRUSTGRAPH_URL", 'http://localhost:8088/')
def show_config(url):
api = Api(url).config()
values = api.get_values(type="tool")
for item in values:
id = item.key
data = json.loads(item.value)
tp = data["type"]
table = []
table.append(("id", id))
table.append(("name", data["name"]))
table.append(("description", data["description"]))
table.append(("type", tp))
if tp == "mcp-tool":
table.append(("mcp-tool", data["mcp-tool"]))
if tp == "knowledge-query":
table.append(("collection", data["collection"]))
if tp == "prompt":
table.append(("template", data["template"]))
for n, arg in enumerate(data["arguments"]):
table.append((
f"arg {n}",
f"{arg['name']}: {arg['type']}\n{arg['description']}"
))
print()
print(tabulate.tabulate(
table,
tablefmt="pretty",
maxcolwidths=[None, 70],
stralign="left"
))
print()
def main():
parser = argparse.ArgumentParser(
prog='tg-show-tools',
description=__doc__,
)
parser.add_argument(
'-u', '--api-url',
default=default_url,
help=f'API URL (default: {default_url})',
)
args = parser.parse_args()
try:
show_config(
url=args.api_url,
)
except Exception as e:
print("Exception:", e, flush=True)
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Starts a processing flow using a defined flow class
"""
@ -68,5 +66,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Submits a library document for processing
"""
@ -99,5 +97,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Stops a processing flow.
"""
@ -50,5 +48,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Removes a library document processing record. This is just a record of
procesing, it doesn't stop in-flight processing at the moment.
@ -61,5 +59,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()

View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
"""
Starts a load operation on a knowledge core which is already stored by
the knowledge manager. You could load a core with tg-put-kg-core and then
@ -68,5 +66,5 @@ def main():
print("Exception:", e, flush=True)
main()
if __name__ == "__main__":
main()