Release/v1.2 (#457)

* Bump setup.py versions for 1.1

* PoC MCP server (#419)

* Very initial MCP server PoC for TrustGraph

* Put service on port 8000

* Add MCP container and packages to buildout

* Update docs for API/CLI changes in 1.0 (#421)

* Update some API basics for the 0.23/1.0 API change

* Add MCP container push (#425)

* Add command args to the MCP server (#426)

* Host and port parameters

* Added websocket arg

* More docs

* MCP client support (#427)

- MCP client service
- Tool request/response schema
- API gateway support for mcp-tool
- Message translation for tool request & response
- Make mcp-tool using configuration service for information
  about where the MCP services are.

* Feature/react call mcp (#428)

Key Features

  - MCP Tool Integration: Added core MCP tool support with ToolClientSpec and ToolClient classes
  - API Enhancement: New mcp_tool method for flow-specific tool invocation
  - CLI Tooling: New tg-invoke-mcp-tool command for testing MCP integration
  - React Agent Enhancement: Fixed and improved multi-tool invocation capabilities
  - Tool Management: Enhanced CLI for tool configuration and management

Changes

  - Added MCP tool invocation to API with flow-specific integration
  - Implemented ToolClientSpec and ToolClient for tool call handling
  - Updated agent-manager-react to invoke MCP tools with configurable types
  - Enhanced CLI with new commands and improved help text
  - Added comprehensive documentation for new CLI commands
  - Improved tool configuration management

Testing

  - Added tg-invoke-mcp-tool CLI command for isolated MCP integration testing
  - Enhanced agent capability to invoke multiple tools simultaneously

* Test suite executed from CI pipeline (#433)

* Test strategy & test cases

* Unit tests

* Integration tests

* Extending test coverage (#434)

* Contract tests

* Testing embeedings

* Agent unit tests

* Knowledge pipeline tests

* Turn on contract tests

* Increase storage test coverage (#435)

* Fixing storage and adding tests

* PR pipeline only runs quick tests

* Empty configuration is returned as empty list, previously was not in response (#436)

* Update config util to take files as well as command-line text (#437)

* Updated CLI invocation and config model for tools and mcp (#438)

* Updated CLI invocation and config model for tools and mcp

* CLI anomalies

* Tweaked the MCP tool implementation for new model

* Update agent implementation to match the new model

* Fix agent tools, now all tested

* Fixed integration tests

* Fix MCP delete tool params

* Update Python deps to 1.2

* Update to enable knowledge extraction using the agent framework (#439)

* Implement KG extraction agent (kg-extract-agent)

* Using ReAct framework (agent-manager-react)
 
* ReAct manager had an issue when emitting JSON, which conflicts which ReAct manager's own JSON messages, so refactored ReAct manager to use traditional ReAct messages, non-JSON structure.
 
* Minor refactor to take the prompt template client out of prompt-template so it can be more readily used by other modules. kg-extract-agent uses this framework.

* Migrate from setup.py to pyproject.toml (#440)

* Converted setup.py to pyproject.toml

* Modern package infrastructure as recommended by py docs

* Install missing build deps (#441)

* Install missing build deps (#442)

* Implement logging strategy (#444)

* Logging strategy and convert all prints() to logging invocations

* Fix/startup failure (#445)

* Fix loggin startup problems

* Fix logging startup problems (#446)

* Fix logging startup problems (#447)

* Fixed Mistral OCR to use current API (#448)

* Fixed Mistral OCR to use current API

* Added PDF decoder tests

* Fix Mistral OCR ident to be standard pdf-decoder (#450)

* Fix Mistral OCR ident to be standard pdf-decoder

* Correct test

* Schema structure refactor (#451)

* Write schema refactor spec

* Implemented schema refactor spec

* Structure data mvp (#452)

* Structured data tech spec

* Architecture principles

* New schemas

* Updated schemas and specs

* Object extractor

* Add .coveragerc

* New tests

* Cassandra object storage

* Trying to object extraction working, issues exist

* Validate librarian collection (#453)

* Fix token chunker, broken API invocation (#454)

* Fix token chunker, broken API invocation (#455)

* Knowledge load utility CLI (#456)

* Knowledge loader

* More tests

tests.manual/README.prompts

@@ -0,0 +1,27 @@
+
+test-prompt-... is tested with this prompt set...
+
+prompt-template \
+    -p pulsar://localhost:6650 \
+    --system-prompt 'You are a {{attitude}}, you are called {{name}}' \
+    --global-term \
+        'name=Craig' \
+        'attitude=LOUD, SHOUTY ANNOYING BOT' \
+    --prompt \
+        'question={{question}}' \
+        'french-question={{question}}' \
+        "analyze=Find the name and age in this text, and output a JSON structure containing just the name and age fields: {{description}}.  Don't add markup, just output the raw JSON object." \
+        "graph-query=Study the following knowledge graph, and then answer the question.\\n\nGraph:\\n{% for edge in knowledge %}({{edge.0}})-[{{edge.1}}]->({{edge.2}})\\n{%endfor%}\\nQuestion:\\n{{question}}" \
+        "extract-definition=Analyse the text provided, and then return a list of terms and definitions.  The output should be a JSON array, each item in the array is an object with fields 'term' and 'definition'.Don't add markup, just output the raw JSON object.  Here is the text:\\n{{text}}" \
+    --prompt-response-type \
+        'question=text' \
+        'analyze=json' \
+        'graph-query=text' \
+        'extract-definition=json' \
+    --prompt-term \
+        'question=name:Bonny' \
+        'french-question=attitude:French-speaking bot' \
+    --prompt-schema \
+        'analyze={ "type" : "object", "properties" : { "age": { "type" : "number" }, "name": { "type" : "string" } } }' \
+        'extract-definition={ "type": "array", "items": { "type": "object", "properties": { "term": { "type": "string" }, "definition": { "type": "string" } }, "required": [ "term", "definition" ] } }'
+    

tests.manual/query

@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+
+from trustgraph.graph_rag import GraphRag
+import sys
+
+query = " ".join(sys.argv[1:])
+
+gr = GraphRag(
+    verbose=True,
+    pulsar_host="pulsar://localhost:6650",
+    pr_request_queue="non-persistent://tg/request/prompt",
+    pr_response_queue="non-persistent://tg/response/prompt-response",
+)
+
+if query == "":
+    query="""This knowledge graph describes the Space Shuttle disaster.
+Present 20 facts which are present in the knowledge graph."""
+
+resp = gr.query(query)
+print(resp)
+

tests.manual/report-chunk-sizes

@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+
+"""
+Accepts entity/vector pairs and writes them to a Milvus store.
+"""
+
+from trustgraph.schema import Chunk
+from trustgraph.schema import chunk_ingest_queue
+from trustgraph.log_level import LogLevel
+from trustgraph.base import Consumer
+from threading import Thread, Lock
+import time
+
+module = "test-chunk-size"
+
+default_input_queue = chunk_ingest_queue
+default_subscriber = module
+default_store_uri = 'http://localhost:19530'
+
+class Processor(Consumer):
+
+    def __init__(self, **params):
+
+        input_queue = params.get("input_queue", default_input_queue)
+        subscriber = params.get("subscriber", default_subscriber)
+        width = params.get("width", 200)
+
+        super(Processor, self).__init__(
+            **params | {
+                "input_queue": input_queue,
+                "subscriber": subscriber,
+                "input_schema": Chunk,
+            }
+        )
+
+        self.sizes = {}
+        self.width = width
+        self.lock = Lock()
+
+        Thread(target=self.report).start()
+
+    def report(self):
+
+        while True:
+            time.sleep(1)
+
+            print()
+
+            with self.lock:
+                tot = 0
+                for i in range(0, 20000, self.width):
+                    k = (i, i + self.width)
+                    if k in self.sizes:
+                        print(f"{i:5d} ..{i+self.width:5d}: {self.sizes[k]}")
+                        tot += self.sizes[k]
+                print(f"{'Total':13s}: {tot}")
+
+                    
+                
+
+    def handle(self, msg):
+
+        v = msg.value()
+
+        chunk = v.chunk.decode("utf-8")
+
+        l = len(chunk)
+
+
+        low = int(l / self.width) * self.width
+        high = low + self.width
+        key = (low, high)
+
+        with self.lock:
+
+            if key not in self.sizes:
+                self.sizes[key] = 0
+
+            self.sizes[key] += 1
+
+    @staticmethod
+    def add_args(parser):
+
+        Consumer.add_args(
+            parser, default_input_queue, default_subscriber,
+        )
+
+        parser.add_argument(
+            '--width',
+            type=int,
+            default=200,
+            help=f'Histogram width (default: 200)',
+        )
+
+def run():
+
+    Processor.start(module, __doc__)
+
+run()
+

tests.manual/test-agent

@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+
+import json
+import textwrap
+from trustgraph.clients.agent_client import AgentClient
+
+def wrap(text, width=75):
+
+    if text is None: text = "n/a"
+
+    out = textwrap.wrap(
+        text, width=width
+    )
+    return "\n".join(out)
+
+def output(text, prefix="> ", width=78):
+
+    out = textwrap.indent(
+        text, prefix=prefix
+    )
+    print(out)
+
+p = AgentClient(
+    pulsar_host="pulsar://pulsar:6650",
+    input_queue = "non-persistent://tg/request/agent:0000",
+    output_queue = "non-persistent://tg/response/agent:0000",
+)
+
+q = "How many cats does Mark have? Calculate that number raised to 0.4 power.  Is that number lower than the numeric part of the mission identifier of the Space Shuttle Challenger on its last mission?  If so, give me an apple pie recipe, otherwise return a poem about cheese."
+
+output(wrap(q), "\U00002753 ")
+print()
+
+def think(x):
+    output(wrap(x), "\U0001f914 ")
+    print()
+
+def observe(x):
+    output(wrap(x), "\U0001f4a1 ")
+    print()
+
+resp = p.request(
+    question=q, think=think, observe=observe,
+)
+
+output(resp, "\U0001f4ac ")
+print()
+

tests.manual/test-config

@@ -0,0 +1,2 @@
+#!/usr/bin/env python3
+

tests.manual/test-doc-embeddings

@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.document_embeddings_client import DocumentEmbeddingsClient
+from trustgraph.clients.embeddings_client import EmbeddingsClient
+
+ec = EmbeddingsClient(pulsar_host="pulsar://localhost:6650")
+
+vectors = ec.request("What caused the space shuttle to explode?")
+
+print(vectors)
+
+llm = DocumentEmbeddingsClient(pulsar_host="pulsar://localhost:6650")
+
+limit=10
+
+resp = llm.request(vectors, limit)
+
+print("Response...")
+for val in resp:
+    print(val)
+

tests.manual/test-doc-prompt

@@ -0,0 +1,25 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+docs = [
+    "In our house there is a big cat and a small cat.",
+    "The small cat is black.",
+    "The big cat is called Fred.",
+    "The orange stripey cat is big.",
+    "The black cat pounces on the big cat.",
+    "The black cat is called Hope."
+]
+
+query="What is the name of the cat who pounces on Fred?  Provide a full explanation."
+
+resp = p.request_document_prompt(
+    query=query,
+    documents=docs,
+)
+
+print(resp)
+

tests.manual/test-doc-rag

@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.document_rag_client import DocumentRagClient
+
+rag = DocumentRagClient(
+    pulsar_host="pulsar://localhost:6650",
+    subscriber="test1",
+    input_queue = "non-persistent://tg/request/document-rag:default",
+    output_queue = "non-persistent://tg/response/document-rag:default",
+)
+
+query="""
+What was the cause of the space shuttle disaster?"""
+
+resp = rag.request(query)
+
+print(resp)
+

tests.manual/test-embeddings

@@ -0,0 +1,18 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.embeddings_client import EmbeddingsClient
+
+embed = EmbeddingsClient(
+    pulsar_host="pulsar://pulsar:6650",
+    input_queue="non-persistent://tg/request/embeddings:default",
+    output_queue="non-persistent://tg/response/embeddings:default",
+    subscriber="test1",
+)
+
+prompt="Write a funny limerick about a llama"
+
+resp = embed.request(prompt)
+
+print(resp)
+

tests.manual/test-flow

@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+
+import requests
+
+url = "http://localhost:8088/"
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "list-classes",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "get-class",
+        "class-name": "default",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "put-class",
+        "class-name": "bunch",
+        "class-definition": "{}",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "get-class",
+        "class-name": "bunch",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "list-classes",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "delete-class",
+        "class-name": "bunch",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "list-classes",
+    }
+)
+
+print(resp)
+print(resp.text)
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "list-flows",
+    }
+)
+
+print(resp)
+print(resp.text)

tests.manual/test-flow-get-class

@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+
+import requests
+
+url = "http://localhost:8088/"
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "get-class",
+        "class-name": "default",
+    }
+)
+
+resp = resp.json()
+
+print(resp["class-definition"])
+
+

tests.manual/test-flow-start-flow

@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+
+import requests
+import json
+
+url = "http://localhost:8088/"
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "start-flow",
+        "flow-id": "0003",
+        "class-name": "default",
+    }
+)
+
+print(resp)
+print(resp.text)
+resp = resp.json()
+
+
+print(resp)
+

tests.manual/test-flow-stop-flow

@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import requests
+import json
+
+url = "http://localhost:8088/"
+
+resp = requests.post(
+    f"{url}/api/v1/flow",
+    json={
+        "operation": "stop-flow",
+        "flow-id": "0003",
+    }
+)
+
+print(resp)
+print(resp.text)
+resp = resp.json()
+
+
+print(resp)
+

tests.manual/test-get-config

@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.config_client import ConfigClient
+
+cli = ConfigClient(pulsar_host="pulsar://localhost:6650")
+
+resp = cli.request_config()
+
+print(resp)
+

tests.manual/test-graph-embeddings

@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.graph_embeddings_client import GraphEmbeddingsClient
+from trustgraph.clients.embeddings_client import EmbeddingsClient
+
+ec = EmbeddingsClient(pulsar_host="pulsar://localhost:6650")
+
+vectors = ec.request("What caused the space shuttle to explode?")
+
+print(vectors)
+
+llm = GraphEmbeddingsClient(pulsar_host="pulsar://localhost:6650")
+
+limit=10
+
+resp = llm.request(vectors, limit)
+
+print("Response...")
+for val in resp:
+    print(val.value)
+

tests.manual/test-graph-rag

@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.graph_rag_client import GraphRagClient
+
+rag = GraphRagClient(
+    pulsar_host="pulsar://localhost:6650",
+    subscriber="test1",
+    input_queue = "non-persistent://tg/request/graph-rag:default",
+    output_queue = "non-persistent://tg/response/graph-rag:default",
+)
+
+#query="""
+#This knowledge graph describes the Space Shuttle disaster.
+#Present 20 facts which are present in the knowledge graph."""
+
+query = "How many cats does Mark have?"
+
+resp = rag.request(query)
+
+print(resp)
+

tests.manual/test-graph-rag2

@@ -0,0 +1,14 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.graph_rag_client import GraphRagClient
+
+rag = GraphRagClient(pulsar_host="pulsar://localhost:6650")
+
+query="""List 20 key points to describe the research that led to the discovery of Leo VI.
+"""
+
+resp = rag.request(query)
+
+print(resp)
+

tests.manual/test-lang-definition

@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+chunk = """I noticed a cat in my garden.  It is a four-legged animal
+which is a mammal and can be tame or wild.  I wonder if it will be friends
+with me.  I think the cat's name is Fred and it has 4 legs.
+
+A cat is a small mammal.
+
+A grapefruit is a citrus fruit.
+
+"""
+
+resp = p.request_definitions(
+    chunk=chunk,
+)
+
+for d in resp:
+    print(d.name, ":", d.definition)
+

tests.manual/test-lang-kg-prompt

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+facts = [
+      ("accident", "evoked", "a wide range of deeply felt public responses"),
+      ("Space Shuttle concept", "had", "genesis"),
+      ("Commission", "had", "a mandate to develop recommendations for corrective or other action based upon the Commission's findings and determinations"),
+      ("Commission", "established", "teams of persons"),
+      ("Space Shuttle Challenger", "http://www.w3.org/2004/02/skos/core#definition", "A space shuttle that was destroyed in an accident during mission 51-L."),
+      ("The mid fuselage", "contains", "the payload bay"),
+      ("Volume I", "contains", "Chapter IX"),
+      ("accident", "resulted in", "firm national resolve that those men and women be forever enshrined in the annals of American heroes"),
+      ("Volume I", "contains", "Chapter IV"),
+      ("Volume I", "contains", "Appendix A"),
+      ("Volume I", "contains", "Appendix B"),
+      ("Volume I", "contains", "The Staff"),
+      ("Commission", "required", "detailed investigation"),
+      ("Commission", "focused", "safety aspects of future flights"),
+      ("Commission", "http://www.w3.org/2004/02/skos/core#definition", "An independent group appointed to investigate the Space Shuttle Challenger accident."),
+      ("Commission", "moved forward with", "its investigation"),
+      ("President", "appointed", "an independent Commission"),
+      ("accident", "interrupted", "one of the most productive engineering, scientific and exploratory programs in history"),
+      ("Volume I", "contains", "Preface"),
+      ("Commission", "believes", "investigation"),
+      ("Volume I", "contains", "Chapter I"),
+      ("President", "was moved and troubled", "by this accident in a very personal way"),
+      ("PRESIDENTIAL COMMISSION", "Report to", "President"),
+      ("Volume I", "contains", "Chapter VI"),
+      ("Commission", "held", "public hearings dealing with the facts leading up to the accident"),
+      ("Volume I", "http://www.w3.org/2004/02/skos/core#definition", "The first volume of a multi-volume publication."),
+      ("Space Shuttle Challenger", "was involved in", "an accident"),
+      ("Volume I", "contains", "Chapter VII"),
+      ("Volume I", "contains", "Chapter II"),
+      ("Volume I", "contains", "Chapter V"),
+      ("Commission", "believes", "its investigation and report have been responsive to the request of the President and hopes that they will serve the best interests of the nation in restoring the United States space program to its preeminent position in the world"),
+      ("Commission", "supported", "panels"),
+      ("Volume I", "contains", "Chapter VIII"),
+      ("NASA", "cooperated", "Commission"),
+      ("liquid oxygen tank", "contains", "oxidizer"),
+      ("President", "http://www.w3.org/2004/02/skos/core#definition", "The head of state of the United States."),
+      ("Volume I", "contains", "Chapter III"),
+      ("Apollo lunar landing spacecraft", "had", "not yet flown"),
+      ("Commission", "construe", "mandate"),
+      ("accident", "became", "a milestone on the way to achieving the full potential that space offers to mankind"),
+      ("Volume I", "contains", "The Commission"),
+      ("Commission", "focused", "attention"),
+      ("Commission", "learned", "lessons"),
+      ("Commission", "required", "interfere with or supersede Congress"),
+      ("Commission", "was made up of", "persons not connected with the mission"),
+      ("Commission", "required", "review budgetary matters"),
+      ("Space Shuttle", "became", "focus of NASA's near-term future"),
+      ("Volume I", "contains", "Appendix C"),
+      ("accident", "caused", "grief and sadness for the loss of seven brave members of the crew"),
+      ("Commission", "http://www.w3.org/2004/02/skos/core#definition", "A group established to investigate the space shuttle accident"),
+      ("Volume I", "contains", "Appendix D"),
+      ("Commission", "had", "a mandate to review the circumstances surrounding the accident to establish the probable cause or causes of the accident"),
+      ("Volume I", "contains", "Recommendations")
+]
+
+query="Present 20 facts which are present in the knowledge graph."
+
+resp = p.request_kg_prompt(
+    query=query,
+    kg=facts,
+)
+
+print(resp)
+

tests.manual/test-lang-relationships

@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+chunk = """I noticed a cat in my garden.  It is a four-legged animal
+which is a mammal and can be tame or wild.  I wonder if it will be friends
+with me.  I think the cat's name is Fred and it has 4 legs"""
+
+resp = p.request_relationships(
+    chunk=chunk,
+)
+
+for d in resp:
+    print(d.s)
+    print("  ", d.p)
+    print("  ", d.o)
+    print("  ", d.o_entity)
+

tests.manual/test-lang-topics

@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+chunk = """I noticed a cat in my garden.  It is a four-legged animal
+which is a mammal and can be tame or wild.  I wonder if it will be friends
+with me.  I think the cat's name is Fred and it has 4 legs"""
+
+resp = p.request_topics(
+    chunk=chunk,
+)
+
+for d in resp:
+    print(d.topic)
+    print("  ", d.definition)
+

tests.manual/test-llm

@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.llm_client import LlmClient
+
+llm = LlmClient(
+    pulsar_host="pulsar://pulsar:6650",
+    input_queue="non-persistent://tg/request/text-completion:default",
+    output_queue="non-persistent://tg/response/text-completion:default",
+    subscriber="test1",
+)
+
+system = "You are a lovely assistant."
+prompt="what is 2 + 2 == 5"
+
+resp = llm.request(system, prompt)
+
+print(resp)
+

tests.manual/test-llm2

@@ -0,0 +1,15 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.llm_client import LlmClient
+
+llm = LlmClient(pulsar_host="pulsar://localhost:6650")
+
+prompt="What is 2 + 12?"
+
+try:
+    resp = llm.request(prompt)
+    print(resp)
+except Exception as e:
+    print(f"{e.__class__.__name__}: {e}")
+

tests.manual/test-llm3

@@ -0,0 +1,15 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.llm_client import LlmClient
+
+llm = LlmClient(pulsar_host="pulsar://localhost:6650")
+
+prompt="What is 2 + 12?"
+
+try:
+    resp = llm.request(prompt)
+    print(resp)
+except Exception as e:
+    print(f"{e.__class__.__name__}: {e}")
+

tests.manual/test-load-pdf

@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+
+import pulsar
+from pulsar.schema import JsonSchema
+import base64
+
+from trustgraph.schema import Document, Metadata
+
+client = pulsar.Client("pulsar://localhost:6650", listener_name="localhost")
+
+prod = client.create_producer(
+    topic="persistent://tg/flow/document-load:0000",
+    schema=JsonSchema(Document),
+    chunking_enabled=True,
+)
+
+path = "../sources/Challenger-Report-Vol1.pdf"
+
+with open(path, "rb") as f:
+    blob = base64.b64encode(f.read()).decode("utf-8")
+
+message = Document(
+    metadata = Metadata(
+        id = "00001",
+        metadata = [],
+        user="trustgraph",
+        collection="default",
+    ),
+    data=blob
+)    
+
+prod.send(message)
+
+prod.close()
+client.close()
+

tests.manual/test-load-text

@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+
+import pulsar
+from pulsar.schema import JsonSchema
+import base64
+
+from trustgraph.schema import TextDocument, Metadata
+
+client = pulsar.Client("pulsar://localhost:6650", listener_name="localhost")
+
+prod = client.create_producer(
+    topic="persistent://tg/flow/text-document-load:0000",
+    schema=JsonSchema(TextDocument),
+    chunking_enabled=True,
+)
+
+path = "../trustgraph/docs/README.cats"
+
+with open(path, "r") as f:
+#    blob = base64.b64encode(f.read()).decode("utf-8")
+    blob = f.read()
+
+message = TextDocument(
+    metadata = Metadata(
+        id = "00001",
+        metadata = [],
+        user="trustgraph",
+        collection="default",
+    ),
+    text=blob
+)    
+
+prod.send(message)
+
+prod.close()
+client.close()
+

tests.manual/test-milvus

@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+
+from langchain_huggingface import HuggingFaceEmbeddings
+
+from trustgraph.direct.milvus import TripleVectors
+
+client = TripleVectors()
+
+embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
+
+text="""A cat is a small animal.  A dog is a large animal.
+Cats say miaow.  Dogs go woof.
+"""
+
+embeds = embeddings.embed_documents([text])[0]
+
+text2="""If you couldn't download the model due to network issues, as a walkaround, you can use random vectors to represent the text and still finish the example. Just note that the search result won't reflect semantic similarity as the vectors are fake ones.
+"""
+
+embeds2 = embeddings.embed_documents([text2])[0]
+
+client.insert(embeds, "animals")
+client.insert(embeds, "vectors")
+
+query="""What noise does a cat make?"""
+
+qembeds = embeddings.embed_documents([query])[0]
+
+res = client.search(
+    qembeds,
+    limit=2
+)
+
+print(res)
+

tests.manual/test-prompt-analyze

@@ -0,0 +1,18 @@
+#!/usr/bin/env python3
+
+import json
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+description = """Fred is a 4-legged cat who is 12 years old"""
+
+resp = p.request(
+    id="analyze",
+    terms = {
+        "description": description,
+    }
+)
+
+print(json.dumps(resp, indent=4))
+

tests.manual/test-prompt-extraction

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+
+import json
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(
+    pulsar_host="pulsar://localhost:6650",
+    input_queue="non-persistent://tg/request/prompt:default",
+    output_queue="non-persistent://tg/response/prompt:default",
+    subscriber="test1",
+)
+
+chunk="""
+ The Space Shuttle was a reusable spacecraft that transported astronauts and cargo to and from Earth's orbit. It was designed to launch like a rocket, maneuver in orbit like a spacecraft, and land like an airplane. The Space Shuttle was NASA's space transportation system and was used for many purposes, including: 
+
+    Carrying astronauts
+    The Space Shuttle could carry up to seven astronauts at a time. 
+
+Launching, recovering, and repairing satellites
+The Space Shuttle could launch satellites into orbit, recover them, and repair them. 
+Building the International Space Station
+The Space Shuttle carried large parts into space to build the International Space Station. 
+Conducting research
+Astronauts conducted experiments in the Space Shuttle, which was like a science lab in space. 
+
+The Space Shuttle was retired in 2011 after the Columbia accident in 2003. The Columbia Accident Investigation Board report found that the Space Shuttle was unsafe and expensive to make safe. 
+Here are some other facts about the Space Shuttle: 
+
+    The Space Shuttle was 184 ft tall and had a diameter of 29 ft. 
+
+The Space Shuttle had a mass of 4,480,000 lb. 
+The Space Shuttle's first flight was on April 12, 1981. 
+The Space Shuttle's last mission was in 2011. 
+"""
+
+q = "Tell me some facts in the knowledge graph"
+
+resp = p.request(
+    id="extract-definitions",
+    variables = {
+        "text": chunk,
+    }
+)
+
+print(resp)
+
+for fact in resp:
+    print(fact["entity"], "::")
+    print(fact["definition"])
+    print()
+

tests.manual/test-prompt-french-question

@@ -0,0 +1,18 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+question = """What is the square root of 16?"""
+
+resp = p.request(
+    id="french-question",
+    terms = {
+        "question": question
+    }
+)
+
+print(resp)
+

tests.manual/test-prompt-knowledge

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+
+import json
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+knowledge = [
+      ("accident", "evoked", "a wide range of deeply felt public responses"),
+      ("Space Shuttle concept", "had", "genesis"),
+      ("Commission", "had", "a mandate to develop recommendations for corrective or other action based upon the Commission's findings and determinations"),
+      ("Commission", "established", "teams of persons"),
+      ("Space Shuttle Challenger", "http://www.w3.org/2004/02/skos/core#definition", "A space shuttle that was destroyed in an accident during mission 51-L."),
+      ("The mid fuselage", "contains", "the payload bay"),
+      ("Volume I", "contains", "Chapter IX"),
+      ("accident", "resulted in", "firm national resolve that those men and women be forever enshrined in the annals of American heroes"),
+      ("Volume I", "contains", "Chapter VII"),
+      ("Volume I", "contains", "Chapter II"),
+      ("Volume I", "contains", "Chapter V"),
+      ("Commission", "believes", "its investigation and report have been responsive to the request of the President and hopes that they will serve the best interests of the nation in restoring the United States space program to its preeminent position in the world"),
+      ("Commission", "construe", "mandate"),
+      ("accident", "became", "a milestone on the way to achieving the full potential that space offers to mankind"),
+      ("Volume I", "contains", "The Commission"),
+      ("Commission", "http://www.w3.org/2004/02/skos/core#definition", "A group established to investigate the space shuttle accident"),
+      ("Volume I", "contains", "Appendix D"),
+      ("Commission", "had", "a mandate to review the circumstances surrounding the accident to establish the probable cause or causes of the accident"),
+      ("Volume I", "contains", "Recommendations")
+]
+
+q = "Tell me some facts in the knowledge graph"
+
+resp = p.request(
+    id="graph-query",
+    terms = {
+        "name": "Jayney",
+        "knowledge": knowledge,
+        "question": q
+    }
+)
+
+print(resp)
+
+
+

tests.manual/test-prompt-question

@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(
+    pulsar_host="pulsar://localhost:6650",
+    input_queue="non-persistent://tg/request/prompt:default",
+    output_queue="non-persistent://tg/response/prompt:default",
+    subscriber="test1",
+)
+
+question = """What is the square root of 16?"""
+
+resp = p.request(
+    id="question",
+    variables = {
+        "question": question
+    }
+)
+
+print(resp)
+

tests.manual/test-prompt-spanish-question

@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+question = """What is the square root of 16?"""
+
+resp = p.request(
+    id="question",
+    terms = {
+        "question": question,
+        "attitude": "Spanish-speaking bot"
+    }
+)
+
+print(resp)
+

tests.manual/test-rows-prompt

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.prompt_client import PromptClient
+from trustgraph.objects.object import Schema
+from trustgraph.objects.field import Field, FieldType
+
+schema = Schema(
+    name="actors",
+    description="actors in this story",
+    fields=[
+        Field(
+            name="name", type=FieldType.STRING,
+            description="Name of the animal or person in the story"
+        ),
+        Field(
+            name="legs", type=FieldType.INT,
+            description="Number of legs of the animal or person"
+        ),
+        Field(
+            name="notes", type=FieldType.STRING,
+            description="Additional notes or observations about this animal or person"
+        ),
+    ]
+)
+
+chunk = """I noticed a cat in my garden.  It is a four-legged animal
+which is a mammal and can be tame or wild.  I wonder if it will be friends
+with me?  I think the cat's name is Fred and it has 4 legs.
+There is also a dog barking outside.  The dog has 4 legs also.
+The dog comes to my call when I shout "Come here, Bernard".
+
+I am also standing in the garden, my name is Steve and I have 2 legs.
+
+My friend Clifford is coming to visit shortly, he has 3 legs due to
+a freak accident at birth.
+"""
+
+p = PromptClient(pulsar_host="pulsar://localhost:6650")
+
+resp = p.request_rows(
+    schema=schema,
+    chunk=chunk,
+)
+
+for d in resp:
+    print(f"Name: {d['name']}")
+    print(f"  No. of legs: {d['legs']}")
+    print(f"  Notes: {d['notes']}")
+    print()
+

tests.manual/test-run-extract-row

@@ -0,0 +1,13 @@
+
+scripts/object-extract-row \
+    -p pulsar://localhost:6650 \
+    --field 'name:string:100:pri:Name of the person in the story' \
+    --field 'job:string:100::Job title or role' \
+    --field 'date:string:20::Date entered into role if known' \
+    --field 'supervisor:string:100::Supervisor or manager of this person, if known' \
+    --field 'location:string:100::Main base or location of work, if known' \
+    --field 'notes:string:1000::Additional notes or observations about this animal or person' \
+    --no-metrics \
+    --name actors \
+    --description 'Relevant people'
+

tests.manual/test-triples

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+
+import pulsar
+from trustgraph.clients.triples_query_client import TriplesQueryClient
+
+tq = TriplesQueryClient(
+    pulsar_host="pulsar://localhost:6650",
+)
+
+e = "http://trustgraph.ai/e/shuttle"
+
+limit=3
+
+def dump(resp):
+    print("Response...")
+    for t in resp:
+        print(t.s.value, t.p.value, t.o.value)
+
+print("-- * ---------------------------")
+
+resp = tq.request(None, None, None, limit)
+dump(resp)
+
+print("-- s ---------------------------")
+
+resp = tq.request("http://trustgraph.ai/e/shuttle", None, None, limit)
+dump(resp)
+
+print("-- p ---------------------------")
+
+resp = tq.request(None, "http://trustgraph.ai/e/landed", None, limit)
+dump(resp)
+
+print("-- o ---------------------------")
+
+resp = tq.request(None, None, "President", limit)
+dump(resp)
+
+print("-- sp ---------------------------")
+
+resp = tq.request(
+    "http://trustgraph.ai/e/shuttle", "http://trustgraph.ai/e/landed", None,
+    limit
+)
+dump(resp)
+
+print("-- so ---------------------------")
+
+resp = tq.request(
+    "http://trustgraph.ai/e/shuttle", None, "the tower",
+    limit
+)
+dump(resp)
+
+print("-- po ---------------------------")
+
+resp = tq.request(
+    None, "http://trustgraph.ai/e/landed",
+    "on the concrete runway at Kennedy Space Center",
+    limit
+)
+dump(resp)
+
+print("-- spo ---------------------------")
+
+resp = tq.request(
+    "http://trustgraph.ai/e/shuttle", "http://trustgraph.ai/e/landed",
+    "on the concrete runway at Kennedy Space Center",
+    limit
+)
+dump(resp)
+