trustgraph/Makefile
cybermaggedon 89be656990
Release/v1.2 (#457)
* Bump setup.py versions for 1.1

* PoC MCP server (#419)

* Very initial MCP server PoC for TrustGraph

* Put service on port 8000

* Add MCP container and packages to buildout

* Update docs for API/CLI changes in 1.0 (#421)

* Update some API basics for the 0.23/1.0 API change

* Add MCP container push (#425)

* Add command args to the MCP server (#426)

* Host and port parameters

* Added websocket arg

* More docs

* MCP client support (#427)

- MCP client service
- Tool request/response schema
- API gateway support for mcp-tool
- Message translation for tool request & response
- Make mcp-tool using configuration service for information
  about where the MCP services are.

* Feature/react call mcp (#428)

Key Features

  - MCP Tool Integration: Added core MCP tool support with ToolClientSpec and ToolClient classes
  - API Enhancement: New mcp_tool method for flow-specific tool invocation
  - CLI Tooling: New tg-invoke-mcp-tool command for testing MCP integration
  - React Agent Enhancement: Fixed and improved multi-tool invocation capabilities
  - Tool Management: Enhanced CLI for tool configuration and management

Changes

  - Added MCP tool invocation to API with flow-specific integration
  - Implemented ToolClientSpec and ToolClient for tool call handling
  - Updated agent-manager-react to invoke MCP tools with configurable types
  - Enhanced CLI with new commands and improved help text
  - Added comprehensive documentation for new CLI commands
  - Improved tool configuration management

Testing

  - Added tg-invoke-mcp-tool CLI command for isolated MCP integration testing
  - Enhanced agent capability to invoke multiple tools simultaneously

* Test suite executed from CI pipeline (#433)

* Test strategy & test cases

* Unit tests

* Integration tests

* Extending test coverage (#434)

* Contract tests

* Testing embeedings

* Agent unit tests

* Knowledge pipeline tests

* Turn on contract tests

* Increase storage test coverage (#435)

* Fixing storage and adding tests

* PR pipeline only runs quick tests

* Empty configuration is returned as empty list, previously was not in response (#436)

* Update config util to take files as well as command-line text (#437)

* Updated CLI invocation and config model for tools and mcp (#438)

* Updated CLI invocation and config model for tools and mcp

* CLI anomalies

* Tweaked the MCP tool implementation for new model

* Update agent implementation to match the new model

* Fix agent tools, now all tested

* Fixed integration tests

* Fix MCP delete tool params

* Update Python deps to 1.2

* Update to enable knowledge extraction using the agent framework (#439)

* Implement KG extraction agent (kg-extract-agent)

* Using ReAct framework (agent-manager-react)
 
* ReAct manager had an issue when emitting JSON, which conflicts which ReAct manager's own JSON messages, so refactored ReAct manager to use traditional ReAct messages, non-JSON structure.
 
* Minor refactor to take the prompt template client out of prompt-template so it can be more readily used by other modules. kg-extract-agent uses this framework.

* Migrate from setup.py to pyproject.toml (#440)

* Converted setup.py to pyproject.toml

* Modern package infrastructure as recommended by py docs

* Install missing build deps (#441)

* Install missing build deps (#442)

* Implement logging strategy (#444)

* Logging strategy and convert all prints() to logging invocations

* Fix/startup failure (#445)

* Fix loggin startup problems

* Fix logging startup problems (#446)

* Fix logging startup problems (#447)

* Fixed Mistral OCR to use current API (#448)

* Fixed Mistral OCR to use current API

* Added PDF decoder tests

* Fix Mistral OCR ident to be standard pdf-decoder (#450)

* Fix Mistral OCR ident to be standard pdf-decoder

* Correct test

* Schema structure refactor (#451)

* Write schema refactor spec

* Implemented schema refactor spec

* Structure data mvp (#452)

* Structured data tech spec

* Architecture principles

* New schemas

* Updated schemas and specs

* Object extractor

* Add .coveragerc

* New tests

* Cassandra object storage

* Trying to object extraction working, issues exist

* Validate librarian collection (#453)

* Fix token chunker, broken API invocation (#454)

* Fix token chunker, broken API invocation (#455)

* Knowledge load utility CLI (#456)

* Knowledge loader

* More tests
2025-08-18 20:56:09 +01:00

154 lines
6 KiB
Makefile

# VERSION=$(shell git describe | sed 's/^v//')
VERSION=0.0.0
DOCKER=podman
all: container
# Not used
wheels:
pip3 wheel --no-deps --wheel-dir dist trustgraph/
pip3 wheel --no-deps --wheel-dir dist trustgraph-base/
pip3 wheel --no-deps --wheel-dir dist trustgraph-flow/
pip3 wheel --no-deps --wheel-dir dist trustgraph-vertexai/
pip3 wheel --no-deps --wheel-dir dist trustgraph-bedrock/
pip3 wheel --no-deps --wheel-dir dist trustgraph-embeddings-hf/
pip3 wheel --no-deps --wheel-dir dist trustgraph-cli/
pip3 wheel --no-deps --wheel-dir dist trustgraph-ocr/
pip3 wheel --no-deps --wheel-dir dist trustgraph-mcp/
packages: update-package-versions
rm -rf dist/
cd trustgraph && python -m build --sdist --outdir ../dist/
cd trustgraph-base && python -m build --sdist --outdir ../dist/
cd trustgraph-flow && python -m build --sdist --outdir ../dist/
cd trustgraph-vertexai && python -m build --sdist --outdir ../dist/
cd trustgraph-bedrock && python -m build --sdist --outdir ../dist/
cd trustgraph-embeddings-hf && python -m build --sdist --outdir ../dist/
cd trustgraph-cli && python -m build --sdist --outdir ../dist/
cd trustgraph-ocr && python -m build --sdist --outdir ../dist/
cd trustgraph-mcp && python -m build --sdist --outdir ../dist/
pypi-upload:
twine upload dist/*-${VERSION}.*
CONTAINER_BASE=docker.io/trustgraph
update-package-versions:
mkdir -p trustgraph-cli/trustgraph
mkdir -p trustgraph/trustgraph
echo __version__ = \"${VERSION}\" > trustgraph-base/trustgraph/base_version.py
echo __version__ = \"${VERSION}\" > trustgraph-flow/trustgraph/flow_version.py
echo __version__ = \"${VERSION}\" > trustgraph-vertexai/trustgraph/vertexai_version.py
echo __version__ = \"${VERSION}\" > trustgraph-bedrock/trustgraph/bedrock_version.py
echo __version__ = \"${VERSION}\" > trustgraph-embeddings-hf/trustgraph/embeddings_hf_version.py
echo __version__ = \"${VERSION}\" > trustgraph-cli/trustgraph/cli_version.py
echo __version__ = \"${VERSION}\" > trustgraph-ocr/trustgraph/ocr_version.py
echo __version__ = \"${VERSION}\" > trustgraph/trustgraph/trustgraph_version.py
echo __version__ = \"${VERSION}\" > trustgraph-mcp/trustgraph/mcp_version.py
container: update-package-versions
${DOCKER} build -f containers/Containerfile.base \
-t ${CONTAINER_BASE}/trustgraph-base:${VERSION} .
${DOCKER} build -f containers/Containerfile.flow \
-t ${CONTAINER_BASE}/trustgraph-flow:${VERSION} .
${DOCKER} build -f containers/Containerfile.bedrock \
-t ${CONTAINER_BASE}/trustgraph-bedrock:${VERSION} .
${DOCKER} build -f containers/Containerfile.vertexai \
-t ${CONTAINER_BASE}/trustgraph-vertexai:${VERSION} .
${DOCKER} build -f containers/Containerfile.hf \
-t ${CONTAINER_BASE}/trustgraph-hf:${VERSION} .
${DOCKER} build -f containers/Containerfile.ocr \
-t ${CONTAINER_BASE}/trustgraph-ocr:${VERSION} .
${DOCKER} build -f containers/Containerfile.mcp \
-t ${CONTAINER_BASE}/trustgraph-mcp:${VERSION} .
some-containers:
${DOCKER} build -f containers/Containerfile.base \
-t ${CONTAINER_BASE}/trustgraph-base:${VERSION} .
${DOCKER} build -f containers/Containerfile.flow \
-t ${CONTAINER_BASE}/trustgraph-flow:${VERSION} .
# ${DOCKER} build -f containers/Containerfile.mcp \
# -t ${CONTAINER_BASE}/trustgraph-mcp:${VERSION} .
# ${DOCKER} build -f containers/Containerfile.vertexai \
# -t ${CONTAINER_BASE}/trustgraph-vertexai:${VERSION} .
# ${DOCKER} build -f containers/Containerfile.bedrock \
# -t ${CONTAINER_BASE}/trustgraph-bedrock:${VERSION} .
basic-containers: update-package-versions
${DOCKER} build -f containers/Containerfile.base \
-t ${CONTAINER_BASE}/trustgraph-base:${VERSION} .
${DOCKER} build -f containers/Containerfile.flow \
-t ${CONTAINER_BASE}/trustgraph-flow:${VERSION} .
container.ocr:
${DOCKER} build -f containers/Containerfile.ocr \
-t ${CONTAINER_BASE}/trustgraph-ocr:${VERSION} .
push:
${DOCKER} push ${CONTAINER_BASE}/trustgraph-base:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-flow:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-bedrock:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-vertexai:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-hf:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-ocr:${VERSION}
${DOCKER} push ${CONTAINER_BASE}/trustgraph-mcp:${VERSION}
clean:
rm -rf wheels/
set-version:
echo '"${VERSION}"' > templates/values/version.jsonnet
TEMPLATES=azure bedrock claude cohere mix llamafile mistral ollama openai vertexai \
openai-neo4j storage
DCS=$(foreach template,${TEMPLATES},${template:%=tg-launch-%.yaml})
MODELS=azure bedrock claude cohere llamafile mistral ollama openai vertexai
GRAPHS=cassandra neo4j falkordb memgraph
# tg-launch-%.yaml: templates/%.jsonnet templates/components/version.jsonnet
# jsonnet -Jtemplates \
# -S ${@:tg-launch-%.yaml=templates/%.jsonnet} > $@
# VECTORDB=milvus
VECTORDB=qdrant
JSONNET_FLAGS=-J templates -J .
# Temporarily going back to how templates were built in 0.9 because this
# is going away in 0.11.
update-templates: update-dcs
JSON_TO_YAML=python -c 'import sys, yaml, json; j=json.loads(sys.stdin.read()); print(yaml.safe_dump(j))'
update-dcs: set-version
for graph in ${GRAPHS}; do \
cm=$${graph},pulsar,${VECTORDB},grafana; \
input=templates/opts-to-docker-compose.jsonnet; \
output=tg-storage-$${graph}.yaml; \
echo $${graph} '->' $${output}; \
jsonnet ${JSONNET_FLAGS} \
--ext-str options=$${cm} $${input} | \
${JSON_TO_YAML} > $${output}; \
done
for model in ${MODELS}; do \
for graph in ${GRAPHS}; do \
cm=$${graph},pulsar,${VECTORDB},embeddings-hf,graph-rag,grafana,trustgraph,$${model}; \
input=templates/opts-to-docker-compose.jsonnet; \
output=tg-launch-$${model}-$${graph}.yaml; \
echo $${model} + $${graph} '->' $${output}; \
jsonnet ${JSONNET_FLAGS} \
--ext-str options=$${cm} $${input} | \
${JSON_TO_YAML} > $${output}; \
done; \
done
docker-hub-login:
cat docker-token.txt | \
${DOCKER} login -u trustgraph --password-stdin registry-1.docker.io