mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-04-26 08:56:21 +02:00
Release/v1.2 (#457)
* Bump setup.py versions for 1.1 * PoC MCP server (#419) * Very initial MCP server PoC for TrustGraph * Put service on port 8000 * Add MCP container and packages to buildout * Update docs for API/CLI changes in 1.0 (#421) * Update some API basics for the 0.23/1.0 API change * Add MCP container push (#425) * Add command args to the MCP server (#426) * Host and port parameters * Added websocket arg * More docs * MCP client support (#427) - MCP client service - Tool request/response schema - API gateway support for mcp-tool - Message translation for tool request & response - Make mcp-tool using configuration service for information about where the MCP services are. * Feature/react call mcp (#428) Key Features - MCP Tool Integration: Added core MCP tool support with ToolClientSpec and ToolClient classes - API Enhancement: New mcp_tool method for flow-specific tool invocation - CLI Tooling: New tg-invoke-mcp-tool command for testing MCP integration - React Agent Enhancement: Fixed and improved multi-tool invocation capabilities - Tool Management: Enhanced CLI for tool configuration and management Changes - Added MCP tool invocation to API with flow-specific integration - Implemented ToolClientSpec and ToolClient for tool call handling - Updated agent-manager-react to invoke MCP tools with configurable types - Enhanced CLI with new commands and improved help text - Added comprehensive documentation for new CLI commands - Improved tool configuration management Testing - Added tg-invoke-mcp-tool CLI command for isolated MCP integration testing - Enhanced agent capability to invoke multiple tools simultaneously * Test suite executed from CI pipeline (#433) * Test strategy & test cases * Unit tests * Integration tests * Extending test coverage (#434) * Contract tests * Testing embeedings * Agent unit tests * Knowledge pipeline tests * Turn on contract tests * Increase storage test coverage (#435) * Fixing storage and adding tests * PR pipeline only runs quick tests * Empty configuration is returned as empty list, previously was not in response (#436) * Update config util to take files as well as command-line text (#437) * Updated CLI invocation and config model for tools and mcp (#438) * Updated CLI invocation and config model for tools and mcp * CLI anomalies * Tweaked the MCP tool implementation for new model * Update agent implementation to match the new model * Fix agent tools, now all tested * Fixed integration tests * Fix MCP delete tool params * Update Python deps to 1.2 * Update to enable knowledge extraction using the agent framework (#439) * Implement KG extraction agent (kg-extract-agent) * Using ReAct framework (agent-manager-react) * ReAct manager had an issue when emitting JSON, which conflicts which ReAct manager's own JSON messages, so refactored ReAct manager to use traditional ReAct messages, non-JSON structure. * Minor refactor to take the prompt template client out of prompt-template so it can be more readily used by other modules. kg-extract-agent uses this framework. * Migrate from setup.py to pyproject.toml (#440) * Converted setup.py to pyproject.toml * Modern package infrastructure as recommended by py docs * Install missing build deps (#441) * Install missing build deps (#442) * Implement logging strategy (#444) * Logging strategy and convert all prints() to logging invocations * Fix/startup failure (#445) * Fix loggin startup problems * Fix logging startup problems (#446) * Fix logging startup problems (#447) * Fixed Mistral OCR to use current API (#448) * Fixed Mistral OCR to use current API * Added PDF decoder tests * Fix Mistral OCR ident to be standard pdf-decoder (#450) * Fix Mistral OCR ident to be standard pdf-decoder * Correct test * Schema structure refactor (#451) * Write schema refactor spec * Implemented schema refactor spec * Structure data mvp (#452) * Structured data tech spec * Architecture principles * New schemas * Updated schemas and specs * Object extractor * Add .coveragerc * New tests * Cassandra object storage * Trying to object extraction working, issues exist * Validate librarian collection (#453) * Fix token chunker, broken API invocation (#454) * Fix token chunker, broken API invocation (#455) * Knowledge load utility CLI (#456) * Knowledge loader * More tests
This commit is contained in:
cybermaggedon
GitHub
parent
c85ba197be
commit
89be656990
509 changed files with 49632 additions and 5159 deletions
243
tests/contract/README.md
Normal file
243
tests/contract/README.md
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
# Contract Tests for TrustGraph
|
||||
|
||||
This directory contains contract tests that verify service interface contracts, message schemas, and API compatibility across the TrustGraph microservices architecture.
|
||||
|
||||
## Overview
|
||||
|
||||
Contract tests ensure that:
|
||||
- **Message schemas remain compatible** across service versions
|
||||
- **API interfaces stay stable** for consumers
|
||||
- **Service communication contracts** are maintained
|
||||
- **Schema evolution** doesn't break existing integrations
|
||||
|
||||
## Test Categories
|
||||
|
||||
### 1. Pulsar Message Schema Contracts (`test_message_contracts.py`)
|
||||
|
||||
Tests the contracts for all Pulsar message schemas used in TrustGraph service communication.
|
||||
|
||||
#### **Coverage:**
|
||||
- ✅ **Text Completion Messages**: `TextCompletionRequest` ↔ `TextCompletionResponse`
|
||||
- ✅ **Document RAG Messages**: `DocumentRagQuery` ↔ `DocumentRagResponse`
|
||||
- ✅ **Agent Messages**: `AgentRequest` ↔ `AgentResponse` ↔ `AgentStep`
|
||||
- ✅ **Graph Messages**: `Chunk` → `Triple` → `Triples` → `EntityContext`
|
||||
- ✅ **Common Messages**: `Metadata`, `Value`, `Error` schemas
|
||||
- ✅ **Message Routing**: Properties, correlation IDs, routing keys
|
||||
- ✅ **Schema Evolution**: Backward/forward compatibility testing
|
||||
- ✅ **Serialization**: Schema validation and data integrity
|
||||
|
||||
#### **Key Features:**
|
||||
- **Schema Validation**: Ensures all message schemas accept valid data and reject invalid data
|
||||
- **Field Contracts**: Validates required vs optional fields and type constraints
|
||||
- **Nested Schema Support**: Tests complex schemas with embedded objects and arrays
|
||||
- **Routing Contracts**: Validates message properties and routing conventions
|
||||
- **Evolution Testing**: Backward compatibility and schema versioning support
|
||||
|
||||
## Running Contract Tests
|
||||
|
||||
### Run All Contract Tests
|
||||
```bash
|
||||
pytest tests/contract/ -m contract
|
||||
```
|
||||
|
||||
### Run Specific Contract Test Categories
|
||||
```bash
|
||||
# Message schema contracts
|
||||
pytest tests/contract/test_message_contracts.py -v
|
||||
|
||||
# Specific test class
|
||||
pytest tests/contract/test_message_contracts.py::TestTextCompletionMessageContracts -v
|
||||
|
||||
# Schema evolution tests
|
||||
pytest tests/contract/test_message_contracts.py::TestSchemaEvolutionContracts -v
|
||||
```
|
||||
|
||||
### Run with Coverage
|
||||
```bash
|
||||
pytest tests/contract/ -m contract --cov=trustgraph.schema --cov-report=html
|
||||
```
|
||||
|
||||
## Contract Test Patterns
|
||||
|
||||
### 1. Schema Validation Pattern
|
||||
```python
|
||||
@pytest.mark.contract
|
||||
def test_schema_contract(self, sample_message_data):
|
||||
"""Test that schema accepts valid data and rejects invalid data"""
|
||||
# Arrange
|
||||
valid_data = sample_message_data["SchemaName"]
|
||||
|
||||
# Act & Assert
|
||||
assert validate_schema_contract(SchemaClass, valid_data)
|
||||
|
||||
# Test field constraints
|
||||
instance = SchemaClass(**valid_data)
|
||||
assert hasattr(instance, 'required_field')
|
||||
assert isinstance(instance.required_field, expected_type)
|
||||
```
|
||||
|
||||
### 2. Serialization Contract Pattern
|
||||
```python
|
||||
@pytest.mark.contract
|
||||
def test_serialization_contract(self, sample_message_data):
|
||||
"""Test schema serialization/deserialization contracts"""
|
||||
# Arrange
|
||||
data = sample_message_data["SchemaName"]
|
||||
|
||||
# Act & Assert
|
||||
assert serialize_deserialize_test(SchemaClass, data)
|
||||
```
|
||||
|
||||
### 3. Evolution Contract Pattern
|
||||
```python
|
||||
@pytest.mark.contract
|
||||
def test_backward_compatibility_contract(self, schema_evolution_data):
|
||||
"""Test that new schema versions accept old data formats"""
|
||||
# Arrange
|
||||
old_version_data = schema_evolution_data["SchemaName_v1"]
|
||||
|
||||
# Act - Should work with current schema
|
||||
instance = CurrentSchema(**old_version_data)
|
||||
|
||||
# Assert - Required fields maintained
|
||||
assert instance.required_field == expected_value
|
||||
```
|
||||
|
||||
## Schema Registry
|
||||
|
||||
The contract tests maintain a registry of all TrustGraph schemas:
|
||||
|
||||
```python
|
||||
schema_registry = {
|
||||
# Text Completion
|
||||
"TextCompletionRequest": TextCompletionRequest,
|
||||
"TextCompletionResponse": TextCompletionResponse,
|
||||
|
||||
# Document RAG
|
||||
"DocumentRagQuery": DocumentRagQuery,
|
||||
"DocumentRagResponse": DocumentRagResponse,
|
||||
|
||||
# Agent
|
||||
"AgentRequest": AgentRequest,
|
||||
"AgentResponse": AgentResponse,
|
||||
|
||||
# Graph/Knowledge
|
||||
"Chunk": Chunk,
|
||||
"Triple": Triple,
|
||||
"Triples": Triples,
|
||||
"Value": Value,
|
||||
|
||||
# Common
|
||||
"Metadata": Metadata,
|
||||
"Error": Error,
|
||||
}
|
||||
```
|
||||
|
||||
## Message Contract Specifications
|
||||
|
||||
### Text Completion Service Contract
|
||||
```yaml
|
||||
TextCompletionRequest:
|
||||
required_fields: [system, prompt]
|
||||
field_types:
|
||||
system: string
|
||||
prompt: string
|
||||
|
||||
TextCompletionResponse:
|
||||
required_fields: [error, response, model]
|
||||
field_types:
|
||||
error: Error | null
|
||||
response: string | null
|
||||
in_token: integer | null
|
||||
out_token: integer | null
|
||||
model: string
|
||||
```
|
||||
|
||||
### Document RAG Service Contract
|
||||
```yaml
|
||||
DocumentRagQuery:
|
||||
required_fields: [query, user, collection]
|
||||
field_types:
|
||||
query: string
|
||||
user: string
|
||||
collection: string
|
||||
doc_limit: integer
|
||||
|
||||
DocumentRagResponse:
|
||||
required_fields: [error, response]
|
||||
field_types:
|
||||
error: Error | null
|
||||
response: string | null
|
||||
```
|
||||
|
||||
### Agent Service Contract
|
||||
```yaml
|
||||
AgentRequest:
|
||||
required_fields: [question, history]
|
||||
field_types:
|
||||
question: string
|
||||
plan: string
|
||||
state: string
|
||||
history: Array<AgentStep>
|
||||
|
||||
AgentResponse:
|
||||
required_fields: [error]
|
||||
field_types:
|
||||
answer: string | null
|
||||
error: Error | null
|
||||
thought: string | null
|
||||
observation: string | null
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Contract Test Design
|
||||
1. **Test Both Valid and Invalid Data**: Ensure schemas accept valid data and reject invalid data
|
||||
2. **Verify Field Constraints**: Test type constraints, required vs optional fields
|
||||
3. **Test Nested Schemas**: Validate complex objects with embedded schemas
|
||||
4. **Test Array Fields**: Ensure array serialization maintains order and content
|
||||
5. **Test Optional Fields**: Verify optional field handling in serialization
|
||||
|
||||
### Schema Evolution
|
||||
1. **Backward Compatibility**: New schema versions must accept old message formats
|
||||
2. **Required Field Stability**: Required fields should never become optional or be removed
|
||||
3. **Additive Changes**: New fields should be optional to maintain compatibility
|
||||
4. **Deprecation Strategy**: Plan deprecation path for schema changes
|
||||
|
||||
### Error Handling
|
||||
1. **Error Schema Consistency**: All error responses use consistent Error schema
|
||||
2. **Error Type Contracts**: Error types follow naming conventions
|
||||
3. **Error Message Format**: Error messages provide actionable information
|
||||
|
||||
## Adding New Contract Tests
|
||||
|
||||
When adding new message schemas or modifying existing ones:
|
||||
|
||||
1. **Add to Schema Registry**: Update `conftest.py` schema registry
|
||||
2. **Add Sample Data**: Create valid sample data in `conftest.py`
|
||||
3. **Create Contract Tests**: Follow existing patterns for validation
|
||||
4. **Test Evolution**: Add backward compatibility tests
|
||||
5. **Update Documentation**: Document schema contracts in this README
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
Contract tests should be run:
|
||||
- **On every commit** to detect breaking changes early
|
||||
- **Before releases** to ensure API stability
|
||||
- **On schema changes** to validate compatibility
|
||||
- **In dependency updates** to catch breaking changes
|
||||
|
||||
```bash
|
||||
# CI/CD pipeline command
|
||||
pytest tests/contract/ -m contract --junitxml=contract-test-results.xml
|
||||
```
|
||||
|
||||
## Contract Test Results
|
||||
|
||||
Contract tests provide:
|
||||
- ✅ **Schema Compatibility Reports**: Which schemas pass/fail validation
|
||||
- ✅ **Breaking Change Detection**: Identifies contract violations
|
||||
- ✅ **Evolution Validation**: Confirms backward compatibility
|
||||
- ✅ **Field Constraint Verification**: Validates data type contracts
|
||||
|
||||
This ensures that TrustGraph services can evolve independently while maintaining stable, compatible interfaces for all service communication.
|
||||
Loading…
Add table
Add a link
Reference in a new issue