Extending test coverage (#434)

* Contract tests

* Testing embeedings

* Agent unit tests

* Knowledge pipeline tests

* Turn on contract tests

tests/contract/README.md

@@ -0,0 +1,243 @@
+# Contract Tests for TrustGraph
+
+This directory contains contract tests that verify service interface contracts, message schemas, and API compatibility across the TrustGraph microservices architecture.
+
+## Overview
+
+Contract tests ensure that:
+- **Message schemas remain compatible** across service versions
+- **API interfaces stay stable** for consumers
+- **Service communication contracts** are maintained
+- **Schema evolution** doesn't break existing integrations
+
+## Test Categories
+
+### 1. Pulsar Message Schema Contracts (`test_message_contracts.py`)
+
+Tests the contracts for all Pulsar message schemas used in TrustGraph service communication.
+
+#### **Coverage:**
+- ✅ **Text Completion Messages**: `TextCompletionRequest` ↔ `TextCompletionResponse`
+- ✅ **Document RAG Messages**: `DocumentRagQuery` ↔ `DocumentRagResponse`
+- ✅ **Agent Messages**: `AgentRequest` ↔ `AgentResponse` ↔ `AgentStep`
+- ✅ **Graph Messages**: `Chunk` → `Triple` → `Triples` → `EntityContext`
+- ✅ **Common Messages**: `Metadata`, `Value`, `Error` schemas
+- ✅ **Message Routing**: Properties, correlation IDs, routing keys
+- ✅ **Schema Evolution**: Backward/forward compatibility testing
+- ✅ **Serialization**: Schema validation and data integrity
+
+#### **Key Features:**
+- **Schema Validation**: Ensures all message schemas accept valid data and reject invalid data
+- **Field Contracts**: Validates required vs optional fields and type constraints
+- **Nested Schema Support**: Tests complex schemas with embedded objects and arrays
+- **Routing Contracts**: Validates message properties and routing conventions
+- **Evolution Testing**: Backward compatibility and schema versioning support
+
+## Running Contract Tests
+
+### Run All Contract Tests
+```bash
+pytest tests/contract/ -m contract
+```
+
+### Run Specific Contract Test Categories
+```bash
+# Message schema contracts
+pytest tests/contract/test_message_contracts.py -v
+
+# Specific test class
+pytest tests/contract/test_message_contracts.py::TestTextCompletionMessageContracts -v
+
+# Schema evolution tests
+pytest tests/contract/test_message_contracts.py::TestSchemaEvolutionContracts -v
+```
+
+### Run with Coverage
+```bash
+pytest tests/contract/ -m contract --cov=trustgraph.schema --cov-report=html
+```
+
+## Contract Test Patterns
+
+### 1. Schema Validation Pattern
+```python
+@pytest.mark.contract
+def test_schema_contract(self, sample_message_data):
+    """Test that schema accepts valid data and rejects invalid data"""
+    # Arrange
+    valid_data = sample_message_data["SchemaName"]
+    
+    # Act & Assert
+    assert validate_schema_contract(SchemaClass, valid_data)
+    
+    # Test field constraints
+    instance = SchemaClass(**valid_data)
+    assert hasattr(instance, 'required_field')
+    assert isinstance(instance.required_field, expected_type)
+```
+
+### 2. Serialization Contract Pattern
+```python
+@pytest.mark.contract  
+def test_serialization_contract(self, sample_message_data):
+    """Test schema serialization/deserialization contracts"""
+    # Arrange
+    data = sample_message_data["SchemaName"]
+    
+    # Act & Assert
+    assert serialize_deserialize_test(SchemaClass, data)
+```
+
+### 3. Evolution Contract Pattern
+```python
+@pytest.mark.contract
+def test_backward_compatibility_contract(self, schema_evolution_data):
+    """Test that new schema versions accept old data formats"""
+    # Arrange
+    old_version_data = schema_evolution_data["SchemaName_v1"]
+    
+    # Act - Should work with current schema
+    instance = CurrentSchema(**old_version_data)
+    
+    # Assert - Required fields maintained
+    assert instance.required_field == expected_value
+```
+
+## Schema Registry
+
+The contract tests maintain a registry of all TrustGraph schemas:
+
+```python
+schema_registry = {
+    # Text Completion
+    "TextCompletionRequest": TextCompletionRequest,
+    "TextCompletionResponse": TextCompletionResponse,
+    
+    # Document RAG  
+    "DocumentRagQuery": DocumentRagQuery,
+    "DocumentRagResponse": DocumentRagResponse,
+    
+    # Agent
+    "AgentRequest": AgentRequest,
+    "AgentResponse": AgentResponse,
+    
+    # Graph/Knowledge
+    "Chunk": Chunk,
+    "Triple": Triple,
+    "Triples": Triples,
+    "Value": Value,
+    
+    # Common
+    "Metadata": Metadata,
+    "Error": Error,
+}
+```
+
+## Message Contract Specifications
+
+### Text Completion Service Contract
+```yaml
+TextCompletionRequest:
+  required_fields: [system, prompt]
+  field_types:
+    system: string
+    prompt: string
+
+TextCompletionResponse:
+  required_fields: [error, response, model]  
+  field_types:
+    error: Error | null
+    response: string | null
+    in_token: integer | null
+    out_token: integer | null
+    model: string
+```
+
+### Document RAG Service Contract
+```yaml
+DocumentRagQuery:
+  required_fields: [query, user, collection]
+  field_types:
+    query: string
+    user: string
+    collection: string
+    doc_limit: integer
+
+DocumentRagResponse:
+  required_fields: [error, response]
+  field_types:
+    error: Error | null
+    response: string | null
+```
+
+### Agent Service Contract
+```yaml
+AgentRequest:
+  required_fields: [question, history]
+  field_types:
+    question: string
+    plan: string
+    state: string
+    history: Array<AgentStep>
+
+AgentResponse:
+  required_fields: [error]
+  field_types:
+    answer: string | null
+    error: Error | null
+    thought: string | null
+    observation: string | null
+```
+
+## Best Practices
+
+### Contract Test Design
+1. **Test Both Valid and Invalid Data**: Ensure schemas accept valid data and reject invalid data
+2. **Verify Field Constraints**: Test type constraints, required vs optional fields
+3. **Test Nested Schemas**: Validate complex objects with embedded schemas
+4. **Test Array Fields**: Ensure array serialization maintains order and content
+5. **Test Optional Fields**: Verify optional field handling in serialization
+
+### Schema Evolution
+1. **Backward Compatibility**: New schema versions must accept old message formats
+2. **Required Field Stability**: Required fields should never become optional or be removed
+3. **Additive Changes**: New fields should be optional to maintain compatibility
+4. **Deprecation Strategy**: Plan deprecation path for schema changes
+
+### Error Handling
+1. **Error Schema Consistency**: All error responses use consistent Error schema
+2. **Error Type Contracts**: Error types follow naming conventions
+3. **Error Message Format**: Error messages provide actionable information
+
+## Adding New Contract Tests
+
+When adding new message schemas or modifying existing ones:
+
+1. **Add to Schema Registry**: Update `conftest.py` schema registry
+2. **Add Sample Data**: Create valid sample data in `conftest.py`
+3. **Create Contract Tests**: Follow existing patterns for validation
+4. **Test Evolution**: Add backward compatibility tests
+5. **Update Documentation**: Document schema contracts in this README
+
+## Integration with CI/CD
+
+Contract tests should be run:
+- **On every commit** to detect breaking changes early
+- **Before releases** to ensure API stability
+- **On schema changes** to validate compatibility
+- **In dependency updates** to catch breaking changes
+
+```bash
+# CI/CD pipeline command
+pytest tests/contract/ -m contract --junitxml=contract-test-results.xml
+```
+
+## Contract Test Results
+
+Contract tests provide:
+- ✅ **Schema Compatibility Reports**: Which schemas pass/fail validation
+- ✅ **Breaking Change Detection**: Identifies contract violations
+- ✅ **Evolution Validation**: Confirms backward compatibility
+- ✅ **Field Constraint Verification**: Validates data type contracts
+
+This ensures that TrustGraph services can evolve independently while maintaining stable, compatible interfaces for all service communication.