mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-14 07:42:11 +02:00
9136526863
Thread existing JSON schemas from prompt definitions through the text-completion service to LLM backends' native structured output APIs. When a prompt has response-type "json" and a strict-mode compatible schema, the LLM constrains token selection at the logit level to guarantee schema-valid output. Wire-level changes: - Add response_format and schema fields to TextCompletionRequest - Update translator to encode/decode new fields - Pass new fields through LlmService, TextCompletionClient, and PromptManager Runtime schema compatibility checker: - New is_strict_mode_compatible() utility validates schemas against LLM provider constraints (additionalProperties, required fields, no unsupported constraints, no open-ended objects) - Per-prompt eligibility decision: compliant schemas use structured output, non-compliant schemas fall back to free-text + post-hoc validation LLM backend implementations: - OpenAI: response_format with json_schema, variant-aware top-level array rejection (openai variant blocks, llama/vllm variants allow) - New vllm variant for the OpenAI backend - vLLM (dedicated): response_format in raw HTTP body - Ollama: format=<schema> parameter - Claude: tool-use trick (forced tool call with schema as input_schema) - Mistral: native json_schema response_format - Llamafile, LM Studio: OpenAI SDK response_format - Azure OpenAI: AzureOpenAI SDK response_format - Azure serverless: response_format in raw HTTP body - TGI: response_format in raw HTTP body - VertexAI Gemini: response_mime_type + response_schema - VertexAI Claude: tool-use trick - Google AI Studio: response_mime_type + response_schema - Bedrock, Cohere: signature-only (no structured output yet) Post-hoc jsonschema.validate() retained as defence-in-depth. Tech spec added: docs/tech-specs/structured-output.md Update tests |
||
|---|---|---|
| .. | ||
| test_async_processor.py | ||
| test_async_processor_config.py | ||
| test_audit_publisher.py | ||
| test_cassandra_config.py | ||
| test_document_embeddings_client.py | ||
| test_flow_base_modules.py | ||
| test_flow_parameter_specs.py | ||
| test_flow_processor.py | ||
| test_i18n.py | ||
| test_llm_service_parameters.py | ||
| test_logging.py | ||
| test_metrics.py | ||
| test_optional_request_response_spec.py | ||
| test_prompt_client_streaming.py | ||
| test_publisher_graceful_shutdown.py | ||
| test_qdrant_config.py | ||
| test_schema_compatibility.py | ||
| test_subscriber_graceful_shutdown.py | ||
| test_subscriber_readiness.py | ||