mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-14 07:42:11 +02:00
9136526863
Thread existing JSON schemas from prompt definitions through the text-completion service to LLM backends' native structured output APIs. When a prompt has response-type "json" and a strict-mode compatible schema, the LLM constrains token selection at the logit level to guarantee schema-valid output. Wire-level changes: - Add response_format and schema fields to TextCompletionRequest - Update translator to encode/decode new fields - Pass new fields through LlmService, TextCompletionClient, and PromptManager Runtime schema compatibility checker: - New is_strict_mode_compatible() utility validates schemas against LLM provider constraints (additionalProperties, required fields, no unsupported constraints, no open-ended objects) - Per-prompt eligibility decision: compliant schemas use structured output, non-compliant schemas fall back to free-text + post-hoc validation LLM backend implementations: - OpenAI: response_format with json_schema, variant-aware top-level array rejection (openai variant blocks, llama/vllm variants allow) - New vllm variant for the OpenAI backend - vLLM (dedicated): response_format in raw HTTP body - Ollama: format=<schema> parameter - Claude: tool-use trick (forced tool call with schema as input_schema) - Mistral: native json_schema response_format - Llamafile, LM Studio: OpenAI SDK response_format - Azure OpenAI: AzureOpenAI SDK response_format - Azure serverless: response_format in raw HTTP body - TGI: response_format in raw HTTP body - VertexAI Gemini: response_mime_type + response_schema - VertexAI Claude: tool-use trick - Google AI Studio: response_mime_type + response_schema - Bedrock, Cohere: signature-only (no structured output yet) Post-hoc jsonschema.validate() retained as defence-in-depth. Tech spec added: docs/tech-specs/structured-output.md Update tests |
||
|---|---|---|
| .. | ||
| test_agent | ||
| test_api | ||
| test_base | ||
| test_bootstrap | ||
| test_chunking | ||
| test_cli | ||
| test_clients | ||
| test_concurrency | ||
| test_config | ||
| test_cores | ||
| test_decoding | ||
| test_direct | ||
| test_embeddings | ||
| test_extract | ||
| test_gateway | ||
| test_iam | ||
| test_knowledge_graph | ||
| test_librarian | ||
| test_provenance | ||
| test_pubsub | ||
| test_query | ||
| test_rdf | ||
| test_reliability | ||
| test_retrieval | ||
| test_rev_gateway | ||
| test_storage | ||
| test_structured_data | ||
| test_tables | ||
| test_text_completion | ||
| test_translators | ||
| __init__.py | ||
| test_prompt_manager.py | ||
| test_prompt_manager_edge_cases.py | ||
| test_python_api_client.py | ||