mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-15 00:02:11 +02:00
feat: LLM-native structured output via JSON schema enforcement (#1037)
Thread existing JSON schemas from prompt definitions through the text-completion service to LLM backends' native structured output APIs. When a prompt has response-type "json" and a strict-mode compatible schema, the LLM constrains token selection at the logit level to guarantee schema-valid output. Wire-level changes: - Add response_format and schema fields to TextCompletionRequest - Update translator to encode/decode new fields - Pass new fields through LlmService, TextCompletionClient, and PromptManager Runtime schema compatibility checker: - New is_strict_mode_compatible() utility validates schemas against LLM provider constraints (additionalProperties, required fields, no unsupported constraints, no open-ended objects) - Per-prompt eligibility decision: compliant schemas use structured output, non-compliant schemas fall back to free-text + post-hoc validation LLM backend implementations: - OpenAI: response_format with json_schema, variant-aware top-level array rejection (openai variant blocks, llama/vllm variants allow) - New vllm variant for the OpenAI backend - vLLM (dedicated): response_format in raw HTTP body - Ollama: format=<schema> parameter - Claude: tool-use trick (forced tool call with schema as input_schema) - Mistral: native json_schema response_format - Llamafile, LM Studio: OpenAI SDK response_format - Azure OpenAI: AzureOpenAI SDK response_format - Azure serverless: response_format in raw HTTP body - TGI: response_format in raw HTTP body - VertexAI Gemini: response_mime_type + response_schema - VertexAI Claude: tool-use trick - Google AI Studio: response_mime_type + response_schema - Bedrock, Cohere: signature-only (no structured output yet) Post-hoc jsonschema.validate() retained as defence-in-depth. Tech spec added: docs/tech-specs/structured-output.md Update tests
This commit is contained in:
parent
f106ae2103
commit
9136526863
27 changed files with 1089 additions and 71 deletions
|
|
@ -34,7 +34,7 @@ class TestPromptStreaming:
|
|||
" of", " artificial", " intelligence", "."
|
||||
]
|
||||
|
||||
async def streaming_text_completion_stream(system, prompt, handler, timeout=600):
|
||||
async def streaming_text_completion_stream(system, prompt, handler, timeout=600, response_format=None, schema=None):
|
||||
"""Simulate streaming text completion via text_completion_stream"""
|
||||
for i, chunk_text in enumerate(chunks):
|
||||
response = TextCompletionResponse(
|
||||
|
|
@ -58,7 +58,7 @@ class TestPromptStreaming:
|
|||
model="test-model",
|
||||
)
|
||||
|
||||
async def non_streaming_text_completion(system, prompt, timeout=600):
|
||||
async def non_streaming_text_completion(system, prompt, timeout=600, response_format=None, schema=None):
|
||||
"""Simulate non-streaming text completion"""
|
||||
full_text = "Machine learning is a field of artificial intelligence."
|
||||
return TextCompletionResult(
|
||||
|
|
@ -230,7 +230,7 @@ class TestPromptStreaming:
|
|||
# Mock text completion client that raises an error
|
||||
text_completion_client = AsyncMock()
|
||||
|
||||
async def failing_stream(system, prompt, handler, timeout=600):
|
||||
async def failing_stream(system, prompt, handler, timeout=600, response_format=None, schema=None):
|
||||
raise RuntimeError("Text completion error")
|
||||
|
||||
text_completion_client.text_completion_stream = AsyncMock(
|
||||
|
|
@ -316,7 +316,7 @@ class TestPromptStreaming:
|
|||
# Mock text completion that sends empty chunks
|
||||
text_completion_client = AsyncMock()
|
||||
|
||||
async def empty_streaming(system, prompt, handler, timeout=600):
|
||||
async def empty_streaming(system, prompt, handler, timeout=600, response_format=None, schema=None):
|
||||
# Send empty chunk followed by final marker
|
||||
await handler(TextCompletionResponse(
|
||||
response="",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue