feat: LLM-native structured output via JSON schema enforcement (#1037)

Thread existing JSON schemas from prompt definitions through the
text-completion service to LLM backends' native structured output
APIs. When a prompt has response-type "json" and a strict-mode
compatible schema, the LLM constrains token selection at the logit
level to guarantee schema-valid output.

Wire-level changes:
- Add response_format and schema fields to TextCompletionRequest
- Update translator to encode/decode new fields
- Pass new fields through LlmService, TextCompletionClient, and
  PromptManager

Runtime schema compatibility checker:
- New is_strict_mode_compatible() utility validates schemas against
  LLM provider constraints (additionalProperties, required fields,
  no unsupported constraints, no open-ended objects)
- Per-prompt eligibility decision: compliant schemas use structured
  output, non-compliant schemas fall back to free-text + post-hoc
  validation

LLM backend implementations:
- OpenAI: response_format with json_schema, variant-aware top-level
  array rejection (openai variant blocks, llama/vllm variants allow)
- New vllm variant for the OpenAI backend
- vLLM (dedicated): response_format in raw HTTP body
- Ollama: format=<schema> parameter
- Claude: tool-use trick (forced tool call with schema as input_schema)
- Mistral: native json_schema response_format
- Llamafile, LM Studio: OpenAI SDK response_format
- Azure OpenAI: AzureOpenAI SDK response_format
- Azure serverless: response_format in raw HTTP body
- TGI: response_format in raw HTTP body
- VertexAI Gemini: response_mime_type + response_schema
- VertexAI Claude: tool-use trick
- Google AI Studio: response_mime_type + response_schema
- Bedrock, Cohere: signature-only (no structured output yet)

Post-hoc jsonschema.validate() retained as defence-in-depth.

Tech spec added: docs/tech-specs/structured-output.md

Update tests
This commit is contained in:
cybermaggedon 2026-07-10 15:28:56 +01:00 committed by GitHub
parent f106ae2103
commit 9136526863
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
27 changed files with 1089 additions and 71 deletions

View file

@ -34,7 +34,7 @@ class TestPromptStreaming:
" of", " artificial", " intelligence", "."
]
async def streaming_text_completion_stream(system, prompt, handler, timeout=600):
async def streaming_text_completion_stream(system, prompt, handler, timeout=600, response_format=None, schema=None):
"""Simulate streaming text completion via text_completion_stream"""
for i, chunk_text in enumerate(chunks):
response = TextCompletionResponse(
@ -58,7 +58,7 @@ class TestPromptStreaming:
model="test-model",
)
async def non_streaming_text_completion(system, prompt, timeout=600):
async def non_streaming_text_completion(system, prompt, timeout=600, response_format=None, schema=None):
"""Simulate non-streaming text completion"""
full_text = "Machine learning is a field of artificial intelligence."
return TextCompletionResult(
@ -230,7 +230,7 @@ class TestPromptStreaming:
# Mock text completion client that raises an error
text_completion_client = AsyncMock()
async def failing_stream(system, prompt, handler, timeout=600):
async def failing_stream(system, prompt, handler, timeout=600, response_format=None, schema=None):
raise RuntimeError("Text completion error")
text_completion_client.text_completion_stream = AsyncMock(
@ -316,7 +316,7 @@ class TestPromptStreaming:
# Mock text completion that sends empty chunks
text_completion_client = AsyncMock()
async def empty_streaming(system, prompt, handler, timeout=600):
async def empty_streaming(system, prompt, handler, timeout=600, response_format=None, schema=None):
# Send empty chunk followed by final marker
await handler(TextCompletionResponse(
response="",