feat: pluggable image-to-text service with OpenAI vision backend (#1038)

Adds a full-stack image description service: schema, base class,                                                                  
OpenAI backend, gateway dispatch, client APIs (sync/async REST +
websocket), tg-describe-image CLI, IAM capability, and specs.                                                                     
                                                                                                                                    
Closes #879
This commit is contained in:
Sunny Yang 2026-07-12 05:47:04 -06:00 committed by GitHub
parent 9136526863
commit 40f01c123b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
42 changed files with 1845 additions and 14 deletions

View file

@ -0,0 +1,27 @@
type: object
description: |
Image-to-text request - describe an image using a vision model.
The image payload is base64-encoded; raw binary is not accepted.
required:
- image
- mime_type
properties:
image:
type: string
description: Base64-encoded image payload
example: iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==
mime_type:
type: string
description: MIME type of the image
example: image/png
prompt:
type: string
description: |
Instruction for the vision model. Optional; the service applies a
default prompt ("Describe this image") when omitted or empty.
example: List the objects visible in this image.
system:
type: string
description: Optional system prompt that sets behavior and context for the vision model
example: You are an expert at describing photographs for visually impaired users.