apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-03 06:51:00 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
trustgraph/trustgraph_configurator/resources/dialog/docs/model/tgi-compose.md
elpresidank 74cc8a4685 Squashed 'ai-context/trustgraph-templates/' content from commit 42a5fd1b 
git-subtree-dir: ai-context/trustgraph-templates
git-subtree-split: 42a5fd1b678f32be378062e30451e2052ccb95dd
2026-04-05 21:09:49 -05:00

512 B
Raw Blame History

Text Generation Inference (TGI) is Hugging Face's production-ready inference server for LLMs. It provides high-performance text generation with features like continuous batching, tensor parallelism, and optimized attention mechanisms.

The TGI service must be running with the required model loaded. The TGI service URL must be provided in an environment variable.

TGI_BASE_URL=http://tgi-host:8080/v1

Replace the URL with the URL of your TGI service, noting the v1 suffix for OpenAI-compatible API.