trustgraph/trustgraph_configurator/resources/dialog/docs/model/tgi-compose.md

Text Generation Inference (TGI) is Hugging Face's production-ready inference server for LLMs. It provides high-performance text generation with features like continuous batching, tensor parallelism, and optimized attention mechanisms.

The TGI service must be running with the required model loaded. The TGI service URL must be provided in an environment variable.

```
TGI_BASE_URL=http://tgi-host:8080/v1
```

Replace the URL with the URL of your TGI service, noting the `v1` suffix for OpenAI-compatible API.
Squashed 'ai-context/trustgraph-templates/' content from commit 42a5fd1b git-subtree-dir: ai-context/trustgraph-templates git-subtree-split: 42a5fd1b678f32be378062e30451e2052ccb95dd 2026-04-05 21:09:49 -05:00			`Text Generation Inference (TGI) is Hugging Face's production-ready inference server for LLMs. It provides high-performance text generation with features like continuous batching, tensor parallelism, and optimized attention mechanisms.`

			`The TGI service must be running with the required model loaded. The TGI service URL must be provided in an environment variable.`

			```
			`TGI_BASE_URL=http://tgi-host:8080/v1`
			```

			Replace the URL with the URL of your TGI service, noting the `v1` suffix for OpenAI-compatible API.