<h2>Adding custom LLM Provider<a@click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)"aria-label="Copy link to this element"class="headerlink"data-tooltip="Copy link to this element"href="#adding-custom-llm-provider"x-intersect.margin.0%.0%.-70%.0%="activeSection ='#adding-custom-llm-provider'"><svgheight="1em"viewbox="0 0 24 24"width="1em"xmlns="http://www.w3.org/2000/svg"><pathd="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
<p>We support any OpenAI compliant LLM for example mistral, openai, ollama etc. We also offer first class support for OpenAI, Anthropic, DeepSeek, Mistral, Groq, and Ollama based models.
You can easily configure an LLM that communicates over the OpenAI API interface, by following the below guide.</p>
<p>For example following code block shows you how to add an ollama-supported LLM in the <cite>arch_config.yaml</cite> file.</p>
<p>And in the following code block shows you how to add mistral llm provider in the <cite>arch_config.yaml</cite> file.</p>
<p>For example following code block shows you how to add an ollama-supported LLM in the <codeclass="docutils literal notranslate"><spanclass="pre">arch_config.yaml</span></code> file.</p>
<p>And in the following code block shows you how to add mistral llm provider in the <codeclass="docutils literal notranslate"><spanclass="pre">arch_config.yaml</span></code> file.</p>
<h2>Example: Using the OpenAI Python SDK<a@click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)"aria-label="Copy link to this element"class="headerlink"data-tooltip="Copy link to this element"href="#example-using-the-openai-python-sdk"x-intersect.margin.0%.0%.-70%.0%="activeSection ='#example-using-the-openai-python-sdk'"><svgheight="1em"viewbox="0 0 24 24"width="1em"xmlns="http://www.w3.org/2000/svg"><pathd="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
</span><spanid="line-8"><spanclass="n">messages</span><spanclass="o">=</span><spanclass="p">[{</span><spanclass="s2">"role"</span><spanclass="p">:</span><spanclass="s2">"user"</span><spanclass="p">,</span><spanclass="s2">"content"</span><spanclass="p">:</span><spanclass="s2">"What is the capital of France?"</span><spanclass="p">}]</span>
</span><spanid="line-6"><spanclass="c1"># Define your model and messages</span>
</span><spanid="line-8"><spanclass="n">messages</span><spanclass="o">=</span><spanclass="p">[{</span><spanclass="s2">"role"</span><spanclass="p">:</span><spanclass="s2">"user"</span><spanclass="p">,</span><spanclass="s2">"content"</span><spanclass="p">:</span><spanclass="s2">"What is the capital of France?"</span><spanclass="p">}]</span>
</span><spanid="line-9">
</span><spanid="line-10"><spanclass="c1"># Send the messages to the LLM through Arch</span>
salmanap