apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-06-17 15:25:17 +02:00
Code Issues Projects Releases Packages Wiki Activity

add comments for v1.1 archfc model endpoint

Browse source
This commit is contained in:
Adil Hafeez 2025-04-15 13:26:43 -07:00
parent 9b538ff074
commit 8b9f48ce9b
No known key found for this signature in database
GPG key ID: 9B18EF7691369645
2 changed files with 8 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
demos/use_cases/llm_routing/arch_config.yaml
View file

@ -30,5 +30,11 @@ llm_providers:
model: deepseek-reasoner
base_url: https://api.deepseek.com/
- name: groq
access_key: $GROQ_API_KEY
provider_interface: openai
model: llama-3.1-8b-instant
base_url: https://api.groq.com
tracing:
random_sampling: 100

2
model_server/src/commons/globals.py
View file

@ -16,6 +16,8 @@ logger = get_model_server_logger()
# Define the client
# ARCH_ENDPOINT = os.getenv("ARCH_ENDPOINT", "https://archfc.katanemo.dev/v1")
# use temporary endpoint until we deprecate archfc-v1.0 from archfc.katanemo.dev
# and officially release archfc-v1.1 on archfc.katanemo.dev
ARCH_ENDPOINT = os.getenv("ARCH_ENDPOINT", "http://34.72.123.163:8000/v1")
ARCH_API_KEY = "EMPTY"
ARCH_CLIENT = OpenAI(base_url=ARCH_ENDPOINT, api_key=ARCH_API_KEY)