plano/demos/use_cases/llm_routing/arch_config.yaml

version: "0.1-beta"

listener:
  address: 0.0.0.0
  port: 10000
  message_format: huggingface
  connect_timeout: 0.005s

llm_providers:
  - name: gpt-4o-mini
    access_key: $OPENAI_API_KEY
    provider_interface: openai
    model: gpt-4o-mini
    default: true

  - name: gpt-3.5-turbo-0125
    access_key: $OPENAI_API_KEY
    provider_interface: openai
    model: gpt-3.5-turbo-0125

  - name: gpt-4o
    access_key: $OPENAI_API_KEY
    provider_interface: openai
    model: gpt-4o

  - name: ministral-3b
    access_key: $MISTRAL_API_KEY
    provider_interface: mistral
    model: ministral-3b-latest

tracing:
  random_sampling: 100
Add support for streaming and fixes few issues (see description) (#202) 2024-10-28 20:05:06 -04:00			`version: "0.1-beta"`

			`listener:`
			`address: 0.0.0.0`
			`port: 10000`
			`message_format: huggingface`
			`connect_timeout: 0.005s`

			`llm_providers:`
			`- name: gpt-4o-mini`
			`access_key: $OPENAI_API_KEY`
Fix llm_routing provider element (#382) * Fix llm_routing provider element We replaced provider with provider_interface to make it more clear to developers about provider api/backend being used. During that upgrade we removed support for mistral in provider to encourage developers to start using provider_interface. But this demo was not updated to use provider_interface as it was using mistral. This code change fixes it by replacing provider with provider_interface. Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> * fix the path * move * add more details * fix * Apply suggestions from code review * fix * fix --------- Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> 2025-01-24 16:34:11 -08:00			`provider_interface: openai`
Add support for streaming and fixes few issues (see description) (#202) 2024-10-28 20:05:06 -04:00			`model: gpt-4o-mini`
			`default: true`

			`- name: gpt-3.5-turbo-0125`
			`access_key: $OPENAI_API_KEY`
Fix llm_routing provider element (#382) * Fix llm_routing provider element We replaced provider with provider_interface to make it more clear to developers about provider api/backend being used. During that upgrade we removed support for mistral in provider to encourage developers to start using provider_interface. But this demo was not updated to use provider_interface as it was using mistral. This code change fixes it by replacing provider with provider_interface. Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> * fix the path * move * add more details * fix * Apply suggestions from code review * fix * fix --------- Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> 2025-01-24 16:34:11 -08:00			`provider_interface: openai`
Add support for streaming and fixes few issues (see description) (#202) 2024-10-28 20:05:06 -04:00			`model: gpt-3.5-turbo-0125`

			`- name: gpt-4o`
			`access_key: $OPENAI_API_KEY`
Fix llm_routing provider element (#382) * Fix llm_routing provider element We replaced provider with provider_interface to make it more clear to developers about provider api/backend being used. During that upgrade we removed support for mistral in provider to encourage developers to start using provider_interface. But this demo was not updated to use provider_interface as it was using mistral. This code change fixes it by replacing provider with provider_interface. Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> * fix the path * move * add more details * fix * Apply suggestions from code review * fix * fix --------- Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> 2025-01-24 16:34:11 -08:00			`provider_interface: openai`
Add support for streaming and fixes few issues (see description) (#202) 2024-10-28 20:05:06 -04:00			`model: gpt-4o`

			`- name: ministral-3b`
			`access_key: $MISTRAL_API_KEY`
Fix llm_routing provider element (#382) * Fix llm_routing provider element We replaced provider with provider_interface to make it more clear to developers about provider api/backend being used. During that upgrade we removed support for mistral in provider to encourage developers to start using provider_interface. But this demo was not updated to use provider_interface as it was using mistral. This code change fixes it by replacing provider with provider_interface. Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> * fix the path * move * add more details * fix * Apply suggestions from code review * fix * fix --------- Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com> 2025-01-24 16:34:11 -08:00			`provider_interface: mistral`
Add support for streaming and fixes few issues (see description) (#202) 2024-10-28 20:05:06 -04:00			`model: ministral-3b-latest`

			`tracing:`
			`random_sampling: 100`