apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-04-25 00:36:34 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
plano/tests/e2e
History
Salman Paracha 2941392ed1
Adding support for wildcard models in the model_providers config (#696) 
* cleaning up plano cli commands

* adding support for wildcard model providers

* fixing compile errors

* fixing bugs related to default model provider, provider hint and duplicates in the model provider list

* fixed cargo fmt issues

* updating tests to always include the model id

* using default for the prompt_gateway path

* fixed the model name, as gpt-5-mini-2025-08-07 wasn't in the config

* making sure that all aliases and models match the config

* fixed the config generator to allow for base_url providers LLMs to include wildcard models

* re-ran the models list utility and added a shell script to run it

* updating docs to mention wildcard model providers

* updated provider_models.json to yaml, added that file to our docs for reference

* updating the build docs to use the new root-based build

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-342.local>
2026-01-28 17:47:33 -08:00
..
.vscode better model names (#517) 2025-07-11 16:42:16 -07:00
common.py Use intent model from archfc to pick prompt gateway (#328) 2024-12-20 13:25:01 -08:00
common_scripts.sh Use intent model from archfc to pick prompt gateway (#328) 2024-12-20 13:25:01 -08:00
config_memory_state_v1_responses.yaml rename cli to plano (#647) 2025-12-23 18:37:58 -08:00
docker-compose.yaml rename cli to plano (#647) 2025-12-23 18:37:58 -08:00
pyproject.toml use uv instead of poetry (#663) 2025-12-26 11:21:42 -08:00
README.md use uv instead of poetry (#663) 2025-12-26 11:21:42 -08:00
response.hex Add support for Amazon Bedrock Converse and ConverseStream (#588) 2025-10-22 11:31:21 -07:00
response_with_tools.hex Add support for Amazon Bedrock Converse and ConverseStream (#588) 2025-10-22 11:31:21 -07:00
run_e2e_tests.sh use uv instead of poetry (#663) 2025-12-26 11:21:42 -08:00
test_model_alias_routing.py making Messages.Content optional, and having the upstream LLM fail if the right fields aren't set (#699) 2026-01-16 16:24:03 -08:00
test_openai_responses_api_client.py Adding support for wildcard models in the model_providers config (#696) 2026-01-28 17:47:33 -08:00
test_openai_responses_api_client_with_state.py enable state management for v1/responses (#631) 2025-12-17 12:18:38 -08:00
test_prompt_gateway.py Adding support for wildcard models in the model_providers config (#696) 2026-01-28 17:47:33 -08:00
uv.lock use uv instead of poetry (#663) 2025-12-26 11:21:42 -08:00

README.md

e2e tests

e2e tests for arch llm gateway and prompt gateway

To be able to run e2e tests successfully run_e2e_script prepares environment in following way,

  1. build and start weather_forecast demo (using docker compose)
  2. build, install and start model server async (using uv)
  3. build and start arch gateway (using docker compose)
  4. wait for model server to be ready
  5. wait for arch gateway to be ready
  6. start e2e tests (using uv)
    1. runs llm gateway tests for llm routing
    2. runs prompt gateway tests to test function calling, parameter gathering and summarization
  7. cleanup
    1. stops arch gateway
    2. stops model server
    3. stops weather_forecast demo

How to run

To run locally make sure that following requirements are met.

Requirements

  • Python 3.10
  • uv
  • Docker

Running tests locally

sh run_e2e_test.sh