mirror of
https://github.com/katanemo/plano.git
synced 2026-05-01 20:03:40 +02:00
Add mock-based E2E tests and gate live tests to main/nightly
Introduce a new mock-based E2E test suite that uses pytest_httpserver to simulate LLM provider responses, eliminating the need for real API keys on PR builds. The mock tests cover model alias routing, protocol transformation (OpenAI↔Anthropic), Responses API passthrough/translation, streaming, tool calls, thinking mode, and multi-turn state management. CI changes: - Add mock-e2e-tests job (zero secrets, runs on every PR) - Gate all live E2E jobs to main pushes + nightly schedule - Scope secrets to only the keys each job actually needs - Add daily cron schedule for full live test coverage Also relaxes exact-match assertions in live e2e tests to structural checks (non-null, non-empty) since LLM output is non-deterministic. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
baeee56f6b
commit
3a6a672c9d
11 changed files with 1758 additions and 43 deletions
13
tests/archgw/docker-compose.mock.yaml
Normal file
13
tests/archgw/docker-compose.mock.yaml
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
services:
|
||||
plano:
|
||||
image: katanemo/plano:latest
|
||||
ports:
|
||||
- "12000:12000"
|
||||
volumes:
|
||||
- ./config_mock_llm.yaml:/app/plano_config.yaml
|
||||
- /etc/ssl/cert.pem:/etc/ssl/cert.pem
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
environment:
|
||||
- OPENAI_API_KEY=mock-key
|
||||
- ANTHROPIC_API_KEY=mock-key
|
||||
Loading…
Add table
Add a link
Reference in a new issue