add listener routes for internal service proxying (#793)

This commit is contained in:
Adil Hafeez 2026-03-01 23:51:14 -08:00
parent 198c912202
commit c2480639b2
No known key found for this signature in database
GPG key ID: 9B18EF7691369645
13 changed files with 219 additions and 2 deletions

View file

@ -77,3 +77,33 @@ listener with address, port, and protocol details:
When you start Plano, you specify a listener address/port that you want to bind downstream. Plano also exposes a
predefined internal listener (``127.0.0.1:12000``) that you can use to proxy egress calls originating from your
application to LLMs (API-based or hosted) via prompt targets.
Internal Service Routes
^^^^^^^^^^^^^^^^^^^^^^^
Listeners support an optional ``routes`` block that lets you forward path-prefix traffic to co-located internal
services (for example, a Jaeger tracing UI or a Prometheus dashboard) through the same listener port:
.. code-block:: yaml
listeners:
- type: agent
name: agent_1
port: 8001
routes:
- path_prefix: /traces
upstream: http://jaeger:16686
- path_prefix: /metrics
upstream: http://localhost:9090
agents:
- id: my_agent
description: my agent
With the configuration above, requests to ``http://localhost:8001/traces`` are proxied to the Jaeger UI, while
all other requests are routed to the agent as usual. Each route entry requires:
- ``path_prefix`` — the URL prefix to match (e.g., ``/traces``).
- ``upstream`` — the full URL of the internal service to forward to.
Routes are evaluated before the default agent catch-all, so specific prefixes always take priority. A listener
can also define only ``routes`` (without ``agents``) to act as a lightweight reverse proxy for internal services.

View file

@ -453,6 +453,30 @@ Handle incoming requests:
print(f"Payment service response: {response.content}")
Exposing a Trace UI via Listener Routes
----------------------------------------
If you run a trace collector with a web UI (such as Jaeger) alongside Plano, you can expose it through the same
listener port using the ``routes`` configuration. This avoids publishing extra ports and gives developers a single
endpoint for both agent traffic and trace inspection.
.. code-block:: yaml
listeners:
- type: agent
name: agent_1
port: 8001
routes:
- path_prefix: /traces
upstream: http://jaeger:16686
agents:
- id: my_agent
description: my agent
With this configuration, browsing to ``http://localhost:8001/traces`` opens the Jaeger UI while all other
requests continue to be routed to the agent. See :ref:`plano_overview_listeners` for more details on the
``routes`` block.
Integrating with Tracing Tools
------------------------------

View file

@ -55,6 +55,10 @@ listeners:
port: 8001
router: plano_orchestrator_v1
address: 0.0.0.0
# Routes forward path-prefix traffic to internal services (e.g., Jaeger UI)
routes:
- path_prefix: /traces
upstream: http://jaeger:16686
agents:
- id: rag_agent
description: virtual assistant for retrieval augmented generation tasks

View file

@ -19,6 +19,10 @@ endpoints:
mistral_local:
endpoint: 127.0.0.1
port: 8001
route_travel_booking_service_traces:
endpoint: jaeger
port: 16686
protocol: http
weather_agent:
endpoint: host.docker.internal
port: 10510
@ -36,6 +40,10 @@ listeners:
name: travel_booking_service
port: 8001
router: plano_orchestrator_v1
routes:
- cluster_name: route_travel_booking_service_traces
path_prefix: /traces
upstream: http://jaeger:16686
type: agent
- address: 0.0.0.0
model_providers: