-
-
-
-
+
-
-
-
-
+Arch Gateway was built by the contributors of [Envoy Proxy](https://www.envoyproxy.io/) with the belief that:
+
+>Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – outside core business logic.*
+
+
+Arch is engineered with purpose-built LLMs to handle critical but pesky tasks related to the handling and processing of prompts. This includes detecting and rejecting [jailbreak](https://github.com/verazuo/jailbreak_llms) attempts, intent-based routing for improved task accuracy, mapping user request into "backend" functions, and managing the observability of prompts and LLM API calls in a centralized way.
+
**Core Features**:
- - Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.
- - Task Routing & Fast Function Calling. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts to build more task-accurate agentic applications.
- - Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d): Arch centralizes guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code.
- - Routing & Traffic Management: Arch centralizes calls to LLMs used by your applications, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
- - Observability: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance.
+
+ - **Intent-based prompt routing & fast ⚡ function-calling via APIs**. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts to build more task-accurate agentic applications.
+ - **Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d)**: Arch centralizes guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code.
+ - **LLM Routing & Traffic Management**: Arch centralizes calls to LLMs used by your applications, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
+ - **Observability**: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance.
+ - **Built on [Envoy](https://envoyproxy.io)**: Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.
**High-Level Sequence Diagram**:
@@ -38,8 +49,9 @@ Arch is engineered with purpose-built LLMs to handle critical but undifferentiat
To get in touch with us, please join our [discord server](https://discord.gg/pGZf2gcwEc). We will be monitoring that actively and offering support there.
## Demos
-* [Weather Forecast](demos/samples_python/weather_forecast/README.md) - Walk through of the core function calling capabilities of arch gateway using weather forecasting service
-* [Network Agent](demos/samples_python/network_switch_operator_agent/README.md) - Build a networking co-pilot/agent agent with Arch
+* [Sample App: Weather Forecast Agent](demos/samples_python/weather_forecast/README.md) - A sample agentic weather forecasting appp that highlights core function calling capabilities of Arch.
+* [Sample App: Network Operator Agent](demos/samples_python/network_switch_operator_agent/README.md) - A simple network device switch operator agent that can retrive device statistics and reboot them.
+* [User Case: Connecting to SaaS APIs](demos/use_cases/spotify_bearer_auth) - Connect 3rd party SaaS APIs to your agentic chat experience.
## Quickstart
diff --git a/demos/use_cases/spotify_bearer_auth/arch_config.yaml b/demos/use_cases/spotify_bearer_auth/arch_config.yaml
index dbed4c8c..a259a539 100644
--- a/demos/use_cases/spotify_bearer_auth/arch_config.yaml
+++ b/demos/use_cases/spotify_bearer_auth/arch_config.yaml
@@ -82,7 +82,7 @@ system_prompt: |
Make sure your output is valid Markdown. And don't say "formatted in Markdown". Thanks!
llm_providers:
- - name: OpenAI
+ - name: openai
provider_interface: openai
access_key: $OPENAI_API_KEY
model: gpt-4o
@@ -120,36 +120,3 @@ prompt_targets:
path: /v1/artists/{artist_id}/top-tracks
http_headers:
Authorization: "Bearer $SPOTIFY_CLIENT_KEY"
-
- # - name: search_for_item
- # description: Get information about albums, artists, playlists, tracks, shows, episodes or audiobooks. You can search for an item by its name, creator, or topic.
- # parameters:
- # - name: q
- # description: Your search query, which can include keywords related to the item name, its creator, or its topic.
- # required: true
- # type: str
- # - name: type
- # type: str
- # description: The type of the item to search for (e.g., album, artist, playlist, track, show, episode, audiobook).
- # enum:
- # - album
- # - artist
- # - playlist
- # - track
- # - show
- # - episode
- # - audiobook
- # required: true
- # - name: market
- # type: str
- # description: A country code
- # default: US
- # - name: limit
- # type: integer
- # description: The maximum number of results to return
- # default: "5"
- # endpoint:
- # name: spotify
- # path: /v1/search
- # http_headers:
- # Authorization: "Bearer $SPOTIFY_CLIENT_KEY"
diff --git a/demos/use_cases/spotify_bearer_auth/docker-compose.yaml b/demos/use_cases/spotify_bearer_auth/docker-compose.yaml
index 32e52c40..ee5465a5 100644
--- a/demos/use_cases/spotify_bearer_auth/docker-compose.yaml
+++ b/demos/use_cases/spotify_bearer_auth/docker-compose.yaml
@@ -1,7 +1,7 @@
services:
chatbot_ui:
build:
- context: ../shared/chatbot_ui
+ context: ../../shared/chatbot_ui
ports:
- "18080:8080"
environment:
@@ -14,7 +14,7 @@ services:
jaeger:
build:
- context: ../shared/jaeger
+ context: ../../shared/jaeger
ports:
- "16686:16686"
- "4317:4317"