diff --git a/README.md b/README.md index 136d48fc..d90fcd52 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,8 @@
-_The proxy server and the universal data plane for AI-native apps._

-Arch handles the *pesky low-level work* in building AI agents like clarifying vague user inputs, routing prompts to the right agents, calling tools for simple tasks, and unifying access to large language models (LLMs) - all without locking you into a framework. Move faster by focusing on the high-level logic of your agents. +_Arch is a proxy server designed as a modular edge and AI gateway for agentic apps_

+ Arch handles the *pesky low-level work* in building agentic apps — like applying guardrails, clarifying vague user input, routing prompts to the right agent, and unifying access to any LLM. It’s a language and framework friendly infrastructure layer designed to help you build and ship agentic apps faster. [Quickstart](#Quickstart) • @@ -80,7 +80,7 @@ Arch's CLI allows you to manage and interact with the Arch gateway efficiently. > We recommend that developers create a new Python virtual environment to isolate dependencies before installing Arch. This ensures that archgw and its dependencies do not interfere with other packages on your system. ```console -$ python -m venv venv +$ python3.12 -m venv venv $ source venv/bin/activate # On Windows, use: venv\Scripts\activate $ pip install archgw==0.3.4 ``` @@ -148,13 +148,10 @@ endpoints: ```sh $ archgw up arch_config.yaml -2024-12-05 16:56:27,979 - cli.main - INFO - Starting archgw cli version: 0.1.5 -... +2024-12-05 16:56:27,979 - cli.main - INFO - Starting archgw cli version: 0.3.4 2024-12-05 16:56:28,485 - cli.utils - INFO - Schema validation successful! 2024-12-05 16:56:28,485 - cli.main - INFO - Starting arch model server and arch gateway -... 2024-12-05 16:56:51,647 - cli.core - INFO - Container is healthy! - ``` Once the gateway is up you can start interacting with at port 10000 using openai chat completion API. diff --git a/docs/source/_static/img/arch-logo.png b/docs/source/_static/img/arch-logo.png index fa51c69a..5b6c0b15 100644 Binary files a/docs/source/_static/img/arch-logo.png and b/docs/source/_static/img/arch-logo.png differ diff --git a/docs/source/concepts/llm_provider.rst b/docs/source/concepts/llm_provider.rst index 551d6074..1c4b2aaf 100644 --- a/docs/source/concepts/llm_provider.rst +++ b/docs/source/concepts/llm_provider.rst @@ -35,28 +35,23 @@ make outbound LLM calls. Adding custom LLM Provider -------------------------- -We support any OpenAI compliant LLM for example mistral, openai, ollama etc. We offer first class support for openai and ollama. You can easily configure an LLM that communicates over the OpenAI API interface, by following the below guide. +We support any OpenAI compliant LLM for example mistral, openai, ollama etc. We also offer first class support for OpenAI, Anthropic, DeepSeek, Mistral, Groq, and Ollama based models. +You can easily configure an LLM that communicates over the OpenAI API interface, by following the below guide. For example following code block shows you how to add an ollama-supported LLM in the `arch_config.yaml` file. .. code-block:: yaml - - - name: local-llama + llm_providers: + - model: some_custom_llm_provider/llama3.2 provider_interface: openai - model: llama3.2 - endpoint: host.docker.internal:11434 + base_url: http://host.docker.internal:11434 - -For example following code block shows you how to add mistral llm provider in the `arch_config.yaml` file. +And in the following code block shows you how to add mistral llm provider in the `arch_config.yaml` file. .. code-block:: yaml - - - name: mistral-ai - provider_interface: openai - model: ministral-3b-latest - endpoint: api.mistral.ai:443 - protocol: https - + llm_providers: + - name: mistral/ministral-3b-latest + access_key: $MISTRAL_API_KEY Example: Using the OpenAI Python SDK ------------------------------------ @@ -65,15 +60,15 @@ Example: Using the OpenAI Python SDK from openai import OpenAI - # Initialize the Arch client - client = OpenAI(base_url="http://127.0.0.12000/") + # Initialize the Arch client + client = OpenAI(base_url="http://127.0.0.1:2000/") - # Define your LLM provider and prompt - llm_provider = "openai" - prompt = "What is the capital of France?" + # Define your model and messages + model = "llama3.2" + messages = [{"role": "user", "content": "What is the capital of France?"}] - # Send the prompt to the LLM through Arch - response = client.completions.create(llm_provider=llm_provider, prompt=prompt) + # Send the messages to the LLM through Arch + response = client.chat.completions.create(model=model, messages=messages) - # Print the response - print("LLM Response:", response) + # Print the response + print("LLM Response:", response.choices[0].message.content) diff --git a/docs/source/get_started/intro_to_arch.rst b/docs/source/get_started/intro_to_arch.rst index da63aedd..6409a546 100644 --- a/docs/source/get_started/intro_to_arch.rst +++ b/docs/source/get_started/intro_to_arch.rst @@ -20,7 +20,7 @@ Arch is designed to solve these problems by providing a unified, out-of-process High-level network flow of where Arch Gateway sits in your agentic stack. Designed for both ingress and egress prompt traffic. -Arch is an AI-native proxy server and the universal data plane for AI built by the contributors of Envoy Proxy with the belief that: +`Arch `_ is a modular edge and AI gateway for AI-native apps - built by the contributors of Envoy Proxy with the belief that: *Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) diff --git a/docs/source/get_started/overview.rst b/docs/source/get_started/overview.rst index afa1936e..8ccdb9b8 100644 --- a/docs/source/get_started/overview.rst +++ b/docs/source/get_started/overview.rst @@ -3,7 +3,7 @@ Overview ============ -`Arch `_ is an AI-native proxy server and the universal data plane for AI - one that is natively designed to handle and process AI prompts, not just network traffic. +`Arch `_ is a modular edge and AI gateway for AI-native apps - one that is natively designed to handle and process prompts, not just network traffic. Built by contributors to the widely adopted `Envoy Proxy `_, Arch helps you move faster by handling the pesky *low-level* work in AI agent development—fast input clarification, intelligent agent routing, seamless prompt-to-tool integration, and unified LLM access and observability—all without locking you into a framework. diff --git a/docs/source/index.rst b/docs/source/index.rst index 13231a53..67cf3423 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -14,7 +14,7 @@ Welcome to Arch! Arch - Build fast, hyper-personalized agents with intelligent infra | Product Hunt -`Arch `_ is an AI-native proxy server and the universal data plane for AI - one that is natively designed to handle and process AI prompts, not just network traffic. +`Arch `_ is a modular edge and AI gateway for AI-native apps - one that is natively designed to handle and process prompts, not just network traffic. Built by contributors to the widely adopted `Envoy Proxy `_, Arch helps you move faster by handling the pesky *low-level* work in AI agent development—fast input clarification, intelligent agent routing, seamless prompt-to-tool integration, and unified LLM access and observability—all without locking you into a framework.