diff --git a/README.md b/README.md index c35ebcb8..c6200745 100644 --- a/README.md +++ b/README.md @@ -19,11 +19,14 @@ Engineered with purpose-built LLMs, Arch handles the critical but undifferentiat **Core Features**: - Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs. - - Function Calling for fast Agents and RAG apps. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts. + - Intent Routing & Fast Function Calling. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts to build smarter more accurate agentic applications. - Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d): Arch centralizes guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code. - - Routing & Traffic Management: Arch manages LLM calls, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability. + - Routing & Traffic Management: Arch centralizes calls to LLMs used by your applications, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability. - Observability: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance. +**High-Level Network Flow**: +![alt text](docs/source/_static/img/arch_network_diagram_high_level.png) + **Jump to our [docs](https://docs.archgw.com)** to learn how you can use Arch to improve the speed, security and personalization of your GenAI apps. > [!IMPORTANT] @@ -57,7 +60,7 @@ Arch's CLI allows you to manage and interact with the Arch gateway efficiently. ```console $ python -m venv venv $ source venv/bin/activate # On Windows, use: venv\Scripts\activate -$ pip install archgw==0.1.7 +$ pip install archgw==0.1.8 ``` ### Build AI Agent with Arch Gateway diff --git a/arch/tools/README.md b/arch/tools/README.md index a679520e..f8cfc0b3 100644 --- a/arch/tools/README.md +++ b/arch/tools/README.md @@ -19,7 +19,7 @@ source venv/bin/activate ### Step 3: Run the build script ```bash -pip install archgw==0.1.7 +pip install archgw==0.1.8 ``` ## Uninstall Instructions: archgw CLI diff --git a/docs/source/_static/img/arch_network_diagram_high_level.png b/docs/source/_static/img/arch_network_diagram_high_level.png new file mode 100644 index 00000000..8a66686f Binary files /dev/null and b/docs/source/_static/img/arch_network_diagram_high_level.png differ diff --git a/docs/source/conf.py b/docs/source/conf.py index 7151fee7..8359b195 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -13,9 +13,9 @@ from sphinxawesome_theme import ThemeOptions from sphinxawesome_theme.postprocess import Icons project = "Arch Docs" -copyright = "2024, Katanemo Labs, Inc" +copyright = "2025, Katanemo Labs, Inc" author = "Katanemo Labs, Inc" -release = " v0.1.7" +release = " v0.1.8" # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration diff --git a/docs/source/get_started/intro_to_arch.rst b/docs/source/get_started/intro_to_arch.rst index 9bd78f1e..97188ab1 100644 --- a/docs/source/get_started/intro_to_arch.rst +++ b/docs/source/get_started/intro_to_arch.rst @@ -8,10 +8,13 @@ Engineered with purpose-built large language models (LLMs), Arch handles all the detecting and rejecting jailbreak attempts, intelligently calling “backend” APIs to fulfill the user's request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM interactions in a centralized way. -.. image:: /_static/img/arch-logo.png +.. figure:: /_static/img/arch_network_diagram_high_level.png :width: 100% :align: center + High-level network flow of where Arch Gateway sits in your agentic stack. Designed for both ingress and egress prompt traffic. + + **The project was born out of the belief that:** *Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests diff --git a/docs/source/get_started/overview.rst b/docs/source/get_started/overview.rst index 60b63b1f..07a347af 100644 --- a/docs/source/get_started/overview.rst +++ b/docs/source/get_started/overview.rst @@ -7,6 +7,12 @@ Welcome to Arch, the intelligent prompt gateway designed to help developers buil In this documentation, you will learn how to quickly set up Arch to trigger API calls via prompts, apply prompt guardrails without writing any application-level logic, simplify the interaction with upstream LLMs, and improve observability all while simplifying your application development process. +.. figure:: /_static/img/arch_network_diagram_high_level.png + :width: 100% + :align: center + + High-level network flow of where Arch Gateway sits in your agentic stack. Designed for both ingress and egress prompt traffic. + Get Started ----------- diff --git a/docs/source/get_started/quickstart.rst b/docs/source/get_started/quickstart.rst index 1e572544..fd9f592c 100644 --- a/docs/source/get_started/quickstart.rst +++ b/docs/source/get_started/quickstart.rst @@ -25,7 +25,7 @@ Arch's CLI allows you to manage and interact with the Arch gateway efficiently. $ python -m venv venv $ source venv/bin/activate # On Windows, use: venv\Scripts\activate - $ pip install archgw==0.1.7 + $ pip install archgw==0.1.8 Build AI Agent with Arch Gateway diff --git a/docs/source/guides/function_calling.rst b/docs/source/guides/function_calling.rst index cb09bc6b..036d0e00 100644 --- a/docs/source/guides/function_calling.rst +++ b/docs/source/guides/function_calling.rst @@ -61,21 +61,6 @@ Key Features Parallel & Multiple Perform both parallel and multiple function calling ========================= =============================================================== - -Supported Languages -~~~~~~~~~~~~~~~~~~~ -.. table:: - :width: 100% - - ========================= =========================================================================================================================================== - **Language** **Data Type** - ========================= =========================================================================================================================================== - Python ``int``, ``str``, ``float``, ``bool``, ``list``, ``set``, ``dict``, ``tuple`` - Java ``byte``, ``short``, ``int``, ``long``, ``float``, ``double``, ``boolean``, ``char``, ``Array``, ``ArrayList``, ``Set``, ``HashMap``, ``Hashtable``, ``Queue``, ``Stack`` - Javascript ``Number``, ``Bigint``, ``String``, ``Boolean``, ``Object``, ``Array``, ``Date`` - ========================= =========================================================================================================================================== - - Implementing Function Calling ----------------------------- @@ -86,7 +71,6 @@ Step 1: Define the Function First, create or identify the backend function you want Arch to call. This could be an API endpoint, a script, or any other executable backend logic. .. code-block:: python - :caption: Example Function import requests @@ -136,63 +120,21 @@ Specify the parameters your function needs and how Arch should interpret these. Step 3: Arch Takes Over ~~~~~~~~~~~~~~~~~~~~~~~ -Once you have defined the functions and configured the prompt targets, Arch takes care of the remaining work. -It will automatically validate parameters validate parameters and ensure that the required parameters (e.g., location) are present in the prompt, and add validation rules if necessary. -Here is ane example validation schema using the `jsonschema `_ library +Once you have defined the functions and configured the prompt targets, Arch Gateway takes care of the remaining work. +It will automatically validate parameters, and ensure that the required parameters (e.g., location) are present in the prompt, and add validation rules if necessary. -.. code-block:: python - :caption: Example Validation Schema +.. figure:: /_static/img/arch_network_diagram_high_level.png + :width: 100% + :align: center - import requests - from jsonschema import validate, ValidationError - - # Define the JSON Schema for parameter validation - weather_validation_schema = { - "type": "object", - "properties": { - "location": { - "type": "string", - "minLength": 1, - "description": "The city and state, e.g. 'San Francisco, New York'" - }, - "unit": { - "type": "string", - "enum": ["celsius", "fahrenheit"], - "description": "The unit of temperature to return" - } - }, - "required": ["location"], - "additionalProperties": False - } - - def get_weather(location: str, unit: str = "fahrenheit"): - # Create the data object for validation - params = { - "location": location, - "unit": unit - } - - # Validate parameters using JSON Schema - try: - validate(instance=params, schema=weather_validation_schema) - except ValidationError as e: - raise ValueError(f"Invalid input: {e.message}") - - # Prepare the API request - api_server = "https://api.yourweatherapp.com" - endpoint = f"{api_server}/weather" - - # Make the API request - response = requests.get(endpoint, params=params) - return response.json() - - # Example usage - weather_info = get_weather("Seattle, WA", "celsius") - print(weather_info) + High-level network flow of where Arch Gateway sits in your agentic stack. Managing incoming and outgoing prompt traffic -Once the functions are called, Arch formats the response and deliver back to users. -By completing these setup steps, you enable Arch to manage the process from validation to response, ensuring users receive consistent, reliable results. +Once a downstream function (API) is called, Arch Gateway takes the response and sends it an upstream LLM to complete the request (for summarization, Q/A, text generation tasks). +For more details on how Arch Gateway enables you to centralize usage of LLMs, please read :ref:`LLM providers `. + +By completing these steps, you enable Arch to manage the process from validation to response, ensuring users receive consistent, reliable results - and that you are focused +on the stuff that matters most. Example Use Cases -----------------