Merge branch 'main' into adil/add_llm_agent_support

demos/use_cases/ollama/arch_config.yaml

@@ -1,9 +1,9 @@
 version: v0.1
 
 listeners:
-  ingress_traffic:
+  egress_traffic:
     address: 0.0.0.0
-    port: 10000
+    port: 12000
     message_format: openai
     timeout: 30s
 

demos/use_cases/ollama/docker-compose.yaml

@@ -6,7 +6,7 @@ services:
       - "18080:8080"
     environment:
       # this is only because we are running the sample app in the same docker container environemtn as archgw
-      - CHAT_COMPLETION_ENDPOINT=http://host.docker.internal:10000/v1
+      - CHAT_COMPLETION_ENDPOINT=http://host.docker.internal:12000/v1
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:

docs/source/concepts/prompt_target.rst

@@ -53,29 +53,28 @@ A prompt target configuration includes the following elements:
 - ``endpoint``: The API endpoint or function that handles the prompt.
 - ``parameters`` (Optional): A list of parameters to extract from the prompt.
 
+.. _defining_prompt_target_parameters:
+
 Defining Parameters
 ~~~~~~~~~~~~~~~~~~~
 Parameters are the pieces of information that Arch needs to extract from the user's prompt to perform the desired action.
-Each parameter can be marked as required or optional.
-Here is a full list of parameter attributes that Arch can support:
+Each parameter can be marked as required or optional. Here is a full list of parameter attributes that Arch can support:
 
 .. table::
     :width: 100%
 
-    ====================      ============================================================================
+    ========================  ============================================================================
     **Attribute**             **Description**
-    ====================      ============================================================================
-    ``name``                  Specifies identifier of parameters
-    ``type``                  Specifies the data type of the parameter.
-    ``description``           Provides a human-readable explanation of the parameter's purpose.
-    ``required``              Indicates whether the parameter is mandatory or optional
+    ========================  ============================================================================
+    ``name (req.)``           Specifies name of the parameter.
+    ``description (req.)``    Provides a human-readable explanation of the parameter's purpose.
+    ``type (req.)``           Specifies the data type. Supported types include: **int**, **str**, **float**, **bool**, **list**, **set**, **dict**, **tuple**
     ``default``               Specifies a default value for the parameter if not provided by the user.
-    ``items``                 Used in the context of arrays to define the schema of items within an array.
-    ``format``                Specifies a format for the parameter value, e.g., date and email
-    ``enum``                  Lists the allowable values for the parameter.
-    ``minimum``               Defines the minimum acceptable value for numeric parameters.
-    ``maximum``               Specifies the maximum acceptable value for numeric parameters.
-    ====================      ============================================================================
+    ``format``                Specifies a format for the parameter value. For example: `2019-12-31` for a date value.
+    ``enum``                  Lists of allowable values for the parameter with data type matching the ``type`` attribute. **Usage Example**: ``enum: ["celsius`", "fahrenheit"]``
+    ``items``                 Specifies the attribute of the elements when type euqals **list**, **set**, **dict**, **tuple**. **Usage Example**: ``items: {"type": "str"}``
+    ``required``              Indicates whether the parameter is mandatory or optional. Valid values: **true** or **false**
+    ========================  ============================================================================
 
 Example Configuration
 ~~~~~~~~~~~~~~~~~~~~~

docs/source/guides/function_calling.rst

@@ -118,6 +118,9 @@ Specify the parameters your function needs and how Arch should interpret these.
           name: api_server
           path: /weather
 
+.. Note::
+    For a complete refernce of attributes that you can configure in a prompt target, see :ref:`here <defining_prompt_target_parameters>`.
+
 Step 3: Arch Takes Over
 ~~~~~~~~~~~~~~~~~~~~~~~
 Once you have defined the functions and configured the prompt targets, Arch Gateway takes care of the remaining work.

tests/model_tests/arch_fc.hurl

@@ -0,0 +1,22 @@
+POST https://archfc.katanemo.dev/v1/chat/completions
+Content-Type: application/json
+
+{
+  "model": "Arch-Intent",
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant.\n\nYou task is to check if there are any tools that can be used to help the last user message in conversations according to the available tools listed below.\n\n<tools>\n{\"index\": \"T0\", \"type\": \"function\", \"function\": {\"name\": \"weather_forecast\", \"parameters\": {\"type\": \"object\", \"properties\": {\"city\": {\"type\": \"str\"}, \"days\": {\"type\": \"int\"}}, \"required\": [\"city\", \"days\"]}}}\n</tools>\n\nProvide your tool assessment for ONLY THE LAST USER MESSAGE in the above conversation:\n- First line must read 'Yes' or 'No'.\n- If yes, a second line must include a comma-separated list of tool indexes.\n"
+    },
+    { "role": "user", "content": "how is the weather in seattle? Are there any tools can help?" }
+  ],
+  "stream": false
+}
+
+HTTP 200
+[Asserts]
+header "content-type" == "application/json"
+jsonpath "$.model" matches /^Arch-Function/
+jsonpath "$.usage" != null
+jsonpath "$.choices[0].message.content" matches /Yes/
+jsonpath "$.choices[0].message.role" == "assistant"