Fix errors and improve Doc (#143)

* Fix link issues and add icons

* Improve Doc

* fix test

* making minor modifications to shuguangs' doc changes

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
This commit is contained in:
Shuguang Chen 2024-10-08 13:18:34 -07:00 committed by GitHub
parent 3ed50e61d2
commit b30ad791f7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
27 changed files with 396 additions and 329 deletions

View file

@ -1,58 +0,0 @@
.. _error_target:
Error Targets
=============
**Error targets** are designed to capture and manage specific issues or exceptions that occur during Arch's function or system's execution.
These endpoints receive errors forwarded from Arch when issues arise, such as improper function/API calls, guardrail violations, or other processing errors.
The errors are communicated to the application via headers like ``X-Arch-[ERROR-TYPE]``, enabling you to respond appropriately and handle errors gracefully.
Key Concepts
------------
**Error Type**: Categorizes the nature of the error, such as "ValidationError" or "RuntimeError." These error types help in identifying what
kind of issue occurred and provide context for troubleshooting.
**Error Message**: A clear, human-readable message describing the error. This should provide enough detail to inform users or developers of
the root cause or required action.
**Target Prompt**: The specific prompt or operation where the error occurred. Understanding where the error happened helps with debugging
and pinpointing the source of the problem.
**Parameter-Specific Errors**: Errors that arise due to invalid or missing parameters when invoking a function. These errors are critical
for ensuring the correctness of inputs.
Error Header Example
--------------------
.. code-block:: http
HTTP/1.1 400 Bad Request
X-Arch-Error-Type: FunctionValidationError
X-Arch-Error-Message: Tools call parsing failure
X-Arch-Target-Prompt: createUser
Content-Type: application/json
"messages": [
{
"role": "user",
"content": "Please create a user with the following ID: 1234"
},
{
"role": "system",
"content": "Expected a string for 'user_id', but got an integer."
}]
Best Practices and Tips
-----------------------
- **Graceful Degradation**: If an error occurs, fail gracefully by providing fallback logic or alternative flows when possible.
- **Log Errors**: Always log errors on the server side for later analysis.
- **Client-Side Handling**: Make sure the client can interpret error responses and provide meaningful feedback to the user. Clients should not display raw error codes or stack traces but rather handle them gracefully.

View file

@ -1,4 +1,4 @@
version: "0.1-beta"
version: v0.1
listener:
address: 0.0.0.0 # or 127.0.0.1
@ -8,9 +8,9 @@ listener:
common_tls_context: # If you configure port 443, you'll need to update the listener with your TLS certificates
tls_certificates:
- certificate_chain:
filename: "/etc/certs/cert.pem"
filename: /etc/certs/cert.pem
private_key:
filename: "/etc/certs/key.pem"
filename: /etc/certs/key.pem
# Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.
endpoints:
@ -18,42 +18,42 @@ endpoints:
# value could be ip address or a hostname with port
# this could also be a list of endpoints for load balancing
# for example endpoint: [ ip1:port, ip2:port ]
endpoint: "127.0.0.1:80"
endpoint: 127.0.0.1:80
# max time to wait for a connection to be established
connect_timeout: 0.005s
mistral_local:
endpoint: "127.0.0.1:8001"
endpoint: 127.0.0.1:8001
error_target:
endpoint: "error_target_1"
endpoint: error_target_1
# Centralized way to manage LLMs, manage keys, retry logic, failover and limits in a central way
llm_providers:
- name: "OpenAI"
provider: "openai"
access_key: $OPENAI_API_KEY
- name: OpenAI
provider: openai
access_key: OPENAI_API_KEY
model: gpt-4o
default: true
stream: true
rate_limits:
selector: #optional headers, to add rate limiting based on http headers like JWT tokens or API keys
http_header:
name: "Authorization"
name: Authorization
value: "" # Empty value means each separate value has a separate limit
limit:
tokens: 100000 # Tokens per unit
unit: "minute"
unit: minute
- name: "Mistral8x7b"
provider: "mistral"
access_key: $MISTRAL_API_KEY
model: "mistral-8x7b"
- name: Mistral8x7b
provider: mistral
access_key: MISTRAL_API_KEY
model: mistral-8x7b
- name: "MistralLocal7b"
provider: "local"
model: "mistral-7b-instruct"
endpoint: "mistral_local"
- name: MistralLocal7b
provider: local
model: mistral-7b-instruct
endpoint: mistral_local
# provides a way to override default settings for the arch system
overrides:
@ -62,44 +62,41 @@ overrides:
prompt_target_intent_matching_threshold: 0.60
# default system prompt used by all prompt targets
system_prompt: |
You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.
system_prompt: You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.
prompt_guards:
input_guards:
jailbreak:
on_exception:
message: "Looks like you're curious about my abilities, but I can only provide assistance within my programmed parameters."
message: Looks like you're curious about my abilities, but I can only provide assistance within my programmed parameters.
prompt_targets:
- name: "reboot_network_device"
description: "Helps network operators perform device operations like rebooting a device."
endpoint:
name: app_server
path: "/agent/action"
parameters:
- name: "device_id"
# additional type options include: int | float | bool | string | list | dict
type: "string"
description: "Identifier of the network device to reboot."
required: true
- name: "confirmation"
type: "string"
description: "Confirmation flag to proceed with reboot."
default: "no"
enum: [yes, no]
- name: "information_extraction"
- name: information_extraction
default: true
description: "This prompt handles all scenarios that are question and answer in nature. Like summarization, information extraction, etc."
description: handel all scenarios that are question and answer in nature. Like summarization, information extraction, etc.
endpoint:
name: app_server
path: "/agent/summary"
path: /agent/summary
# Arch uses the default LLM and treats the response from the endpoint as the prompt to send to the LLM
auto_llm_dispatch_on_response: true
# override system prompt for this prompt target
system_prompt: |
You are a helpful information extraction assistant. Use the information that is provided to you.
system_prompt: You are a helpful information extraction assistant. Use the information that is provided to you.
- name: reboot_network_device
description: Reboot a specific network device
endpoint:
name: app_server
path: /agent/action
parameters:
- name: device_id
type: str
description: Identifier of the network device to reboot.
required: true
- name: confirmation
type: bool
description: Confirmation flag to proceed with reboot.
default: false
enum: [true, false]
error_target:
endpoint: