Adil/fix salman docs (#75)

* added the first set of docs for our technical docs

* more docuemtnation changes

* added support for prompt processing and updated life of a request

* updated docs to including getting help sections and updated life of a request

* committing local changes for getting started guide, sample applications, and full reference spec for prompt-config

* updated configuration reference, added sample app skeleton, updated favico

* fixed the configuration refernce file, and made minor changes to the intent detection. commit v1 for now

* Updated docs with use cases and example code, updated what is arch, and made minor changes throughout

* fixed imaged and minor doc fixes

* add sphinx_book_theme

* updated README, and make some minor fixes to documetnation

* fixed README.md

* fixed image width

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
This commit is contained in:
Salman Paracha 2024-09-24 13:54:17 -07:00 committed by GitHub
parent 2d31aeaa36
commit 13dff3089d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
33 changed files with 931 additions and 287 deletions

View file

@ -0,0 +1,41 @@
version: "0.1-beta"
listen:
address: 127.0.0.1 | 0.0.0.0
port_value: 8080 #If you configure port 443, you'll need to update the listener with tls_certificates
system_prompts:
- name: network_assistant
content: You are a network assistant that just offers facts about the operational health of the network
llm_providers:
- name: "OpenAI"
access_key: $OPEN_AI_KEY
model: gpt-4o
default: true
prompt_targets:
- name: reboot_devices
description: >
This prompt target handles user requests to reboot devices.
It ensures that when users request to reboot specific devices or device groups, the system processes the reboot commands accurately.
**Examples of user prompts:**
- "Please reboot device 12345."
- "Restart all devices in tenant group tenant-XYZ
- "I need to reboot devices A, B, and C."
path: /agent/device_reboot
parameters:
- name: "device_ids"
type: list # Options: integer | float | list | dictionary | set
description: "A list of device identifiers (IDs) to reboot."
required: false
- name: "device_group"
type: string # Options: string | integer | float | list | dictionary | set
description: "The name of the device group to reboot."
required: false
prompt_endpoints:
- "http://127.0.0.2"
- "http://127.0.0.1"

View file

@ -1,21 +1,22 @@
version: "0.1-beta"
listen:
address: 127.0.0.1 | 0.0.0.0
port_value: 8080 #If you configure port 443, you'll need to update the listener to with your tls_certificates
messages: tuple | hugging-face-messages-api
port_value: 8080 #If you configure port 443, you'll need to update the listener with tls_certificates
messages: tuple | hugging-face-messages-api
system_prompts:
- name: network_assistant
content: you are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions
content: You are a network assistant that just offers facts about the operational health of the network
llm_providers:
llm_providers:
- name: "OpenAI"
access_key: $OPEN_AI_KEY
model: gpt-4
model: gpt-4o
default: true
- name: "Mistral"
access_key: $MISTRAL_KEY
model: "mixtral8-7B"
model: mixtral8-7B
prompt_endpoints:
- "http://127.0.0.2"
- "http://127.0.0.1"
@ -23,17 +24,18 @@ prompt_endpoints:
prompt_guards:
input-guard:
- name: #jailbreak
on-exception-message: Looks like you are curious about my abilities. But I can only
prompt_targets:
on-exception-message: Looks like you are curious about my abilities. But I can only
prompt_targets:
- name: information_extraction
type: RAG
type: RAG
description: this prompt handles all information extractions scenarios
path: /agent/summary
- name: reboot_network_device
path: /agent/action
description: used to help network operators with perform device operations like rebooting a device.
parameters:
error_target: #handle errors from Bolt or upstream LLMs
parameters:
error_target: #handle errors from Bolt or upstream LLMs
name: “error_handler”
path: /errors
path: /errors

View file

@ -2,7 +2,7 @@ version: "0.1-beta"
listener:
address: 0.0.0.0 # or 127.0.0.1
port_value: 8080
port_value: 8080
messages: "hugging-face-messages-json" # Defines how Arch should parse the content from application/json or text/pain Content-type in the http request
common_tls_context: # If you configure port 443, you'll need to update the listener with your TLS certificates
tls_certificates:
@ -16,18 +16,17 @@ system_prompts:
content: |
You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.
#Centralized way to manage LLM providers that the application has access to. Manage keys retry logic, failover, and limits in a central way
llm_providers:
llm_providers: #Centralized way to manage LLMs, manage keys, retry logic, failover and limits in a central way
- name: "OpenAI"
access_key: $OPENAI_API_KEY
model: "gpt-40"
model: gpt-4o
default: true
stream: true
rate_limit:
selector: #optional headers, to add rate limiting based on http headers like JWT tokens or API keys
http-header:
name: "Authorization"
value: "" # Empty value means each separate value has a separate limit
value: "" # Empty value means each separate value has a separate limit
limit:
tokens: 100000 # Tokens per unit
unit: "minute"
@ -44,7 +43,6 @@ prompt_guards:
- name: "jailbreak"
on_exception:
forward_to_error_target: true
# Additional guard configurations can be added here
- name: "toxicity"
on_exception:
message: "Looks like you're curious about my abilities, but I can only provide assistance within my programmed parameters."
@ -74,5 +72,7 @@ error_target:
name: "error_handler"
path: "/errors"
intent-detection-threshold-override: 0.60 # By default Arch uses an NLI + embedding approach to match an incomming prompt to a prompt target.
# The intent matching threshold is kept at 0.80, you can overide this behavior if you would like
tracing: 100 #sampling rate. Note by default Arch works on OpenTelemetry compatible tracing.
intent-detection-threshold-override: 0.60 # By default Arch uses an NLI + embedding approach to match an incomming prompt to a prompt target.
# The intent matching threshold is kept at 0.80, you can overide this behavior if you would like

View file

@ -0,0 +1,40 @@
version: "0.1-beta"
listen:
address: 127.0.0.1 | 0.0.0.0
port_value: 8080 #If you configure port 443, you'll need to update the listener with tls_certificates
system_prompts:
- name: network_assistant
content: You are a network assistant that just offers facts about the operational health of the network
llm_providers:
- name: "OpenAI"
access_key: $OPEN_AI_KEY
model: gpt-4o
default: true
prompt_targets:
- name: get_device_statistics
description: >
This prompt target ensures that when users request device-related statistics, the system accurately retrieves and presents the relevant data
based on the specified devices and time range. Examples of user prompts, include:
- "Show me the performance stats for device 12345 over the past week."
- "What are the error rates for my devices in the last 24 hours?"
- "I need statistics on device 789 over the last 10 days."
path: /agent/device_summary
parameters:
- name: "device_ids"
type: list # Options: integer | float | list | dictionary | set
description: "A list of device identifiers (IDs) for which the statistics are requested."
required: true
- name: "time_range"
type: integer # Options: integer | float | list | dictionary | set
description: "The number of days in the past over which to retrieve device statistics. Defaults to 7 days if not specified."
required: false
default: 7
prompt_endpoints:
- "http://127.0.0.2"
- "http://127.0.0.1"