Fix errors and improve Doc (#143)

* Fix link issues and add icons

* Improve Doc

* fix test

* making minor modifications to shuguangs' doc changes

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
This commit is contained in:
Shuguang Chen 2024-10-08 13:18:34 -07:00 committed by GitHub
parent 3ed50e61d2
commit b30ad791f7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
27 changed files with 396 additions and 329 deletions

View file

@ -1,17 +1,17 @@
.. _arch_overview_prompt_handling:
Prompt
=================
Prompts
=======
Arch's primary design point is to securely accept, process and handle prompts. To do that effectively,
Arch relies on Envoy's HTTP `connection management <https://www.envoyproxy.io/docs/envoy/v1.31.2/intro/arch_overview/http/http_connection_management>`_,
subsystem and its **prompt handler** subsystem engineered with purpose-built LLMs to
implement critical functionality on behalf of developers so that you can stay focused on business logic.
.. Note::
Arch's **prompt handler** subsystem interacts with the **model** subsytem through Envoy's cluster manager
system to ensure robust, resilient and fault-tolerant experience in managing incoming prompts. Read more
about the :ref:`model subsystem <arch_model_serving>` and how the LLMs are hosted in Arch.
Arch's **prompt handler** subsystem interacts with the **model subsytem** through Envoy's cluster manager system to ensure robust, resilient and fault-tolerant experience in managing incoming prompts.
.. seealso::
Read more about the :ref:`model subsystem <model_serving>` and how the LLMs are hosted in Arch.
Messages
--------
@ -24,7 +24,7 @@ containing two key-value pairs:
- **Content**: Contains the actual text of the message.
Prompt Guardrails
Prompt Guard
-----------------
Arch is engineered with :ref:`Arch-Guard <prompt_guard>`, an industry leading safety layer, powered by a
@ -36,12 +36,12 @@ To add jailbreak guardrails, see example below:
.. literalinclude:: ../includes/arch_config.yaml
:language: yaml
:linenos:
:lines: 1-45
:emphasize-lines: 22-26
:lines: 1-25
:emphasize-lines: 21-25
:caption: Example Configuration
.. Note::
As a roadmap item, Arch will expose the ability for developers to define custom guardrails via Arch-Guard-v2,
As a roadmap item, Arch will expose the ability for developers to define custom guardrails via Arch-Guard,
and add support for additional safety checks defined by developers and hazardous categories like, violent crimes, privacy, hate,
etc. To offer feedback on our roadmap, please visit our `github page <https://github.com/orgs/katanemo/projects/1>`_
@ -59,10 +59,14 @@ Configuring ``prompt_targets`` is simple. See example below:
.. literalinclude:: ../includes/arch_config.yaml
:language: yaml
:linenos:
:emphasize-lines: 29-38
:emphasize-lines: 39-53
:caption: Example Configuration
.. seealso::
Check :ref:`Prompt Target <prompt_target>` for more details!
Intent Detection and Prompt Matching:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -127,10 +131,6 @@ Example: Using OpenAI Client with Arch as an Egress Gateway
print("OpenAI Response:", response.choices[0].text.strip())
In these examples:
The OpenAI client is used to send traffic directly through the Arch egress proxy to the LLM of your choice, such as OpenAI.
The OpenAI client is configured to route traffic via Arch by setting the proxy to 127.0.0.1:51001, assuming Arch is
running locally and bound to that address and port.
In these examples, the OpenAI client is used to send traffic directly through the Arch egress proxy to the LLM of your choice, such as OpenAI.
The OpenAI client is configured to route traffic via Arch by setting the proxy to ``127.0.0.1:51001``, assuming Arch is running locally and bound to that address and port.
This setup allows you to take advantage of Arch's advanced traffic management features while interacting with LLM APIs like OpenAI.