mirror of
https://github.com/katanemo/plano.git
synced 2026-06-29 15:49:40 +02:00
deploy: b30ad791f7
This commit is contained in:
salmanap
parent
f4b686c7fc
commit
3e881c6eec
28 changed files with 819 additions and 820 deletions
|
|
@ -101,9 +101,10 @@
|
|||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/terminology.html">Terminology</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/threading_model.html">Threading Model</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/listener.html">Listener</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/prompt.html">Prompts</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/model_serving.html">Model Serving</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/prompt.html">Prompt</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/request_lifecycle.html">Request Lifecycle</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/error_target.html">Error Target</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../concepts/llm_provider.html">LLM Provider</a></li>
|
||||
|
|
@ -128,7 +129,6 @@
|
|||
<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../resources/configuration_reference.html">Configuration Reference</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../resources/error_target.html">Error Targets</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -160,7 +160,7 @@ you have the flexibility to support “agentic” apps tailored to specific use
|
|||
claims to creating ad campaigns - via prompts.</p>
|
||||
<p>Arch analyzes prompts, extracts critical information from prompts, engages in lightweight conversation with
|
||||
the user to gather any missing parameters and makes API calls so that you can focus on writing business logic.
|
||||
Arch does this via its purpose-built <a class="reference internal" href="../guides/function_calling.html#function-calling"><span class="std std-ref">Arch-FC LLM</span></a> - the fastest (200ms p90 - 10x faser than GPT-4o)
|
||||
Arch does this via its purpose-built <a class="reference internal" href="../guides/function_calling.html#function-calling"><span class="std std-ref">Arch-Function</span></a> - the fastest (200ms p90 - 10x faser than GPT-4o)
|
||||
and cheapest (100x than GPT-40) function-calling LLM that matches performance with frontier models.</p>
|
||||
<a class="reference internal image-reference" href="../_images/function-calling-flow.jpg"><img alt="../_images/function-calling-flow.jpg" class="align-center" src="../_images/function-calling-flow.jpg" style="width: 100%;"/>
|
||||
</a>
|
||||
|
|
@ -169,175 +169,171 @@ and cheapest (100x than GPT-40) function-calling LLM that matches performance wi
|
|||
<p>In the most common scenario, users will request a single action via prompts, and Arch efficiently processes the
|
||||
request by extracting relevant parameters, validating the input, and calling the designated function or API. Here
|
||||
is how you would go about enabling this scenario with Arch:</p>
|
||||
<section id="step-1-define-prompt-targets-with-functions">
|
||||
<h3>Step 1: Define prompt targets with functions<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-1-define-prompt-targets-with-functions" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-1-define-prompt-targets-with-functions'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<div class="literal-block-wrapper docutils container" id="id1">
|
||||
<div class="code-block-caption"><span class="caption-text">Define prompt targets that can enable users to engage with API and backened functions of an app</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id1"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="s">"0.1-beta"</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span><span class="nt">listen</span><span class="p">:</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="w"> </span><span class="nt">address</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">127.0.0.1 | 0.0.0.0</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span><span class="w"> </span><span class="nt">port_value</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">8080</span><span class="w"> </span><span class="c1">#If you configure port 443, you'll need to update the listener with tls_certificates</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="nt">system_prompt</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">|</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="no">You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.</span>
|
||||
<section id="step-1-define-prompt-targets">
|
||||
<h3>Step 1: Define Prompt Targets<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-1-define-prompt-targets" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-1-define-prompt-targets'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<div class="literal-block-wrapper docutils container" id="id2">
|
||||
<div class="code-block-caption"><span class="caption-text">Prompt Target Example Configuration</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id2"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v0.1</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="nt">listen</span><span class="p">:</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span><span class="w"> </span><span class="nt">address</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.0.0.0</span><span class="w"> </span><span class="c1"># or 127.0.0.1</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">10000</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="w"> </span><span class="c1"># Defines how Arch should parse the content from application/json or text/pain Content-type in the http request</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="nt">message_format</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">huggingface</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="nt">llm_providers</span><span class="p">:</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"OpenAI"</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="nt">provider</span><span class="p">:</span><span class="w"> </span><span class="s">"openai"</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">access_key</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OPENAI_API_KEY</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="nt">model</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gpt-4o</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">stream</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span>
|
||||
</span><span id="line-16"><mark><span class="linenos">16</span><span class="nt">prompt_targets</span><span class="p">:</span>
|
||||
</mark></span><span id="line-17"><mark><span class="linenos">17</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">reboot_devices</span>
|
||||
</mark></span><span id="line-18"><mark><span class="linenos">18</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">></span>
|
||||
</mark></span><span id="line-19"><mark><span class="linenos">19</span><span class="w"> </span><span class="no">This prompt target handles user requests to reboot devices.</span>
|
||||
</mark></span><span id="line-20"><mark><span class="linenos">20</span><span class="w"> </span><span class="no">It ensures that when users request to reboot specific devices or device groups, the system processes the reboot commands accurately.</span>
|
||||
</mark></span><span id="line-21"><mark><span class="linenos">21</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span><span class="w"> </span><span class="no">**Examples of user prompts:**</span>
|
||||
</mark></span><span id="line-23"><mark><span class="linenos">23</span>
|
||||
</mark></span><span id="line-24"><mark><span class="linenos">24</span><span class="w"> </span><span class="no">- "Please reboot device 12345."</span>
|
||||
</mark></span><span id="line-25"><mark><span class="linenos">25</span><span class="w"> </span><span class="no">- "Restart all devices in tenant group tenant-XYZ</span>
|
||||
</mark></span><span id="line-26"><mark><span class="linenos">26</span><span class="w"> </span><span class="no">- "I need to reboot devices A, B, and C."</span>
|
||||
</mark></span><span id="line-27"><mark><span class="linenos">27</span>
|
||||
</mark></span><span id="line-28"><mark><span class="linenos">28</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_reboot</span>
|
||||
</mark></span><span id="line-29"><mark><span class="linenos">29</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</mark></span><span id="line-30"><mark><span class="linenos">30</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"device_ids"</span>
|
||||
</mark></span><span id="line-31"><mark><span class="linenos">31</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span><span class="w"> </span><span class="c1"># Options: integer | float | list | dictionary | set</span>
|
||||
</mark></span><span id="line-32"><mark><span class="linenos">32</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"A</span><span class="nv"> </span><span class="s">list</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">identifiers</span><span class="nv"> </span><span class="s">(IDs)</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">reboot."</span>
|
||||
</mark></span><span id="line-33"><mark><span class="linenos">33</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-34"><mark><span class="linenos">34</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"device_group"</span>
|
||||
</mark></span><span id="line-35"><mark><span class="linenos">35</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">string</span><span class="w"> </span><span class="c1"># Options: string | integer | float | list | dictionary | set</span>
|
||||
</mark></span><span id="line-36"><mark><span class="linenos">36</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"The</span><span class="nv"> </span><span class="s">name</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">group</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">reboot."</span>
|
||||
</mark></span><span id="line-37"><mark><span class="linenos">37</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-38"><span class="linenos">38</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span><span class="c1"># Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="nt">endpoints</span><span class="p">:</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span><span class="w"> </span><span class="nt">app_server</span><span class="p">:</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="w"> </span><span class="c1"># value could be ip address or a hostname with port</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span><span class="w"> </span><span class="c1"># this could also be a list of endpoints for load balancing</span>
|
||||
</span><span id="line-44"><span class="linenos">44</span><span class="w"> </span><span class="c1"># for example endpoint: [ ip1:port, ip2:port ]</span>
|
||||
</span><span id="line-45"><span class="linenos">45</span><span class="w"> </span><span class="nt">endpoint</span><span class="p">:</span><span class="w"> </span><span class="s">"127.0.0.1:80"</span>
|
||||
</span><span id="line-46"><span class="linenos">46</span><span class="w"> </span><span class="c1"># max time to wait for a connection to be established</span>
|
||||
</span><span id="line-47"><span class="linenos">47</span><span class="w"> </span><span class="nt">connect_timeout</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.005s</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="c1"># Centralized way to manage LLMs, manage keys, retry logic, failover and limits in a central way</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="nt">llm_providers</span><span class="p">:</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OpenAI</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">provider</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">openai</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="nt">access_key</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OPENAI_API_KEY</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">model</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gpt-4o</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="w"> </span><span class="nt">default</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span><span class="w"> </span><span class="nt">stream</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span><span class="c1"># default system prompt used by all prompt targets</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span><span class="nt">system_prompt</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span>
|
||||
</span><span id="line-21"><mark><span class="linenos">21</span><span class="nt">prompt_targets</span><span class="p">:</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">reboot_devices</span>
|
||||
</mark></span><span id="line-23"><mark><span class="linenos">23</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Reboot specific devices or device groups</span>
|
||||
</mark></span><span id="line-24"><mark><span class="linenos">24</span>
|
||||
</mark></span><span id="line-25"><mark><span class="linenos">25</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_reboot</span>
|
||||
</mark></span><span id="line-26"><mark><span class="linenos">26</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</mark></span><span id="line-27"><mark><span class="linenos">27</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">device_ids</span>
|
||||
</mark></span><span id="line-28"><mark><span class="linenos">28</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span>
|
||||
</mark></span><span id="line-29"><mark><span class="linenos">29</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">A list of device identifiers (IDs) to reboot.</span>
|
||||
</mark></span><span id="line-30"><mark><span class="linenos">30</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-31"><mark><span class="linenos">31</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">device_group</span>
|
||||
</mark></span><span id="line-32"><mark><span class="linenos">32</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">str</span>
|
||||
</mark></span><span id="line-33"><mark><span class="linenos">33</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">The name of the device group to reboot</span>
|
||||
</mark></span><span id="line-34"><mark><span class="linenos">34</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-35"><span class="linenos">35</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span><span class="c1"># Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span><span class="nt">endpoints</span><span class="p">:</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span><span class="w"> </span><span class="nt">app_server</span><span class="p">:</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span><span class="w"> </span><span class="c1"># value could be ip address or a hostname with port</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="w"> </span><span class="c1"># this could also be a list of endpoints for load balancing</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span><span class="w"> </span><span class="c1"># for example endpoint: [ ip1:port, ip2:port ]</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="w"> </span><span class="nt">endpoint</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">127.0.0.1:80</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span><span class="w"> </span><span class="c1"># max time to wait for a connection to be established</span>
|
||||
</span><span id="line-44"><span class="linenos">44</span><span class="w"> </span><span class="nt">connect_timeout</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.005s</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="step-2-process-request-parameters-in-flask">
|
||||
<h3>Step 2: Process request parameters in Flask<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-process-request-parameters-in-flask" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-process-request-parameters-in-flask'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<section id="step-2-process-request-parameters">
|
||||
<h3>Step 2: Process Request Parameters<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-process-request-parameters" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-process-request-parameters'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<p>Once the prompt targets are configured as above, handling those parameters is</p>
|
||||
<div class="literal-block-wrapper docutils container" id="id2">
|
||||
<div class="code-block-caption"><span class="caption-text">Parameter handling with Flask</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id2"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="literal-block-wrapper docutils container" id="id3">
|
||||
<div class="code-block-caption"><span class="caption-text">Parameter handling with Flask</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id3"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">jsonify</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/agent/device_summary'</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s1">'POST'</span><span class="p">])</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="k">def</span> <span class="nf">get_device_summary</span><span class="p">():</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="sd"> Endpoint to retrieve device statistics based on device IDs and an optional time range.</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="sd"> """</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span> <span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_json</span><span class="p">()</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span> <span class="c1"># Validate 'device_ids' parameter</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span> <span class="n">device_ids</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'device_ids'</span><span class="p">)</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">device_ids</span> <span class="ow">or</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">device_ids</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s2">"'device_ids' parameter is required and must be a list"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="c1"># Validate 'time_range' parameter (optional, defaults to 7)</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="n">time_range</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'time_range'</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">time_range</span><span class="p">,</span> <span class="nb">int</span><span class="p">):</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s2">"'time_range' must be an integer"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span>
|
||||
</span><span id="line-22"><span class="linenos">22</span> <span class="c1"># Simulate retrieving statistics for the given device IDs and time range</span>
|
||||
</span><span id="line-23"><span class="linenos">23</span> <span class="c1"># In a real application, you would query your database or external service here</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span> <span class="n">statistics</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="k">for</span> <span class="n">device_id</span> <span class="ow">in</span> <span class="n">device_ids</span><span class="p">:</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="c1"># Placeholder for actual data retrieval</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="n">stats</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="s1">'device_id'</span><span class="p">:</span> <span class="n">device_id</span><span class="p">,</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="s1">'time_range'</span><span class="p">:</span> <span class="sa">f</span><span class="s1">'Last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s1"> days'</span><span class="p">,</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="s1">'data'</span><span class="p">:</span> <span class="sa">f</span><span class="s1">'Statistics data for device </span><span class="si">{</span><span class="n">device_id</span><span class="si">}</span><span class="s1"> over the last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s1"> days.'</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="p">}</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="n">statistics</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span> <span class="n">response</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="s1">'statistics'</span><span class="p">:</span> <span class="n">statistics</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span> <span class="p">}</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">response</span><span class="p">),</span> <span class="mi">200</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/agent/device_summary"</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s2">"POST"</span><span class="p">])</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="k">def</span> <span class="nf">get_device_summary</span><span class="p">():</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="sd"> Endpoint to retrieve device statistics based on device IDs and an optional time range.</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="sd"> """</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span> <span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_json</span><span class="p">()</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span> <span class="c1"># Validate 'device_ids' parameter</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="n">device_ids</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"device_ids"</span><span class="p">)</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">device_ids</span> <span class="ow">or</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">device_ids</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="p">{</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"'device_ids' parameter is required and must be a list"</span><span class="p">}</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="p">),</span> <span class="mi">400</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="c1"># Validate 'time_range' parameter (optional, defaults to 7)</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span> <span class="n">time_range</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"time_range"</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
|
||||
</span><span id="line-22"><span class="linenos">22</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">time_range</span><span class="p">,</span> <span class="nb">int</span><span class="p">):</span>
|
||||
</span><span id="line-23"><span class="linenos">23</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"'time_range' must be an integer"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="c1"># Simulate retrieving statistics for the given device IDs and time range</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="c1"># In a real application, you would query your database or external service here</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="n">statistics</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="k">for</span> <span class="n">device_id</span> <span class="ow">in</span> <span class="n">device_ids</span><span class="p">:</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="c1"># Placeholder for actual data retrieval</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="n">stats</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="s2">"device_id"</span><span class="p">:</span> <span class="n">device_id</span><span class="p">,</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="s2">"time_range"</span><span class="p">:</span> <span class="sa">f</span><span class="s2">"Last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s2"> days"</span><span class="p">,</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span> <span class="s2">"data"</span><span class="p">:</span> <span class="sa">f</span><span class="s2">"Statistics data for device </span><span class="si">{</span><span class="n">device_id</span><span class="si">}</span><span class="s2"> over the last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s2"> days."</span><span class="p">,</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span> <span class="p">}</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="n">statistics</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span> <span class="n">response</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"statistics"</span><span class="p">:</span> <span class="n">statistics</span><span class="p">}</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">response</span><span class="p">),</span> <span class="mi">200</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="parallel-multiple-function-calling">
|
||||
<h2>Parallel/ Multiple Function Calling<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#parallel-multiple-function-calling" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#parallel-multiple-function-calling'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
|
||||
<h2>Parallel & Multiple Function Calling<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#parallel-multiple-function-calling" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#parallel-multiple-function-calling'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
|
||||
<p>In more complex use cases, users may request multiple actions or need multiple APIs/functions to be called
|
||||
simultaneously or sequentially. With Arch, you can handle these scenarios efficiently using parallel or multiple
|
||||
function calling. This allows your application to engage in a broader range of interactions, such as updating
|
||||
different datasets, triggering events across systems, or collecting results from multiple services in one prompt.</p>
|
||||
<p>Arch-FC1B is built to manage these parallel tasks efficiently, ensuring low latency and high throughput, even
|
||||
when multiple functions are invoked. It provides two mechanisms to handle these cases:</p>
|
||||
<section id="step-1-define-multiple-function-targets">
|
||||
<h3>Step 1: Define Multiple Function Targets<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-1-define-multiple-function-targets" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-1-define-multiple-function-targets'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<section id="id1">
|
||||
<h3>Step 1: Define Prompt Targets<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id1" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#id1'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<p>When enabling multiple function calling, define the prompt targets in a way that supports multiple functions or
|
||||
API calls based on the user’s prompt. These targets can be triggered in parallel or sequentially, depending on
|
||||
the user’s intent.</p>
|
||||
<p>Example of Multiple Prompt Targets in YAML:</p>
|
||||
<div class="literal-block-wrapper docutils container" id="id3">
|
||||
<div class="code-block-caption"><span class="caption-text">Define prompt targets that can enable users to engage with API and backened functions of an app</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id3"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="s">"0.1-beta"</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span><span class="nt">listen</span><span class="p">:</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="w"> </span><span class="nt">address</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">127.0.0.1 | 0.0.0.0</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span><span class="w"> </span><span class="nt">port_value</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">8080</span><span class="w"> </span><span class="c1">#If you configure port 443, you'll need to update the listener with tls_certificates</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="nt">system_prompt</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">|</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="no">You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.</span>
|
||||
<div class="literal-block-wrapper docutils container" id="id4">
|
||||
<div class="code-block-caption"><span class="caption-text">Prompt Target Example Configuration</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id4"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v0.1</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="nt">listen</span><span class="p">:</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span><span class="w"> </span><span class="nt">address</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.0.0.0</span><span class="w"> </span><span class="c1"># or 127.0.0.1</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">10000</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="w"> </span><span class="c1"># Defines how Arch should parse the content from application/json or text/pain Content-type in the http request</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="nt">message_format</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">huggingface</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="nt">llm_providers</span><span class="p">:</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"OpenAI"</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="nt">provider</span><span class="p">:</span><span class="w"> </span><span class="s">"openai"</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">access_key</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OPENAI_API_KEY</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="nt">model</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gpt-4o</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">stream</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span>
|
||||
</span><span id="line-16"><mark><span class="linenos">16</span><span class="nt">prompt_targets</span><span class="p">:</span>
|
||||
</mark></span><span id="line-17"><mark><span class="linenos">17</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">reboot_devices</span>
|
||||
</mark></span><span id="line-18"><mark><span class="linenos">18</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">></span>
|
||||
</mark></span><span id="line-19"><mark><span class="linenos">19</span><span class="w"> </span><span class="no">This prompt target handles user requests to reboot devices.</span>
|
||||
</mark></span><span id="line-20"><mark><span class="linenos">20</span><span class="w"> </span><span class="no">It ensures that when users request to reboot specific devices or device groups, the system processes the reboot commands accurately.</span>
|
||||
</mark></span><span id="line-21"><mark><span class="linenos">21</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span><span class="w"> </span><span class="no">**Examples of user prompts:**</span>
|
||||
</mark></span><span id="line-23"><mark><span class="linenos">23</span>
|
||||
</mark></span><span id="line-24"><mark><span class="linenos">24</span><span class="w"> </span><span class="no">- "Please reboot device 12345."</span>
|
||||
</mark></span><span id="line-25"><mark><span class="linenos">25</span><span class="w"> </span><span class="no">- "Restart all devices in tenant group tenant-XYZ</span>
|
||||
</mark></span><span id="line-26"><mark><span class="linenos">26</span><span class="w"> </span><span class="no">- "I need to reboot devices A, B, and C."</span>
|
||||
</mark></span><span id="line-27"><mark><span class="linenos">27</span>
|
||||
</mark></span><span id="line-28"><mark><span class="linenos">28</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_reboot</span>
|
||||
</mark></span><span id="line-29"><mark><span class="linenos">29</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</mark></span><span id="line-30"><mark><span class="linenos">30</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"device_ids"</span>
|
||||
</mark></span><span id="line-31"><mark><span class="linenos">31</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span><span class="w"> </span><span class="c1"># Options: integer | float | list | dictionary | set</span>
|
||||
</mark></span><span id="line-32"><mark><span class="linenos">32</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"A</span><span class="nv"> </span><span class="s">list</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">identifiers</span><span class="nv"> </span><span class="s">(IDs)</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">reboot."</span>
|
||||
</mark></span><span id="line-33"><mark><span class="linenos">33</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-34"><mark><span class="linenos">34</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"device_group"</span>
|
||||
</mark></span><span id="line-35"><mark><span class="linenos">35</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">string</span><span class="w"> </span><span class="c1"># Options: string | integer | float | list | dictionary | set</span>
|
||||
</mark></span><span id="line-36"><mark><span class="linenos">36</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"The</span><span class="nv"> </span><span class="s">name</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">group</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">reboot."</span>
|
||||
</mark></span><span id="line-37"><mark><span class="linenos">37</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-38"><span class="linenos">38</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span><span class="c1"># Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="nt">endpoints</span><span class="p">:</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span><span class="w"> </span><span class="nt">app_server</span><span class="p">:</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="w"> </span><span class="c1"># value could be ip address or a hostname with port</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span><span class="w"> </span><span class="c1"># this could also be a list of endpoints for load balancing</span>
|
||||
</span><span id="line-44"><span class="linenos">44</span><span class="w"> </span><span class="c1"># for example endpoint: [ ip1:port, ip2:port ]</span>
|
||||
</span><span id="line-45"><span class="linenos">45</span><span class="w"> </span><span class="nt">endpoint</span><span class="p">:</span><span class="w"> </span><span class="s">"127.0.0.1:80"</span>
|
||||
</span><span id="line-46"><span class="linenos">46</span><span class="w"> </span><span class="c1"># max time to wait for a connection to be established</span>
|
||||
</span><span id="line-47"><span class="linenos">47</span><span class="w"> </span><span class="nt">connect_timeout</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.005s</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="c1"># Centralized way to manage LLMs, manage keys, retry logic, failover and limits in a central way</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="nt">llm_providers</span><span class="p">:</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OpenAI</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">provider</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">openai</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="nt">access_key</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">OPENAI_API_KEY</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">model</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gpt-4o</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="w"> </span><span class="nt">default</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span><span class="w"> </span><span class="nt">stream</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span><span class="c1"># default system prompt used by all prompt targets</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span><span class="nt">system_prompt</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span>
|
||||
</span><span id="line-21"><mark><span class="linenos">21</span><span class="nt">prompt_targets</span><span class="p">:</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">reboot_devices</span>
|
||||
</mark></span><span id="line-23"><mark><span class="linenos">23</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Reboot specific devices or device groups</span>
|
||||
</mark></span><span id="line-24"><mark><span class="linenos">24</span>
|
||||
</mark></span><span id="line-25"><mark><span class="linenos">25</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_reboot</span>
|
||||
</mark></span><span id="line-26"><mark><span class="linenos">26</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</mark></span><span id="line-27"><mark><span class="linenos">27</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">device_ids</span>
|
||||
</mark></span><span id="line-28"><mark><span class="linenos">28</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span>
|
||||
</mark></span><span id="line-29"><mark><span class="linenos">29</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">A list of device identifiers (IDs) to reboot.</span>
|
||||
</mark></span><span id="line-30"><mark><span class="linenos">30</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-31"><mark><span class="linenos">31</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">device_group</span>
|
||||
</mark></span><span id="line-32"><mark><span class="linenos">32</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">str</span>
|
||||
</mark></span><span id="line-33"><mark><span class="linenos">33</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">The name of the device group to reboot</span>
|
||||
</mark></span><span id="line-34"><mark><span class="linenos">34</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</mark></span><span id="line-35"><span class="linenos">35</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span><span class="c1"># Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span><span class="nt">endpoints</span><span class="p">:</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span><span class="w"> </span><span class="nt">app_server</span><span class="p">:</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span><span class="w"> </span><span class="c1"># value could be ip address or a hostname with port</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="w"> </span><span class="c1"># this could also be a list of endpoints for load balancing</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span><span class="w"> </span><span class="c1"># for example endpoint: [ ip1:port, ip2:port ]</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="w"> </span><span class="nt">endpoint</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">127.0.0.1:80</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span><span class="w"> </span><span class="c1"># max time to wait for a connection to be established</span>
|
||||
</span><span id="line-44"><span class="linenos">44</span><span class="w"> </span><span class="nt">connect_timeout</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0.005s</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -365,12 +361,12 @@ the user’s intent.</p>
|
|||
<div class="sticky top-16 -mt-10 max-h-[calc(100vh-5rem)] overflow-y-auto pt-6 space-y-2"><p class="font-medium">On this page</p>
|
||||
<ul>
|
||||
<li><a :data-current="activeSection === '#single-function-call'" class="reference internal" href="#single-function-call">Single Function Call</a><ul>
|
||||
<li><a :data-current="activeSection === '#step-1-define-prompt-targets-with-functions'" class="reference internal" href="#step-1-define-prompt-targets-with-functions">Step 1: Define prompt targets with functions</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-process-request-parameters-in-flask'" class="reference internal" href="#step-2-process-request-parameters-in-flask">Step 2: Process request parameters in Flask</a></li>
|
||||
<li><a :data-current="activeSection === '#step-1-define-prompt-targets'" class="reference internal" href="#step-1-define-prompt-targets">Step 1: Define Prompt Targets</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-process-request-parameters'" class="reference internal" href="#step-2-process-request-parameters">Step 2: Process Request Parameters</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a :data-current="activeSection === '#parallel-multiple-function-calling'" class="reference internal" href="#parallel-multiple-function-calling">Parallel/ Multiple Function Calling</a><ul>
|
||||
<li><a :data-current="activeSection === '#step-1-define-multiple-function-targets'" class="reference internal" href="#step-1-define-multiple-function-targets">Step 1: Define Multiple Function Targets</a></li>
|
||||
<li><a :data-current="activeSection === '#parallel-multiple-function-calling'" class="reference internal" href="#parallel-multiple-function-calling">Parallel & Multiple Function Calling</a><ul>
|
||||
<li><a :data-current="activeSection === '#id1'" class="reference internal" href="#id1">Step 1: Define Prompt Targets</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
|
|
|||
|
|
@ -101,9 +101,10 @@
|
|||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/terminology.html">Terminology</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/threading_model.html">Threading Model</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/listener.html">Listener</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/prompt.html">Prompts</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/model_serving.html">Model Serving</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/prompt.html">Prompt</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/request_lifecycle.html">Request Lifecycle</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="../concepts/tech_overview/error_target.html">Error Target</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../concepts/llm_provider.html">LLM Provider</a></li>
|
||||
|
|
@ -128,7 +129,6 @@
|
|||
<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
|
||||
<ul>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../resources/configuration_reference.html">Configuration Reference</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../resources/error_target.html">Error Targets</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
|
@ -157,49 +157,72 @@
|
|||
Retrieval-Augmented Generation (RAG) applications.</p>
|
||||
<section id="intent-drift-detection">
|
||||
<h2>Intent-drift Detection<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#intent-drift-detection" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#intent-drift-detection'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
|
||||
<p>Developers struggle to handle <a class="reference external" href="https://www.reddit.com/r/ChatGPTPromptGenius/comments/17dzmpy/how_to_use_rag_with_conversation_history_for/?" rel="nofollow noopener">follow-up<svg fill="currentColor" height="1em" stroke="none" viewbox="0 96 960 960" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M188 868q-11-11-11-28t11-28l436-436H400q-17 0-28.5-11.5T360 336q0-17 11.5-28.5T400 296h320q17 0 28.5 11.5T760 336v320q0 17-11.5 28.5T720 696q-17 0-28.5-11.5T680 656V432L244 868q-11 11-28 11t-28-11Z"></path></svg></a>
|
||||
or <a class="reference external" href="https://www.reddit.com/r/LocalLLaMA/comments/18mqwg6/best_practice_for_rag_with_followup_chat/" rel="nofollow noopener">clarifying<svg fill="currentColor" height="1em" stroke="none" viewbox="0 96 960 960" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M188 868q-11-11-11-28t11-28l436-436H400q-17 0-28.5-11.5T360 336q0-17 11.5-28.5T400 296h320q17 0 28.5 11.5T760 336v320q0 17-11.5 28.5T720 696q-17 0-28.5-11.5T680 656V432L244 868q-11 11-28 11t-28-11Z"></path></svg></a>
|
||||
questions. Specifically, when users ask for changes or additions to previous responses their AI applications often
|
||||
generate entirely new responses instead of adjusting previous ones. Arch offers <em>intent-drift</em> tracking as a feature so
|
||||
that developers can know when the user has shifted away from a previous intent so that they can dramatically improve
|
||||
retrieval accuracy, lower overall token cost and improve the speed of their responses back to users.</p>
|
||||
<p>Developers struggle to handle <code class="docutils literal notranslate"><span class="pre">follow-up</span></code> or <code class="docutils literal notranslate"><span class="pre">clarification</span></code> questions.
|
||||
Specifically, when users ask for changes or additions to previous responses their AI applications often generate entirely new responses instead of adjusting previous ones.
|
||||
Arch offers <strong>intent-drift</strong> tracking as a feature so that developers can know when the user has shifted away from a previous intent so that they can dramatically improve retrieval accuracy, lower overall token cost and improve the speed of their responses back to users.</p>
|
||||
<p>Arch uses its built-in lightweight NLI and embedding models to know if the user has steered away from an active intent.
|
||||
Arch’s intent-drift detection mechanism is based on its’ <em>prompt_targets</em> primtive. Arch tries to match an incoming
|
||||
prompt to one of the <em>prompt_targets</em> configured in the gateway. Once it detects that the user has moved away from an active
|
||||
Arch’s intent-drift detection mechanism is based on its’ <a class="reference internal" href="../concepts/prompt_target.html#prompt-target"><span class="std std-ref">prompt_targets</span></a> primtive. Arch tries to match an incoming
|
||||
prompt to one of the prompt_targets configured in the gateway. Once it detects that the user has moved away from an active
|
||||
active intent, Arch adds the <code class="docutils literal notranslate"><span class="pre">x-arch-intent-drift</span></code> headers to the request before sending it your application servers.</p>
|
||||
<div class="literal-block-wrapper docutils container" id="id1">
|
||||
<div class="code-block-caption"><span class="caption-text">Intent Detection Example</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id1"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/process_rag'</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s1">'POST'</span><span class="p">])</span>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/process_rag"</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s2">"POST"</span><span class="p">])</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span><span class="k">def</span> <span class="nf">process_rag</span><span class="p">():</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span> <span class="c1"># Extract JSON data from the request</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span> <span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_json</span><span class="p">()</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span> <span class="n">user_id</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'user_id'</span><span class="p">)</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span> <span class="n">user_id</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"user_id"</span><span class="p">)</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">user_id</span><span class="p">:</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s1">'User ID is required'</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"User ID is required"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span> <span class="n">client_messages</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'messages'</span><span class="p">)</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span> <span class="n">client_messages</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"messages"</span><span class="p">)</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">client_messages</span> <span class="ow">or</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">client_messages</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s1">'Messages array is required'</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"Messages array is required"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span>
|
||||
</span><span id="line-14"><mark><span class="linenos">14</span> <span class="c1"># Extract the intent change marker from Arch's headers if present for the current prompt</span>
|
||||
</mark></span><span id="line-15"><mark><span class="linenos">15</span> <span class="n">intent_changed_header</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'x-arch-intent-marker'</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
|
||||
</mark></span><span id="line-16"><mark><span class="linenos">16</span> <span class="k">if</span> <span class="n">intent_changed_header</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">''</span><span class="p">,</span> <span class="s1">'false'</span><span class="p">]:</span>
|
||||
</mark></span><span id="line-15"><mark><span class="linenos">15</span> <span class="n">intent_changed_header</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"x-arch-intent-marker"</span><span class="p">,</span> <span class="s2">""</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
|
||||
</mark></span><span id="line-16"><mark><span class="linenos">16</span> <span class="k">if</span> <span class="n">intent_changed_header</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">""</span><span class="p">,</span> <span class="s2">"false"</span><span class="p">]:</span>
|
||||
</mark></span><span id="line-17"><mark><span class="linenos">17</span> <span class="n">intent_changed</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</mark></span><span id="line-18"><mark><span class="linenos">18</span> <span class="k">elif</span> <span class="n">intent_changed_header</span> <span class="o">==</span> <span class="s1">'true'</span><span class="p">:</span>
|
||||
</mark></span><span id="line-18"><mark><span class="linenos">18</span> <span class="k">elif</span> <span class="n">intent_changed_header</span> <span class="o">==</span> <span class="s2">"true"</span><span class="p">:</span>
|
||||
</mark></span><span id="line-19"><mark><span class="linenos">19</span> <span class="n">intent_changed</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</mark></span><span id="line-20"><mark><span class="linenos">20</span> <span class="k">else</span><span class="p">:</span>
|
||||
</mark></span><span id="line-21"><mark><span class="linenos">21</span> <span class="c1"># Invalid value provided</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s1">'Invalid value for x-arch-prompt-intent-change header'</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</mark></span><span id="line-23"><span class="linenos">23</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span> <span class="c1"># Update user conversation based on intent change</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="n">memory</span> <span class="o">=</span> <span class="n">update_user_conversation</span><span class="p">(</span><span class="n">user_id</span><span class="p">,</span> <span class="n">client_messages</span><span class="p">,</span> <span class="n">intent_changed</span><span class="p">)</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="c1"># Retrieve messages since last intent change for LLM</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="n">messages_for_llm</span> <span class="o">=</span> <span class="n">get_messages_since_last_intent</span><span class="p">(</span><span class="n">memory</span><span class="o">.</span><span class="n">chat_memory</span><span class="o">.</span><span class="n">messages</span><span class="p">)</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="c1"># Forward messages to upstream LLM</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="n">llm_response</span> <span class="o">=</span> <span class="n">forward_to_llm</span><span class="p">(</span><span class="n">messages_for_llm</span><span class="p">)</span>
|
||||
</mark></span><span id="line-22"><mark><span class="linenos">22</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span>
|
||||
</mark></span><span id="line-23"><mark><span class="linenos">23</span> <span class="p">{</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"Invalid value for x-arch-prompt-intent-change header"</span><span class="p">}</span>
|
||||
</mark></span><span id="line-24"><mark><span class="linenos">24</span> <span class="p">),</span> <span class="mi">400</span>
|
||||
</mark></span><span id="line-25"><span class="linenos">25</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="c1"># Update user conversation based on intent change</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="n">memory</span> <span class="o">=</span> <span class="n">update_user_conversation</span><span class="p">(</span><span class="n">user_id</span><span class="p">,</span> <span class="n">client_messages</span><span class="p">,</span> <span class="n">intent_changed</span><span class="p">)</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="c1"># Retrieve messages since last intent change for LLM</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="n">messages_for_llm</span> <span class="o">=</span> <span class="n">get_messages_since_last_intent</span><span class="p">(</span><span class="n">memory</span><span class="o">.</span><span class="n">chat_memory</span><span class="o">.</span><span class="n">messages</span><span class="p">)</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="c1"># Forward messages to upstream LLM</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span> <span class="n">llm_response</span> <span class="o">=</span> <span class="n">forward_to_llm</span><span class="p">(</span><span class="n">messages_for_llm</span><span class="p">)</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="c1"># Prepare the messages to return</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span> <span class="n">messages_to_return</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span> <span class="k">for</span> <span class="n">message</span> <span class="ow">in</span> <span class="n">memory</span><span class="o">.</span><span class="n">chat_memory</span><span class="o">.</span><span class="n">messages</span><span class="p">:</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span> <span class="n">role</span> <span class="o">=</span> <span class="s2">"user"</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">HumanMessage</span><span class="p">)</span> <span class="k">else</span> <span class="s2">"assistant"</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span> <span class="n">content</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">content</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span> <span class="n">metadata</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">additional_kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"metadata"</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span> <span class="n">message_entry</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span> <span class="s2">"uuid"</span><span class="p">:</span> <span class="n">metadata</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"uuid"</span><span class="p">),</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span> <span class="s2">"timestamp"</span><span class="p">:</span> <span class="n">metadata</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"timestamp"</span><span class="p">),</span>
|
||||
</span><span id="line-44"><span class="linenos">44</span> <span class="s2">"role"</span><span class="p">:</span> <span class="n">role</span><span class="p">,</span>
|
||||
</span><span id="line-45"><span class="linenos">45</span> <span class="s2">"content"</span><span class="p">:</span> <span class="n">content</span><span class="p">,</span>
|
||||
</span><span id="line-46"><span class="linenos">46</span> <span class="s2">"intent_changed"</span><span class="p">:</span> <span class="n">metadata</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"intent_changed"</span><span class="p">,</span> <span class="kc">False</span><span class="p">),</span>
|
||||
</span><span id="line-47"><span class="linenos">47</span> <span class="p">}</span>
|
||||
</span><span id="line-48"><span class="linenos">48</span> <span class="n">messages_to_return</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">message_entry</span><span class="p">)</span>
|
||||
</span><span id="line-49"><span class="linenos">49</span>
|
||||
</span><span id="line-50"><span class="linenos">50</span> <span class="c1"># Prepare the response</span>
|
||||
</span><span id="line-51"><span class="linenos">51</span> <span class="n">response</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-52"><span class="linenos">52</span> <span class="s2">"user_id"</span><span class="p">:</span> <span class="n">user_id</span><span class="p">,</span>
|
||||
</span><span id="line-53"><span class="linenos">53</span> <span class="s2">"messages"</span><span class="p">:</span> <span class="n">messages_to_return</span><span class="p">,</span>
|
||||
</span><span id="line-54"><span class="linenos">54</span> <span class="s2">"llm_response"</span><span class="p">:</span> <span class="n">llm_response</span><span class="p">,</span>
|
||||
</span><span id="line-55"><span class="linenos">55</span> <span class="p">}</span>
|
||||
</span><span id="line-56"><span class="linenos">56</span>
|
||||
</span><span id="line-57"><span class="linenos">57</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">response</span><span class="p">),</span> <span class="mi">200</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -224,20 +247,20 @@ so that you can use the most relevant prompts for your retrieval and for prompti
|
|||
</span><span id="line-10"><span class="linenos">10</span><span class="c1"># Global dictionary to keep track of user memories</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="n">user_memories</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="k">def</span> <span class="nf">get_user_conversation</span><span class="p">(</span><span class="n">user_id</span><span class="p">):</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="sd"> Retrieve the user's conversation memory using LangChain.</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span><span class="sd"> If the user does not exist, initialize their conversation memory.</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span><span class="sd"> """</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="k">if</span> <span class="n">user_id</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">user_memories</span><span class="p">:</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="n">user_memories</span><span class="p">[</span><span class="n">user_id</span><span class="p">]</span> <span class="o">=</span> <span class="n">ConversationBufferMemory</span><span class="p">(</span><span class="n">return_messages</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="k">return</span> <span class="n">user_memories</span><span class="p">[</span><span class="n">user_id</span><span class="p">]</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="k">def</span> <span class="nf">get_user_conversation</span><span class="p">(</span><span class="n">user_id</span><span class="p">):</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span><span class="sd"> Retrieve the user's conversation memory using LangChain.</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span><span class="sd"> If the user does not exist, initialize their conversation memory.</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span><span class="sd"> """</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="k">if</span> <span class="n">user_id</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">user_memories</span><span class="p">:</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="n">user_memories</span><span class="p">[</span><span class="n">user_id</span><span class="p">]</span> <span class="o">=</span> <span class="n">ConversationBufferMemory</span><span class="p">(</span><span class="n">return_messages</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span> <span class="k">return</span> <span class="n">user_memories</span><span class="p">[</span><span class="n">user_id</span><span class="p">]</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="step-2-update-conversationbuffermemory-w-intent">
|
||||
<h3>Step 2: Update ConversationBufferMemory w/ intent<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-update-conversationbuffermemory-w-intent" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-update-conversationbuffermemory-w-intent'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<section id="step-2-update-conversationbuffermemory-with-intents">
|
||||
<h3>Step 2: Update ConversationBufferMemory with Intents<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-update-conversationbuffermemory-with-intents" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-update-conversationbuffermemory-with-intents'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="k">def</span> <span class="nf">update_user_conversation</span><span class="p">(</span><span class="n">user_id</span><span class="p">,</span> <span class="n">client_messages</span><span class="p">,</span> <span class="n">intent_changed</span><span class="p">):</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="sd"> Update the user's conversation memory with new messages using LangChain.</span>
|
||||
|
|
@ -253,26 +276,26 @@ so that you can use the most relevant prompts for your retrieval and for prompti
|
|||
</span><span id="line-13"><span class="linenos">13</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="c1"># Process each new message</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span> <span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">message</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">new_messages</span><span class="p">):</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span> <span class="n">role</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'role'</span><span class="p">)</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="n">content</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'content'</span><span class="p">)</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span> <span class="n">role</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"role"</span><span class="p">)</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="n">content</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"content"</span><span class="p">)</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="n">metadata</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="s1">'uuid'</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()),</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="s1">'timestamp'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">utcnow</span><span class="p">()</span><span class="o">.</span><span class="n">isoformat</span><span class="p">(),</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span> <span class="s1">'intent_changed'</span><span class="p">:</span> <span class="kc">False</span> <span class="c1"># Default value</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="s2">"uuid"</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()),</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="s2">"timestamp"</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">utcnow</span><span class="p">()</span><span class="o">.</span><span class="n">isoformat</span><span class="p">(),</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span> <span class="s2">"intent_changed"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> <span class="c1"># Default value</span>
|
||||
</span><span id="line-22"><span class="linenos">22</span> <span class="p">}</span>
|
||||
</span><span id="line-23"><span class="linenos">23</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span> <span class="c1"># Mark the intent change on the last message if detected</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="k">if</span> <span class="n">intent_changed</span> <span class="ow">and</span> <span class="n">index</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">new_messages</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="n">metadata</span><span class="p">[</span><span class="s1">'intent_changed'</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="n">metadata</span><span class="p">[</span><span class="s2">"intent_changed"</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="c1"># Create a new message with metadata</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="k">if</span> <span class="n">role</span> <span class="o">==</span> <span class="s1">'user'</span><span class="p">:</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="k">if</span> <span class="n">role</span> <span class="o">==</span> <span class="s2">"user"</span><span class="p">:</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="n">memory</span><span class="o">.</span><span class="n">chat_memory</span><span class="o">.</span><span class="n">add_message</span><span class="p">(</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="n">HumanMessage</span><span class="p">(</span><span class="n">content</span><span class="o">=</span><span class="n">content</span><span class="p">,</span> <span class="n">additional_kwargs</span><span class="o">=</span><span class="p">{</span><span class="s1">'metadata'</span><span class="p">:</span> <span class="n">metadata</span><span class="p">})</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="n">HumanMessage</span><span class="p">(</span><span class="n">content</span><span class="o">=</span><span class="n">content</span><span class="p">,</span> <span class="n">additional_kwargs</span><span class="o">=</span><span class="p">{</span><span class="s2">"metadata"</span><span class="p">:</span> <span class="n">metadata</span><span class="p">})</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="p">)</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span> <span class="k">elif</span> <span class="n">role</span> <span class="o">==</span> <span class="s1">'assistant'</span><span class="p">:</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span> <span class="k">elif</span> <span class="n">role</span> <span class="o">==</span> <span class="s2">"assistant"</span><span class="p">:</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span> <span class="n">memory</span><span class="o">.</span><span class="n">chat_memory</span><span class="o">.</span><span class="n">add_message</span><span class="p">(</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="n">AIMessage</span><span class="p">(</span><span class="n">content</span><span class="o">=</span><span class="n">content</span><span class="p">,</span> <span class="n">additional_kwargs</span><span class="o">=</span><span class="p">{</span><span class="s1">'metadata'</span><span class="p">:</span> <span class="n">metadata</span><span class="p">})</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="n">AIMessage</span><span class="p">(</span><span class="n">content</span><span class="o">=</span><span class="n">content</span><span class="p">,</span> <span class="n">additional_kwargs</span><span class="o">=</span><span class="p">{</span><span class="s2">"metadata"</span><span class="p">:</span> <span class="n">metadata</span><span class="p">})</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span> <span class="p">)</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span> <span class="c1"># Handle other roles if necessary</span>
|
||||
|
|
@ -292,11 +315,12 @@ so that you can use the most relevant prompts for your retrieval and for prompti
|
|||
</span><span id="line-6"><span class="linenos"> 6</span> <span class="k">for</span> <span class="n">message</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">messages</span><span class="p">):</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span> <span class="c1"># Insert message at the beginning to maintain correct order</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span> <span class="n">messages_since_intent</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">message</span><span class="p">)</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span> <span class="n">metadata</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">additional_kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'metadata'</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span> <span class="n">metadata</span> <span class="o">=</span> <span class="n">message</span><span class="o">.</span><span class="n">additional_kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"metadata"</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span> <span class="c1"># Break if intent_changed is True</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span> <span class="k">if</span> <span class="n">metadata</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'intent_changed'</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span> <span class="o">==</span> <span class="kc">True</span><span class="p">:</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span> <span class="k">if</span> <span class="n">metadata</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"intent_changed"</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span> <span class="o">==</span> <span class="kc">True</span><span class="p">:</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span> <span class="k">break</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span> <span class="k">return</span> <span class="n">messages_since_intent</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="k">return</span> <span class="n">messages_since_intent</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
<p>You can used the last set of messages that match to an intent to prompt an LLM, use it with an vector-DB for
|
||||
|
|
@ -311,37 +335,31 @@ enabling Arch to retrieve critical information in a structured way for processin
|
|||
retrieval quality and speed of your application. By extracting parameters from the conversation, you can pull
|
||||
the appropriate chunks from a vector database or SQL-like data store to enhance accuracy. With Arch, you can
|
||||
streamline data retrieval and processing to build more efficient and precise RAG applications.</p>
|
||||
<section id="step-1-define-prompt-targets-with-parameter-definitions">
|
||||
<h3>Step 1: Define prompt targets with parameter definitions<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-1-define-prompt-targets-with-parameter-definitions" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-1-define-prompt-targets-with-parameter-definitions'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<section id="step-1-define-prompt-targets">
|
||||
<h3>Step 1: Define Prompt Targets<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-1-define-prompt-targets" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-1-define-prompt-targets'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<div class="literal-block-wrapper docutils container" id="id2">
|
||||
<div class="code-block-caption"><span class="caption-text">Prompt Targets</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id2"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="linenos"> 1</span><span class="nt">prompt_targets</span><span class="p">:</span>
|
||||
</span><span id="line-2"><span class="linenos"> 2</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">get_device_statistics</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">></span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span><span class="w"> </span><span class="no">This prompt target ensures that when users request device-related statistics, the system accurately retrieves and presents the relevant data</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="w"> </span><span class="no">based on the specified devices and time range. Examples of user prompts, include:</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="no">- "Show me the performance stats for device 12345 over the past week."</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="w"> </span><span class="no">- "What are the error rates for my devices in the last 24 hours?"</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="w"> </span><span class="no">- "I need statistics on device 789 over the last 10 days."</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_summary</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"device_ids"</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span><span class="w"> </span><span class="c1"># Options: integer | float | list | dictionary | set</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"A</span><span class="nv"> </span><span class="s">list</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">identifiers</span><span class="nv"> </span><span class="s">(IDs)</span><span class="nv"> </span><span class="s">for</span><span class="nv"> </span><span class="s">which</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">statistics</span><span class="nv"> </span><span class="s">are</span><span class="nv"> </span><span class="s">requested."</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s">"time_range"</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">integer</span><span class="w"> </span><span class="c1"># Options: integer | float | list | dictionary | set</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">"The</span><span class="nv"> </span><span class="s">number</span><span class="nv"> </span><span class="s">of</span><span class="nv"> </span><span class="s">days</span><span class="nv"> </span><span class="s">in</span><span class="nv"> </span><span class="s">the</span><span class="nv"> </span><span class="s">past</span><span class="nv"> </span><span class="s">over</span><span class="nv"> </span><span class="s">which</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">retrieve</span><span class="nv"> </span><span class="s">device</span><span class="nv"> </span><span class="s">statistics.</span><span class="nv"> </span><span class="s">Defaults</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">7</span><span class="nv"> </span><span class="s">days</span><span class="nv"> </span><span class="s">if</span><span class="nv"> </span><span class="s">not</span><span class="nv"> </span><span class="s">specified."</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span><span class="w"> </span><span class="nt">default</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">7</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Retrieve and present the relevant data based on the specified devices and time range</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/agent/device_summary</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="w"> </span><span class="nt">parameters</span><span class="p">:</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">device_ids</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">list</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">A list of device identifiers (IDs) to reboot.</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">true</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">time_range</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span><span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">int</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span><span class="w"> </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">The number of days in the past over which to retrieve device statistics</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span><span class="w"> </span><span class="nt">required</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">false</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span><span class="w"> </span><span class="nt">default</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">7</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="step-2-process-request-parameters-in-flask">
|
||||
<h3>Step 2: Process request parameters in Flask<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-process-request-parameters-in-flask" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-process-request-parameters-in-flask'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<h3>Step 2: Process Request Parameters in Flask<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#step-2-process-request-parameters-in-flask" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#step-2-process-request-parameters-in-flask'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<p>Once the prompt targets are configured as above, handling those parameters is</p>
|
||||
<div class="literal-block-wrapper docutils container" id="id3">
|
||||
<div class="code-block-caption"><span class="caption-text">Parameter handling with Flask</span><a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#id3"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></div>
|
||||
|
|
@ -349,43 +367,45 @@ streamline data retrieval and processing to build more efficient and precise RAG
|
|||
</span><span id="line-2"><span class="linenos"> 2</span>
|
||||
</span><span id="line-3"><span class="linenos"> 3</span><span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
|
||||
</span><span id="line-4"><span class="linenos"> 4</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">'/agent/device_summary'</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s1">'POST'</span><span class="p">])</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="k">def</span> <span class="nf">get_device_summary</span><span class="p">():</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="sd"> Endpoint to retrieve device statistics based on device IDs and an optional time range.</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="sd"> """</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span> <span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_json</span><span class="p">()</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span> <span class="c1"># Validate 'device_ids' parameter</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span> <span class="n">device_ids</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'device_ids'</span><span class="p">)</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">device_ids</span> <span class="ow">or</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">device_ids</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s2">"'device_ids' parameter is required and must be a list"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="c1"># Validate 'time_range' parameter (optional, defaults to 7)</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="n">time_range</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'time_range'</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">time_range</span><span class="p">,</span> <span class="nb">int</span><span class="p">):</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s1">'error'</span><span class="p">:</span> <span class="s2">"'time_range' must be an integer"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span>
|
||||
</span><span id="line-22"><span class="linenos">22</span> <span class="c1"># Simulate retrieving statistics for the given device IDs and time range</span>
|
||||
</span><span id="line-23"><span class="linenos">23</span> <span class="c1"># In a real application, you would query your database or external service here</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span> <span class="n">statistics</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="k">for</span> <span class="n">device_id</span> <span class="ow">in</span> <span class="n">device_ids</span><span class="p">:</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="c1"># Placeholder for actual data retrieval</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="n">stats</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="s1">'device_id'</span><span class="p">:</span> <span class="n">device_id</span><span class="p">,</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="s1">'time_range'</span><span class="p">:</span> <span class="sa">f</span><span class="s1">'Last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s1"> days'</span><span class="p">,</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="s1">'data'</span><span class="p">:</span> <span class="sa">f</span><span class="s1">'Statistics data for device </span><span class="si">{</span><span class="n">device_id</span><span class="si">}</span><span class="s1"> over the last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s1"> days.'</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="p">}</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="n">statistics</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span> <span class="n">response</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="s1">'statistics'</span><span class="p">:</span> <span class="n">statistics</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span> <span class="p">}</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">response</span><span class="p">),</span> <span class="mi">200</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="line-5"><span class="linenos"> 5</span>
|
||||
</span><span id="line-6"><span class="linenos"> 6</span><span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/agent/device_summary"</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s2">"POST"</span><span class="p">])</span>
|
||||
</span><span id="line-7"><span class="linenos"> 7</span><span class="k">def</span> <span class="nf">get_device_summary</span><span class="p">():</span>
|
||||
</span><span id="line-8"><span class="linenos"> 8</span><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="line-9"><span class="linenos"> 9</span><span class="sd"> Endpoint to retrieve device statistics based on device IDs and an optional time range.</span>
|
||||
</span><span id="line-10"><span class="linenos">10</span><span class="sd"> """</span>
|
||||
</span><span id="line-11"><span class="linenos">11</span> <span class="n">data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get_json</span><span class="p">()</span>
|
||||
</span><span id="line-12"><span class="linenos">12</span>
|
||||
</span><span id="line-13"><span class="linenos">13</span> <span class="c1"># Validate 'device_ids' parameter</span>
|
||||
</span><span id="line-14"><span class="linenos">14</span> <span class="n">device_ids</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"device_ids"</span><span class="p">)</span>
|
||||
</span><span id="line-15"><span class="linenos">15</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">device_ids</span> <span class="ow">or</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">device_ids</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="line-16"><span class="linenos">16</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span>
|
||||
</span><span id="line-17"><span class="linenos">17</span> <span class="p">{</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"'device_ids' parameter is required and must be a list"</span><span class="p">}</span>
|
||||
</span><span id="line-18"><span class="linenos">18</span> <span class="p">),</span> <span class="mi">400</span>
|
||||
</span><span id="line-19"><span class="linenos">19</span>
|
||||
</span><span id="line-20"><span class="linenos">20</span> <span class="c1"># Validate 'time_range' parameter (optional, defaults to 7)</span>
|
||||
</span><span id="line-21"><span class="linenos">21</span> <span class="n">time_range</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"time_range"</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
|
||||
</span><span id="line-22"><span class="linenos">22</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">time_range</span><span class="p">,</span> <span class="nb">int</span><span class="p">):</span>
|
||||
</span><span id="line-23"><span class="linenos">23</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s2">"error"</span><span class="p">:</span> <span class="s2">"'time_range' must be an integer"</span><span class="p">}),</span> <span class="mi">400</span>
|
||||
</span><span id="line-24"><span class="linenos">24</span>
|
||||
</span><span id="line-25"><span class="linenos">25</span> <span class="c1"># Simulate retrieving statistics for the given device IDs and time range</span>
|
||||
</span><span id="line-26"><span class="linenos">26</span> <span class="c1"># In a real application, you would query your database or external service here</span>
|
||||
</span><span id="line-27"><span class="linenos">27</span> <span class="n">statistics</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="line-28"><span class="linenos">28</span> <span class="k">for</span> <span class="n">device_id</span> <span class="ow">in</span> <span class="n">device_ids</span><span class="p">:</span>
|
||||
</span><span id="line-29"><span class="linenos">29</span> <span class="c1"># Placeholder for actual data retrieval</span>
|
||||
</span><span id="line-30"><span class="linenos">30</span> <span class="n">stats</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="line-31"><span class="linenos">31</span> <span class="s2">"device_id"</span><span class="p">:</span> <span class="n">device_id</span><span class="p">,</span>
|
||||
</span><span id="line-32"><span class="linenos">32</span> <span class="s2">"time_range"</span><span class="p">:</span> <span class="sa">f</span><span class="s2">"Last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s2"> days"</span><span class="p">,</span>
|
||||
</span><span id="line-33"><span class="linenos">33</span> <span class="s2">"data"</span><span class="p">:</span> <span class="sa">f</span><span class="s2">"Statistics data for device </span><span class="si">{</span><span class="n">device_id</span><span class="si">}</span><span class="s2"> over the last </span><span class="si">{</span><span class="n">time_range</span><span class="si">}</span><span class="s2"> days."</span><span class="p">,</span>
|
||||
</span><span id="line-34"><span class="linenos">34</span> <span class="p">}</span>
|
||||
</span><span id="line-35"><span class="linenos">35</span> <span class="n">statistics</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">stats</span><span class="p">)</span>
|
||||
</span><span id="line-36"><span class="linenos">36</span>
|
||||
</span><span id="line-37"><span class="linenos">37</span> <span class="n">response</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"statistics"</span><span class="p">:</span> <span class="n">statistics</span><span class="p">}</span>
|
||||
</span><span id="line-38"><span class="linenos">38</span>
|
||||
</span><span id="line-39"><span class="linenos">39</span> <span class="k">return</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">response</span><span class="p">),</span> <span class="mi">200</span>
|
||||
</span><span id="line-40"><span class="linenos">40</span>
|
||||
</span><span id="line-41"><span class="linenos">41</span>
|
||||
</span><span id="line-42"><span class="linenos">42</span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
|
||||
</span><span id="line-43"><span class="linenos">43</span> <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -414,13 +434,13 @@ streamline data retrieval and processing to build more efficient and precise RAG
|
|||
<ul>
|
||||
<li><a :data-current="activeSection === '#intent-drift-detection'" class="reference internal" href="#intent-drift-detection">Intent-drift Detection</a><ul>
|
||||
<li><a :data-current="activeSection === '#step-1-define-conversationbuffermemory'" class="reference internal" href="#step-1-define-conversationbuffermemory">Step 1: Define ConversationBufferMemory</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-update-conversationbuffermemory-w-intent'" class="reference internal" href="#step-2-update-conversationbuffermemory-w-intent">Step 2: Update ConversationBufferMemory w/ intent</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-update-conversationbuffermemory-with-intents'" class="reference internal" href="#step-2-update-conversationbuffermemory-with-intents">Step 2: Update ConversationBufferMemory with Intents</a></li>
|
||||
<li><a :data-current="activeSection === '#step-3-get-messages-based-on-latest-drift'" class="reference internal" href="#step-3-get-messages-based-on-latest-drift">Step 3: Get Messages based on latest drift</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><a :data-current="activeSection === '#parameter-extraction-for-rag'" class="reference internal" href="#parameter-extraction-for-rag">Parameter Extraction for RAG</a><ul>
|
||||
<li><a :data-current="activeSection === '#step-1-define-prompt-targets-with-parameter-definitions'" class="reference internal" href="#step-1-define-prompt-targets-with-parameter-definitions">Step 1: Define prompt targets with parameter definitions</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-process-request-parameters-in-flask'" class="reference internal" href="#step-2-process-request-parameters-in-flask">Step 2: Process request parameters in Flask</a></li>
|
||||
<li><a :data-current="activeSection === '#step-1-define-prompt-targets'" class="reference internal" href="#step-1-define-prompt-targets">Step 1: Define Prompt Targets</a></li>
|
||||
<li><a :data-current="activeSection === '#step-2-process-request-parameters-in-flask'" class="reference internal" href="#step-2-process-request-parameters-in-flask">Step 2: Process Request Parameters in Flask</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue