mirror of
https://github.com/katanemo/plano.git
synced 2026-05-15 11:02:39 +02:00
deploy: dafd245332
This commit is contained in:
parent
805883eadb
commit
e4698a2be5
4 changed files with 16 additions and 14 deletions
|
|
@ -660,7 +660,8 @@ traces, then events tell you <em>which messages</em> fired <em>which signals</em
|
|||
<h3>Visual Flag Marker<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#visual-flag-marker" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#visual-flag-marker'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h3>
|
||||
<p>When concerning signals are detected (disengagement present, stagnation
|
||||
count > 2, any execution failure / loop, or overall quality <code class="docutils literal notranslate"><span class="pre">poor</span></code>/
|
||||
<code class="docutils literal notranslate"><span class="pre">severe</span></code>), the marker <code class="docutils literal notranslate"><span class="pre">[!]</span></code> is appended to the span’s operation name.
|
||||
<code class="docutils literal notranslate"><span class="pre">severe</span></code>), the marker 🚩 (U+1F6A9) is appended to the span’s operation
|
||||
name.
|
||||
This makes flagged sessions immediately visible in trace UIs without
|
||||
requiring attribute filtering.</p>
|
||||
</section>
|
||||
|
|
@ -677,7 +678,7 @@ requiring attribute filtering.</p>
|
|||
</span><span id="line-8"><span class="n">signals</span><span class="o">.</span><span class="n">environment</span><span class="o">.</span><span class="n">exhaustion</span><span class="o">.</span><span class="n">count</span> <span class="o">></span> <span class="mi">0</span>
|
||||
</span></code></pre></div>
|
||||
</div>
|
||||
<p>For flagged sessions, search for <code class="docutils literal notranslate"><span class="pre">[!]</span></code> in span names.</p>
|
||||
<p>For flagged sessions, search for 🚩 in span names.</p>
|
||||
<a class="reference internal image-reference" href="../_images/signals_trace.png"><img alt="../_images/signals_trace.png" class="align-center" src="../_images/signals_trace.png" style="width: 100%;"/>
|
||||
</a>
|
||||
</section>
|
||||
|
|
@ -753,7 +754,7 @@ observability platform filters on the signal attributes described above.</p>
|
|||
<h2>Example Span<a @click.prevent="window.navigator.clipboard.writeText($el.href); $el.setAttribute('data-tooltip', 'Copied!'); setTimeout(() => $el.setAttribute('data-tooltip', 'Copy link to this element'), 2000)" aria-label="Copy link to this element" class="headerlink" data-tooltip="Copy link to this element" href="#example-span" x-intersect.margin.0%.0%.-70%.0%="activeSection = '#example-span'"><svg height="1em" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M3.9 12c0-1.71 1.39-3.1 3.1-3.1h4V7H7c-2.76 0-5 2.24-5 5s2.24 5 5 5h4v-1.9H7c-1.71 0-3.1-1.39-3.1-3.1zM8 13h8v-2H8v2zm9-6h-4v1.9h4c1.71 0 3.1 1.39 3.1 3.1s-1.39 3.1-3.1 3.1h-4V17h4c2.76 0 5-2.24 5-5s-2.24-5-5-5z"></path></svg></a></h2>
|
||||
<p>A concerning session, showing both layered attributes and a per-instance
|
||||
event:</p>
|
||||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="c1"># Span name: "POST /v1/chat/completions gpt-5.2 [!]"</span>
|
||||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="c1"># Span name: "POST /v1/chat/completions gpt-5.2 🚩"</span>
|
||||
</span><span id="line-2">
|
||||
</span><span id="line-3"><span class="c1"># Top-level</span>
|
||||
</span><span id="line-4"><span class="n">signals</span><span class="o">.</span><span class="n">quality</span> <span class="o">=</span> <span class="s2">"severe"</span>
|
||||
|
|
@ -868,7 +869,7 @@ causes.</p>
|
|||
</div>
|
||||
<div class="admonition tip">
|
||||
<p class="admonition-title">Tip</p>
|
||||
<p>The <code class="docutils literal notranslate"><span class="pre">[!]</span></code> marker in the span name provides instant visual feedback in
|
||||
<p>The 🚩 marker in the span name provides instant visual feedback in
|
||||
trace UIs, while the structured attributes (<code class="docutils literal notranslate"><span class="pre">signals.quality</span></code>,
|
||||
<code class="docutils literal notranslate"><span class="pre">signals.interaction.disengagement.severity</span></code>, etc.) and per-instance
|
||||
span events enable powerful querying and drill-down in your observability
|
||||
|
|
|
|||
|
|
@ -262,9 +262,9 @@ tools.</p>
|
|||
<li><p><strong>Environment layer</strong>: exhaustion (API errors, timeouts, rate limits, context overflow)</p></li>
|
||||
</ul>
|
||||
<p><strong>Visual Flag Markers</strong></p>
|
||||
<p>When concerning signals are detected (disengagement, execution failures / loops, stagnation > 2, or <code class="docutils literal notranslate"><span class="pre">poor</span></code> / <code class="docutils literal notranslate"><span class="pre">severe</span></code> quality), Plano automatically appends a <code class="docutils literal notranslate"><span class="pre">[!]</span></code> marker to the span’s operation name. This makes problematic traces immediately visible in your tracing UI without requiring additional queries.</p>
|
||||
<p>When concerning signals are detected (disengagement, execution failures / loops, stagnation > 2, or <code class="docutils literal notranslate"><span class="pre">poor</span></code> / <code class="docutils literal notranslate"><span class="pre">severe</span></code> quality), Plano automatically appends a 🚩 marker to the span’s operation name. This makes problematic traces immediately visible in your tracing UI without requiring additional queries.</p>
|
||||
<p><strong>Example Span with Signals</strong>:</p>
|
||||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="c1"># Span name: "POST /v1/chat/completions gpt-4 [!]"</span>
|
||||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><code><span id="line-1"><span class="c1"># Span name: "POST /v1/chat/completions gpt-4 🚩"</span>
|
||||
</span><span id="line-2"><span class="c1"># Standard LLM attributes:</span>
|
||||
</span><span id="line-3"><span class="n">llm</span><span class="o">.</span><span class="n">model</span> <span class="o">=</span> <span class="s2">"gpt-4"</span>
|
||||
</span><span id="line-4"><span class="n">llm</span><span class="o">.</span><span class="n">usage</span><span class="o">.</span><span class="n">total_tokens</span> <span class="o">=</span> <span class="mi">225</span>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
Plano Docs v0.4.20
|
||||
llms.txt (auto-generated)
|
||||
Generated (UTC): 2026-04-24T19:32:12.216149+00:00
|
||||
Generated (UTC): 2026-04-24T20:55:22.182534+00:00
|
||||
|
||||
Table of contents
|
||||
- Agents (concepts/agents)
|
||||
|
|
@ -3108,7 +3108,8 @@ Visual Flag Marker
|
|||
|
||||
When concerning signals are detected (disengagement present, stagnation
|
||||
count > 2, any execution failure / loop, or overall quality poor/
|
||||
severe), the marker [!] is appended to the span’s operation name.
|
||||
severe), the marker 🚩 (U+1F6A9) is appended to the span’s operation
|
||||
name.
|
||||
This makes flagged sessions immediately visible in trace UIs without
|
||||
requiring attribute filtering.
|
||||
|
||||
|
|
@ -3125,7 +3126,7 @@ signals.interaction.satisfaction.count > 0 AND signals.quality = "good"
|
|||
signals.execution.failure.count > 0
|
||||
signals.environment.exhaustion.count > 0
|
||||
|
||||
For flagged sessions, search for [!] in span names.
|
||||
For flagged sessions, search for 🚩 in span names.
|
||||
|
||||
|
||||
|
||||
|
|
@ -3219,7 +3220,7 @@ Example Span
|
|||
A concerning session, showing both layered attributes and a per-instance
|
||||
event:
|
||||
|
||||
# Span name: "POST /v1/chat/completions gpt-5.2 [!]"
|
||||
# Span name: "POST /v1/chat/completions gpt-5.2 🚩"
|
||||
|
||||
# Top-level
|
||||
signals.quality = "severe"
|
||||
|
|
@ -3350,7 +3351,7 @@ response quality evaluation. Use signals to prioritize which traces to
|
|||
inspect, then apply domain expertise and outcome checks to diagnose root
|
||||
causes.
|
||||
|
||||
The [!] marker in the span name provides instant visual feedback in
|
||||
The 🚩 marker in the span name provides instant visual feedback in
|
||||
trace UIs, while the structured attributes (signals.quality,
|
||||
signals.interaction.disengagement.severity, etc.) and per-instance
|
||||
span events enable powerful querying and drill-down in your observability
|
||||
|
|
@ -4919,11 +4920,11 @@ Environment layer: exhaustion (API errors, timeouts, rate limits, context overfl
|
|||
|
||||
Visual Flag Markers
|
||||
|
||||
When concerning signals are detected (disengagement, execution failures / loops, stagnation > 2, or poor / severe quality), Plano automatically appends a [!] marker to the span’s operation name. This makes problematic traces immediately visible in your tracing UI without requiring additional queries.
|
||||
When concerning signals are detected (disengagement, execution failures / loops, stagnation > 2, or poor / severe quality), Plano automatically appends a 🚩 marker to the span’s operation name. This makes problematic traces immediately visible in your tracing UI without requiring additional queries.
|
||||
|
||||
Example Span with Signals:
|
||||
|
||||
# Span name: "POST /v1/chat/completions gpt-4 [!]"
|
||||
# Span name: "POST /v1/chat/completions gpt-4 🚩"
|
||||
# Standard LLM attributes:
|
||||
llm.model = "gpt-4"
|
||||
llm.usage.total_tokens = 225
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
Loading…
Add table
Add a link
Reference in a new issue