signals: feature parity with the latest Signals paper. Porting logic from python repo (#903)

* signals: port to layered taxonomy with dual-emit OTel

Made-with: Cursor

* fix: silence collapsible_match clippy lint (rustc 1.95)

Made-with: Cursor

* test: parity harness for rust vs python signals analyzer

Validates the brightstaff signals port against the katanemo/signals Python
reference on lmsys/lmsys-chat-1m. Adds a signals_replay bin emitting python-
compatible JSON, a pyarrow-based driver (bypasses the datasets loader pickle
bug on python 3.14), a 3-tier comparator, and an on-demand workflow_dispatch
CI job.

Made-with: Cursor

* Remove signals test from the gitops flow

* style: format parity harness with black

Made-with: Cursor

* signals: group summary by taxonomy, factor misalignment_ratio

Addresses #903 review feedback from @nehcgs:

- generate_summary() now renders explicit Interaction / Execution /
  Environment headers so the paper taxonomy is visible at a glance,
  even when no signals fired in a given layer. Quality-driving callouts
  (high misalignment rate, looping detected, escalation requested) are
  appended after the layer summary as an alerts tail.

- repair_ratio (legacy taxonomy name) renamed to misalignment_ratio
  and factored into a single InteractionSignals::misalignment_ratio()
  helper so assess_quality and generate_summary share one source of
  truth instead of recomputing the same divide twice.

Two new unit tests pin the layer headers and the (sev N) severity
suffix. Parity with the python reference is preserved at the Tier-A
level (per-type counts + overall_quality); only the human-readable
summary string diverges, which the parity comparator already classifies
as Tier-C.

Made-with: Cursor
This commit is contained in:
Syed A. Hashmi 2026-04-23 12:02:30 -07:00 committed by GitHub
parent 6701195a5d
commit c8079ac971
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
31 changed files with 5246 additions and 3261 deletions

View file

@ -317,11 +317,10 @@ impl TryFrom<Message> for BedrockMessage {
Role::User => {
// Convert user message content to content blocks
match message.content {
Some(MessageContent::Text(text)) => {
if !text.is_empty() {
content_blocks.push(ContentBlock::Text { text });
}
Some(MessageContent::Text(text)) if !text.is_empty() => {
content_blocks.push(ContentBlock::Text { text });
}
Some(MessageContent::Text(_)) => {}
Some(MessageContent::Parts(parts)) => {
// Convert OpenAI content parts to Bedrock ContentBlocks
for part in parts {