apunkt/dograh
1
0
Fork 0
mirror of https://github.com/dograh-hq/dograh.git synced 2026-06-13 08:15:21 +02:00
Code Issues Projects Releases Packages Wiki Activity

fix(qa): tolerate non-dict JSON from QA LLM instead of crashing

Browse source 
parse_llm_json is explicitly designed to return a list when the model emits a
top-level JSON array (it has a dedicated test for that). The QA analyzers then
call parsed.get("tags", ...) directly on the result. When parsed is a list,
that raises AttributeError, which is NOT caught by the surrounding
except (json.JSONDecodeError, ValueError) — so a single stray array response
from the QA model crashed the entire QA analysis run instead of degrading to
empty results.

The live variable-extraction path already guards this exact case with an
isinstance(..., dict) check; mirror it in both QA analysis call sites
(_run_qa_analysis per-node and _run_whole_call_qa_analysis fallback) so a
non-dict parse result coerces to {} and the run produces empty defaults.

Adds a regression test that drives the whole-call analyzer with an array
response and asserts empty results rather than a crash.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Mubashir Rahim 2026-06-03 14:24:03 +05:00
parent acc2ef9e96
commit 6ae9ba5d9e
2 changed files with 74 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
api/services/workflow/qa/analysis.py
View file

@ -206,6 +206,11 @@ async def run_per_node_qa_analysis(
}
try:
parsed = parse_llm_json(raw_response)
# parse_llm_json can return a list (e.g. when the model emits a
# top-level JSON array); coerce non-dict results so the .get()
# lookups below don't raise AttributeError.
if not isinstance(parsed, dict):
parsed = {}
node_result["tags"] = parsed.get("tags", [])
node_result["summary"] = parsed.get("summary", "")
node_result["score"] = parsed.get("call_quality_score")
@ -296,6 +301,11 @@ async def _run_whole_call_qa_analysis(
}
try:
parsed = parse_llm_json(raw_response)
# parse_llm_json can return a list (e.g. when the model emits a
# top-level JSON array); coerce non-dict results so the .get()
# lookups below don't raise AttributeError.
if not isinstance(parsed, dict):
parsed = {}
node_result["tags"] = parsed.get("tags", [])
node_result["summary"] = parsed.get("summary", "")
node_result["score"] = parsed.get("call_quality_score")