Summary
In backend/app/services/llm_analysis.py, the _extract_json() method naively searches for the first { and the last } to slice out the JSON payload. If the LLM generates any valid braces before or after the main JSON block, the slice becomes invalid JSON. Additionally, the method invokes json.loads without catching json.JSONDecodeError, allowing the exception to bubble up and crash async tasks unexpectedly.
Impact
Frequent 500 errors or failed background jobs when the LLM hallucination rate is high, reducing service stability and reliability.
Proposed Fix
Implement a robust JSON extraction mechanism (e.g., using regex to find matching top-level JSON objects) and wrap json.loads in a try...except block, raising a custom LLMAnalysisError upon failure.
Summary
In
backend/app/services/llm_analysis.py, the_extract_json()method naively searches for the first{and the last}to slice out the JSON payload. If the LLM generates any valid braces before or after the main JSON block, the slice becomes invalid JSON. Additionally, the method invokesjson.loadswithout catchingjson.JSONDecodeError, allowing the exception to bubble up and crash async tasks unexpectedly.Impact
Frequent 500 errors or failed background jobs when the LLM hallucination rate is high, reducing service stability and reliability.
Proposed Fix
Implement a robust JSON extraction mechanism (e.g., using regex to find matching top-level JSON objects) and wrap
json.loadsin atry...exceptblock, raising a customLLMAnalysisErrorupon failure.