Expose operational telemetry and SLO gates for cognitive dissonance dspy (multi-agent llm detecting resolving cognitive)

[haasonsaas]

Summary

Define the observable contract for latency, cost, correctness, and degraded-mode behavior.

This issue was generated from an org-wide EvalOps mining pass on 2026-05-10 07:57 UTC. It combines live GitHub repo signals with a per-repo arXiv search. Treat the research links as grounding for a concrete implementation, not as a request for a literature review.

Repo Evidence

• Repository description: A multi-agent LLM system for detecting and resolving cognitive dissonance.
• Tree signals: 0 docs files, 1 workflows, 0 proto files, 19 test-like files.
• README.md:12 includes latent-spec language: The paper studies a narrow problem: how should a system evaluate and resolve formalizable claim disagreements when proof is available as a resolution
• README.md:17 includes latent-spec language: > Proof-first conflict resolution should be evaluated by separating > deterministic canonicalization, provider-assisted extraction, proof outcome,
• README.md:28 includes latent-spec language: The paper contributes an evaluation decomposition with four distinct layers:
• README.md:39 includes latent-spec language: This should be read as a methods paper with a narrow empirical stress test, not as a broad systems paper.
• README.md:93 includes latent-spec language: - the necessity ablation is neutral on this benchmark - necessity should not be positioned as the paper’s main novelty
• README.md:137 includes latent-spec language: closed - preservation auditing is therefore part of the resolution contract, not just a reporting detail

Research Grounding

Repo axes: research, evaluation, tooling, security

Search keywords: proof, extraction, should, formalizable, research, benchmark, claim, not, paper, preservation, https, cases

• arXiv:2506.19773v2 Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study (Nandana Mihindukulasooriya, Niharika S. D'Souza, Faisal Chowdhury, Horst Samulowitz), 2025.
• arXiv:2507.03620v1 Is It Time To Treat Prompts As Code? A Multi-Use Case Study For Prompt Optimization Using DSPy (Francisca Lemos, Victor Alves, Filipa Ferraz), 2025.
• arXiv:2412.15298v1 A Comparative Study of DSPy Teleprompter Algorithms for Aligning Large Language Models Evaluation Metrics to Human Evaluation (Bhaskarjit Sarmah, Kriti Dutta, Anna Grigoryan, Sachin Tiwari, Stefano Pasquali, Dhagash Mehta), 2024.
• arXiv:2604.04869v1 Optimizing LLM Prompt Engineering with DSPy Based Declarative Learning (Shiek Ruksana, Sailesh Kiran Kurra, Thipparthi Sanjay Baradwaj), 2026.
• arXiv:2503.11118v1 UMB@PerAnsSumm 2025: Enhancing Perspective-Aware Summarization with Prompt Optimization and Supervised Fine-Tuning (Kristin Qi, Youxiang Zhu, Xiaohui Liang), 2025.
• arXiv:2605.02244v1 The Conversations Beneath the Code: Triadic Data for Long-Horizon Software Engineering Agents (Yelin Kim), 2026.
• arXiv:2503.23803v2 Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute (Yingwei Ma, Yongbin Li, Yihong Dong, Xue Jiang, Rongyu Cao, Jue Chen), 2025.
• arXiv:2508.04660v1 Multi-module GRPO: Composing Policy Gradients and Prompt Optimization for Language Model Programs (Noah Ziems, Dilara Soylu, Lakshya A Agrawal, Isaac Miller, Liheng Lai, Chen Qian), 2025.
• arXiv:2602.00997v1 Error Taxonomy-Guided Prompt Optimization (Mayank Singh, Vikas Yadav, Eduardo Blanco), 2026.
• arXiv:2602.03411v2 SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training (Huatong Song, Lisheng Huang, Shuang Sun, Jinhao Jiang, Ran Le, Daixuan Cheng), 2026.

What To Build

• Name the key service/user journey SLOs and their required dimensions.
• Emit metrics/log fields for success, failure, cost/latency, and reasoned fallback.
• Add a dashboard/runbook stub or CLI report that makes the new signals operator-visible.

Acceptance Criteria

• A short design note names the repo-specific workflow, threat or correctness model, and the research assumptions being adopted.
• A runnable check, fixture, or verifier exercises the new contract in CI or an equivalent local command documented in the repo.
• The implementation emits or stores enough evidence for a downstream agent/operator to cite inputs, decisions, and outputs.
• At least one negative/degraded-mode case is covered so failures are observable rather than silently accepted.
• Documentation links the new behavior to the relevant EvalOps platform primitive or explicitly records why this repo remains standalone.

Notes

• Generated issue 4/5 for evalops/cognitive-dissonance-dspy by evalops_org_miner.py.
• Before implementation, confirm the sampled latent-spec snippets still match main; this issue intentionally cites exact file paths/lines where the mining pass saw them.