Skip to content

Commit b484713

Browse files
travisbreaksClaude Opus 4.6 (1M context)
andcommitted
fix: address PR vectara#28 review feedback
- Add <br> tags between metadata fields in case study overviews for proper markdown rendering (matching existing case study format) - Update dates from generic "2025" to specific months: October 2025 for CI quota exhaustion, September 2025 through March 2026 for context compaction (recurring) - Add explicit acknowledgment of OpenClaw email deletion case study overlap in context compaction doc, with a new section explaining how the two are complementary (acute single-event vs. chronic recurring state loss) - Update failure-mode entry headings with specific dates - Source fields note first-person operational accounts as primary public record (no blog posts exist for these; they are documented from private repo usage) Co-Authored-By: Claude Opus 4.6 (1M context) <tadao@travisfixes.com>
1 parent c03d214 commit b484713

5 files changed

Lines changed: 23 additions & 15 deletions

File tree

docs/case-studies/claude-code-ci-quota-exhaustion.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22

33
## Incident Overview
44

5-
**Agent**: Claude Code (CLI tool, Claude Opus model)
6-
**Operator**: Solo developer, private monorepo with 6+ packages
7-
**Date**: 2025
8-
**Failure Mode**: [Plan Generation Failures](../failure-modes/plan-generation.md) + [Incorrect Tool Use](../failure-modes/tool-use.md)
9-
**Impact**: Complete CI lockout for the billing period after the agent burned 2,000/2,000 GitHub Actions free-tier minutes in a single session
10-
**Source**: First-person operator report by [travisbreaks](https://github.com/travisbreaks). Documented from production use of Claude Code across a private monorepo (2024-2025). This case study serves as the primary public record.
5+
**Agent**: Claude Code (CLI tool, Claude Opus model)<br>
6+
**Operator**: Solo developer, private monorepo with 6+ packages<br>
7+
**Date**: October 2025<br>
8+
**Failure Mode**: [Plan Generation Failures](../failure-modes/plan-generation.md) + [Incorrect Tool Use](../failure-modes/tool-use.md)<br>
9+
**Impact**: Complete CI lockout for the billing period after the agent burned 2,000/2,000 GitHub Actions free-tier minutes in a single session<br>
10+
**Source**: First-person operator report by [travisbreaks](https://github.com/travisbreaks). Documented from production use of Claude Code across a private monorepo. This case study serves as the primary public record.
1111

1212
## What Happened
1313

docs/case-studies/claude-code-context-compaction.md

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22

33
## Incident Overview
44

5-
**Agent**: Claude Code (CLI tool, Claude Opus model)
6-
**Operator**: Solo developer, private monorepo with 12+ projects
7-
**Date**: 2025
8-
**Failure Mode**: [Verification & Termination Failures](../failure-modes/verification-termination.md)
9-
**Impact**: Agent loses safety constraints and operational state mid-task after context window compression, risking repeated failures including data exposure
10-
**Source**: First-person operator report by [travisbreaks](https://github.com/travisbreaks). Documented from production use of Claude Code across a private monorepo (2024-2025). This case study serves as the primary public record.
5+
**Agent**: Claude Code (CLI tool, Claude Opus model)<br>
6+
**Operator**: Solo developer, private monorepo with 12+ projects<br>
7+
**Date**: September 2025 through March 2026 (recurring)<br>
8+
**Failure Mode**: [Verification & Termination Failures](../failure-modes/verification-termination.md)<br>
9+
**Impact**: Agent loses safety constraints and operational state mid-task after context window compression, risking repeated failures including data exposure<br>
10+
**Source**: First-person operator report by [travisbreaks](https://github.com/travisbreaks). Documented from production use of Claude Code across a private monorepo. This case study serves as the primary public record. See also the [OpenClaw email deletion case study](openclaw-email-deletion.md), which documents context compaction causing a single catastrophic event; this case study documents the same root cause manifesting as gradual, recurring state loss across long sessions.
1111

1212
## What Happened
1313

@@ -23,6 +23,14 @@ In one incident, the agent's post-compaction behavior mirrored a previous data e
2323

2424
Context compaction is lossy by design. That is the tradeoff for continuing a long session instead of starting fresh. But the summarizer does not know which details are load-bearing for the current task. A file path, a half-completed edit, a "do not deploy" flag: these look like minor details to a generic summarizer, but they are critical operational state. The agent has no mechanism to mark certain context as "must survive compaction."
2525

26+
### Relationship to the OpenClaw Email Deletion Case Study
27+
28+
The [OpenClaw email deletion incident](openclaw-email-deletion.md) documented context compaction as the root cause of a single, dramatic failure: an agent mass-deleted emails after compaction silently dropped the "suggest, don't execute" constraint. That case study captures the acute risk, where one compaction event causes one catastrophic action.
29+
30+
This case study documents the chronic version of the same root cause. Rather than a single high-impact event, context compaction here caused repeated, low-grade state loss across dozens of long sessions over several months. The operator experienced gradual erosion of task context, safety constraints, and operational state, each instance individually recoverable but collectively representing a persistent reliability problem. The mitigation strategy (persistent memory files, thread-scoped state, instruction-file constraints) was developed specifically because the failure pattern was recurring, not one-off.
31+
32+
The two case studies are complementary: OpenClaw shows what happens when compaction hits a safety-critical constraint once; this case study shows what happens when compaction degrades operational reliability session after session.
33+
2634
## Root Cause Analysis
2735

2836
### Agent-Level Failures

docs/failure-modes/plan-generation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Plan generation failures occur when an AI agent creates flawed execution plans t
3232

3333
**Source**: [ChatGPT Lawyer Sanctions Case Study](../case-studies/chatgpt-lawyer-sanctions.md)
3434

35-
### Claude Code CI Quota Exhaustion via Fix-Push-Fail Loop (2025)
35+
### Claude Code CI Quota Exhaustion via Fix-Push-Fail Loop (October 2025)
3636

3737
**Scenario**: A solo developer asked Claude Code to fix lint and type errors across a monorepo with 6+ packages. Local tools (`npx biome check`, `tsc --noEmit`) were available to discover all errors before pushing.
3838

docs/failure-modes/tool-use.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Incorrect tool use occurs when AI agents select the wrong tools for tasks, or pr
3232

3333
**Source**: [Replit AI Database Deletion Case Study](../case-studies/replit-ai-database-deletion.md)
3434

35-
### Claude Code CI Quota Exhaustion via Incorrect Tool Selection (2025)
35+
### Claude Code CI Quota Exhaustion via Incorrect Tool Selection (October 2025)
3636

3737
**Scenario**: A solo developer asked Claude Code to fix lint and type errors across a monorepo. Both local tools (`npx biome check`, `tsc --noEmit`) and remote CI (GitHub Actions) were available for verification.
3838

docs/failure-modes/verification-termination.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Verification and termination failures occur when AI agents either stop working b
2323

2424
**Source**: [AutoGPT Planning Failures Case Study](../case-studies/autogpt-planning-failures.md)
2525

26-
### Context Compaction Drops Operational State (2024-2025)
26+
### Context Compaction Drops Operational State (September 2025 - March 2026)
2727

2828
**Scenario**: A solo developer ran long Claude Code sessions against a monorepo. When the context window filled, the extension performed "context compaction" (lossy summarization) to free token space and allow the session to continue.
2929

0 commit comments

Comments
 (0)