perf: fix CPU spike — add early $match before $unwind in CNRepos pipeline#203
Conversation
…line
Investigated 2026-04-23T22:12Z Atlas CPU spike (98.5% CPU). Query Profiler
showed comfy-pr.CNRepos reads caused 67.44 CPU-hours in 10.5h window:
737 ops × avg 5.49 min, with Uses Index: No.
Root cause: baseCRPullStatusPipeline() in analyzePullsStatus.ts ran a
full-collection scan, then $unwind on crPulls.data array, then $match —
processing every document and every crPull entry before any filtering.
Fix:
- Add early $match { crPulls.data: { $elemMatch: { comments.state: "ok" } } }
before $unwind to skip repos with no fetched PR comments
- Add supporting multikey index on crPulls.data.comments.state (scalar Task
field — avoids "Cannot index parallel arrays" from using comments.data)
- Add sparse indexes on GithubActionUpdateTask.error and .pullRequestMessage
(Performance Advisor flagged 4310-doc scans for regex queries)
Reviewed by: Gemini 2.5 Flash (LGTM), Gemini 2.5 Pro (flagged parallel-array
index issue → fixed), Codex gpt-5.4 (no issues with specific changes)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR addresses a production MongoDB Atlas CPU spike by reducing CNRepos aggregation work (filtering earlier in the pipeline) and adding supporting indexes for the affected query patterns.
Changes:
- Add an early
$match(via$elemMatchoncrPulls.data.comments.state) before$unwindinbaseCRPullStatusPipeline()to avoid scanning/unwinding repos that have no fetched PR comments. - Add a multikey index on
CNReposforcrPulls.data.comments.stateto support the early filter without hitting the “parallel arrays” limitation. - Add sparse indexes on
GithubActionUpdateTask.errorand.pullRequestMessageintended to reduce scans flagged by Performance Advisor.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/analyzePullsStatus.ts | Adds early $match stage before $unwind to reduce work in the CNRepos aggregation pipeline. |
| src/CNRepos.ts | Adds a multikey index on crPulls.data.comments.state to support the new early filter. |
| src/GithubActionUpdateTask/GithubActionUpdateTask.ts | Adds sparse indexes on error and pullRequestMessage for query performance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…limit Codex review flagged that pullRequestMessage values are ~1.6KB (generated from the PR template), which can exceed MongoDB's B-tree key size limit and cause write failures with "key too large to index" error. Keep only the error field index (short strings, safe to index). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot review flagged that .catch(() => {}) silently hides index creation
failures, masking schema/parallel-array issues in production. Change to
.catch(console.error) so startup logs show problems while still being
non-blocking (performance-only indexes, not correctness constraints).
Also clarify that the error index primarily helps { error: { $exists: false } }
(the main task scanner query), not the unanchored regex queries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Addressed Copilot review comments (1f7c609): Comment 1 & 3 (.catch silencing): Changed Comment 2 (regex + B-tree): Acknowledged — unanchored regex like |
Gemini 2.5 Pro review flagged that a sparse index cannot support
{ error: { $exists: false } } queries — sparse indexes exclude docs
where the field is missing, so $exists:false still requires a full scan.
Use a regular (non-sparse) index so all documents are included,
enabling the main task scanner query to use the index.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
🎉 This PR is included in version 1.4.2 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Summary
comfy-pr.CNReposreads → 737 ops × avg 5.49 min = 67.44 CPU-hours in 10.5h window;Uses Index: NobaseCRPullStatusPipeline()scanned all CNRepos docs,$unwinded everycrPulls.dataentry, then filtered — processing everything before discarding irrelevant documentsFixes:
$match { crPulls.data: { $elemMatch: { "comments.state": "ok" } } }before$unwindto skip repos with no fetched PR commentscrPulls.data.comments.state(scalarTaskfield — usingcomments.datawould fail with "Cannot index parallel arrays")GithubActionUpdateTask.errorand.pullRequestMessage(Performance Advisor flagged 4310-doc regex scans at 0.42/hr)Test plan
idx_crpulls_comments_stateindex builds successfully in Atlas (no "parallel arrays" error)/detailspage still renders correctly with the filtered pipelineanalyzePullsStatus()still returns same data (the post-$unwind.match({ "crPulls.data.comments.data": { $exists: true } })is kept as safety net)Investigation Evidence
Atlas Query Profiler (2026-04-23 22:00 JST → 2026-04-24 08:30 JST):
Actual query logged:
aggregate CNRepos → $set → $unwind "$crPulls.data" → $match → ...(Num Yields: 5,043, Uses Index: No)Reviewed by Gemini 2.5 Flash ✅, Gemini 2.5 Pro ✅ (flagged parallel-array index issue → fixed), Codex gpt-5.4 ✅
🤖 Generated with Claude Code