Repository navigation
Foundation: fewer engine steps per run; health check reports failing vendor calls - #1247
Merged
Merged
Conversation
Foundation - Product Classification_V3 ran a chain of twelve playbook tasks (getList, a condition, a DeleteContext and nine Set tasks). Each is its own engine step; on a busy tenant each step waits in the queue, and one measured run spent 125 of 213 seconds there. SOCFWClassifyProduct resolves the same SOCFramework.Product keys in one script task, and still falls back to SOCFWInferProductCategory when the map has no category. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VK24hKuKXVJfuVHg4Min9y
Every Universal Command call that errors is already written to xsiam_socfw_ir_execution_raw with has_error = true, but nothing read it, so a broken instance failed on every issue unnoticed (deathstar: CrowdStrike Falcon 401 on cs-falcon-search-device, 119 times in a week). The health check now lists failures from the last 24 hours per vendor and command: FAIL for a call that errored, WARN for a command no enabled instance serves, each with the last error and what to do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VK24hKuKXVJfuVHg4Min9y
Foundation - Upon Trigger V3 called four sub-playbooks that each wrapped one script (Data Integrity, Normalize Artifacts, Enrichment, Case Sync). Every sub-playbook adds its own engine steps, and on a busy tenant each step waits in the queue. Upon Trigger now runs SOCFWClassifyProduct, SOCNormalizeFromList, SOCEnrichFromList and SOCFWCaseLedgerWrite as tasks of its own. Dropped on the way: extractIndicators over the whole issue (nothing reads its output) and Foundation - Error Handling_V3 (it only printed errors). Context is unchanged: the platform already passed every SOCFramework.Product key (response, responses, Inferred) out of the separate-context sub-playbook, so vendor routing is the same; verified by reading the issue context of matching old and new runs. The sub-playbooks stay in the pack for anything else that calls them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VK24hKuKXVJfuVHg4Min9y
Each title is an engine step that does nothing; on a busy tenant one waited 9.4 s in the queue. Upon Trigger is now start, then its five script tasks in a straight line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VK24hKuKXVJfuVHg4Min9y
scottbrumley
force-pushed
the
unified/fewer-playbook-steps
branch
from
November 12, 2026 01:38
6ce8257 to
6cc623b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Every
EP_IR_NIST - AIrun is one engine step per task, and on a busy tenant each step waits in the playbook queue. Most of a run was queueing, not work: one measured run spent 125 of 213 s in Product Classification's twelve tiny tasks. This cuts the Foundation steps a run takes and adds a way to see broken integrations.SOCFWClassifyProductreplaces the twelve-task chain insideFoundation - Product Classification_V3; same keys, same inference fallbackxsiam_socfw_ir_execution_raw(has_error = true): FAIL for a call that errored, WARN for a command no enabled instance servesextractIndicatorsover the whole issue (output unread) andFoundation - Error Handling_V3(prints only) droppedSteps per run: 52 → 16 with this PR alone; 12 with the companion
soc-framework-nist-ir-aiPR (Assessment Decisions).Tested on deathstar
SOCFWClassifyProduct(mapped, product override, unmapped → inference, no tag, missing map, unparseable map, platform error, stale keys cleared, argument shapes); 4 new for the health check; every other script test in the pack passes exceptSOCCommandWrapper_test.py::test_main_production_rate_limited_soft_fails, which fails onmainalready.SOCFramework.Productidentical, includingresponseandresponses, so vendor routing is unchanged. Category, normalization band and status identical across 18 alert types and 973 old runs. Wrapper rows by action / vendor / outcome / mode: no new combinations.Deploying
Ships through the Package Manager like any
soc-optimization-unifiedrelease; povkit installs it with the SOCFW base. Every script these playbooks call is in this pack. The old Foundation sub-playbooks stay in the pack, so nothing else that calls them changes.Upload in a quiet window. On deathstar, issues that were queued while content was uploaded stayed at
newand never started (twice today); issues created afterwards ran normally.After the install, run
!SOCFWHealthCheckand fix or disable any instance it reports failing — each failing call costs every run its wait.🤖 Generated with Claude Code
https://claude.ai/code/session_01VK24hKuKXVJfuVHg4Min9y