fix(toolkit-lib): restore the replacement guard for Express Mode deployments - #1969
Draft
sanjanaravikumar-az wants to merge 5 commits into
Draft
sanjanaravikumar-az wants to merge 5 commits into
sanjanaravikumar-az wants to merge 5 commits into
Conversation
…oyments CloudFormation rejects replacement-type updates while rollback is disabled. Express Mode disables rollback unless --rollback is passed, so `cdk deploy --express` submits replacements CloudFormation refuses, and the stack is left in UPDATE_FAILED with no rollback available. #1745 removed the express half of the guard condition (and relaxed the test covering it), #1785 restructured what was left into `if (!this.options.express)`. Derive the condition once in `rollbackDisabled()` and apply the guard in both modes, returning `replacement-requires-rollback` so the existing confirm-and-retry-with-rollback prompt engages. Fix the same condition in the failure diagnostic, which reported rollback as enabled under --express and advised users to re-run with --no-rollback. `--method=direct` is deliberately not refused up front, because redeploying the previous configuration that way is how a stuck stack is unwedged; instead the rejection is detected after the fact and the user is routed to `--express --rollback`. That routing reads the activity monitor's errors, so the monitor is now flushed before they are read. Fixes #1931
sanjanaravikumar-az
had a problem deploying
to
automation
September 18, 2026 05:59 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
This was referenced Sep 18, 2026
… gated Move three rationales into the code, where they are durable: - --express --method=direct is deliberately not refused up front, because replaying the previous configuration that way is the only exit from a stack already stranded in UPDATE_FAILED. - Replacement: 'Conditional' is deliberately excluded, because CDKMetadata reports it on essentially every CDK deployment. - The CloudFormation reason match should be replaced with a structured discriminator if one ever becomes available.
added 2 commits
September 18, 2026 20:34
…terminally when already wedged
Two defects in the replacement guard.
The direct-path routing was gated on rollbackDisabled() alone, which is also true
for a standard-mode `--no-rollback` deployment. Since the guidance names Express
Mode flags, a wedged standard-mode user was told to switch to Express Mode, which
is sticky and gives up `cdk rollback` - a strictly worse position for a problem a
plain `cdk deploy` fixes. The notify is now gated on `express`, matching the
change-set arm.
On express with a replacement against an already-failed stack, the guard printed
that `--express --rollback` cannot update the stack and then returned
`replacement-requires-rollback`, so the toolkit offered exactly that deployment.
The confirmation defaults to yes, so a non-interactive caller ran it and hit
`ValidationError: non-terminal [UPDATE_FAILED]`. That case now throws
`ReplacementRequiresUnwedge` carrying the unwedge steps, so it is reported once
and CI cannot auto-run a deployment that is known to fail.
Also: `ReplacedResource.logicalId` is optional and both fabricated fallbacks
('<unknown>' and the stack name) are gone, so `replacements` can be genuinely
empty as documented; `shouldDisableRollback` renamed to
`shouldSendDisableRollbackFlag`; and the comment claiming express cannot send a
top-level `DisableRollback` is corrected (`--express --no-rollback` does send it).
… current invocation A change set carries its own rollback policy. CloudFormation persists DeploymentConfig on CreateChangeSet, returns it from DescribeChangeSet, and ExecuteChangeSet has no DeploymentConfig field, so it cannot override it. The guard read the current invocation's --express/--rollback flags instead, which is only correct when one invocation both creates and executes the change set. Split across invocations - `--method=change-set --no-execute` then `--method=execute-change-set`, or an explicit execute-change-set retry - the second command's flags decided nothing. Passing --rollback, or simply omitting --express, made the guard conclude rollback was enabled and execute a still-rollback-disabled change set containing a replacement, putting the prohibited combination back in front of CloudFormation and reaching #1931 again. Rollback state now comes from the persisted DeploymentConfig when it pins the answer (Express only; standard mode decides at execute time via the DisableRollback flag, so current options stay authoritative there). A requested policy that conflicts with the persisted one is refused with ChangeSetRollbackPolicyMismatch telling the user to create a new change set, rather than silently doing the opposite of what was asked. The fake could not express any of this because its DescribeChangeSet dropped DeploymentConfig; it now returns it, and createChangeSetSync accepts one. Also restricts the replay recovery guidance to UPDATE_FAILED. isRollbackable also covers CREATE_FAILED and UPDATE_ROLLBACK_FAILED, and a stack that never deployed successfully has no previous configuration to replay, so those states get their own guidance instead of impossible instructions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1931. CloudFormation refuses replacement-type updates while rollback is disabled. Express Mode disables rollback unless
--rollbackis passed, socdk deploy --expresssubmitted replacements that CloudFormation rejected, leaving the stack in UPDATE_FAILED with no rollback available. #1745 removed the express half of the guard condition and relaxed the test covering it; #1785 restructured the remainder intoif (!this.options.express).The condition is now derived once in
rollbackDisabled()(express ? rollback !== true : rollback === false) and consumed by the guard,deployConfig()and the failure diagnostic — which previously derivedrollbackEnabledasrollback !== false, so under--expressit reported rollback as enabled and advised re-running with--no-rollback. The guard is a single cross-mode check rather than an express-specific branch, and returnsreplacement-requires-rollbackso the existing confirm-and-retry-with-rollback prompt engages. A newCDK_TOOLKIT_W5903carries a typed payload withdetectedBy: 'change-set' | 'service-error'.When the stack is already in a failed state the guard instead throws
ReplacementRequiresUnwedgecarrying recovery guidance for that state - onlyUPDATE_FAILEDis told to replay the previous configuration, since a failed initial create has none - because deploying with rollback enabled cannot update an already-failed stack; since the confirmation defaults to yes, returning a result there meant a non-interactive caller ran a deployment CloudFormation rejects withValidationError: non-terminal [UPDATE_FAILED].--method=directhas no change set to inspect and is deliberately not refused, because replaying the previous configuration that way is the only exit from an already-stranded stack; the rejection is detected after the fact and the user is routed to--express --rollback, with the original CloudFormation error still propagating. That routing is gated onexpress, since a standard-mode--no-rollbackdeployment hits the same CloudFormation rejection and must not be pushed towards a sticky mode that gives upcdk rollback.Verified against CloudFormation on a live ECS stack (Memory 1024 to 2048, a genuine replacing change): the gate fires and submits nothing, and retrying with rollback enabled cannot rescue a stack that is already failed. The direct-path hook firing on CloudFormation's resource-level reason, and the documented unwedge still succeeding with the gate upstream, were verified the same way.
Notes:
lib/index.tsre-exports./payloads, soReplacementRequiresRollbackandReplacedResourceare supported surface; the matched reason constant stays private, asdeploy-stack.tsis not in the barrel.FakeCloudFormationchanged meaning for all future tests: express implies rollback disabled,FailReasonproduces resource-level failure events, andDescribeChangeSetnow returns the persistedDeploymentConfigas the real API does. No other suite needed changing.PolicyActiononly, unchanged and sufficient for (cli): #1745 removed the express-mode replacement guard, so --express deploys replacements with rollback disabled #1931, pinned by a negative test and tracked in Possible gap: replacement detection only inspects PolicyAction, not Replacement #1971.CreateChangeSetandExecuteChangeSetcannot override it, so rollback safety is read from the persistedDeploymentConfigwhen executing an existing change set and a conflicting execute request is refused withChangeSetRollbackPolicyMismatchrather than silently executed or recreated.Related: #1972. Docs in #1970. Time-bound: CloudFormation's server-side fix has a tentative ECD of 2026-11-15, after which the guard, the direct-path hook, the matched reason string and
CDK_TOOLKIT_W5903should be removed together. Draft pending a scoped re-review.#NNNNhere meansaws/aws-cdk-cli; noteaws/aws-cdk#1931is an unrelated closed issue.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.