Skip to content

[Fix][CDC] Avoid replaying completed snapshot splits after restore#10859

Open
DanielLeens wants to merge 1 commit intoapache:devfrom
DanielLeens:daniel-st-3488-upstream
Open

[Fix][CDC] Avoid replaying completed snapshot splits after restore#10859
DanielLeens wants to merge 1 commit intoapache:devfrom
DanielLeens:daniel-st-3488-upstream

Conversation

@DanielLeens
Copy link
Copy Markdown
Contributor

Summary

  • skip add-back for snapshot splits that are already finished and checkpointed in enumerator state
  • preserve checkpointed completion state so snapshot phase can advance to incremental phase after restore
  • add focused tests for completed+checkpointed vs finished-but-uncheckpointed snapshot split restore paths

Background

During CDC snapshot failover, SnapshotSplitAssigner#addSplits can re-enqueue a snapshot split that has already finished reading and whose completion watermark has already been checkpointed. That can keep the snapshot phase from completing and block the job from entering incremental reading.

Verification

  • ./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-base -am -nsu -Dgit.skip=true -Dmaven.gitcommitid.skip=true spotless:apply
  • attempted targeted tests for SnapshotSplitAssignerTest,HybridSplitAssignerTest, but the clean dev checkout currently hits unrelated compile failures before those tests can complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant