Skip to content

TEZ-4759: An external AM never re-registers after a ZooKeeper session loss - #543

Open
deniskuzZ wants to merge 1 commit into
apache:masterfrom
deniskuzZ:TEZ-4759
Open

deniskuzZ wants to merge 1 commit into
apache:masterfrom
deniskuzZ:TEZ-4759

Conversation

@deniskuzZ

Copy link
Copy Markdown
Member
  • ZkAMRegistry.add() wrote the AMRecord once, as a bare EPHEMERAL node; a session expiry deletes it and nothing re-creates it, so clients see an empty registry while the AM keeps running
  • Hold a Curator PersistentNode per AMRecord: it re-creates the node after a session loss or a delete; remove() closes it
  • add() fails when the node is not created within the ZooKeeper connection timeout
  • TestZkAMRegistry: after the ZooKeeper server is replaced, the AMRecord node is back

… loss

- ZkAMRegistry.add() wrote the AMRecord once, as a bare EPHEMERAL node; a session expiry deletes it and nothing
  re-creates it, so clients see an empty registry while the AM keeps running
- Hold a Curator PersistentNode per AMRecord: it re-creates the node after a session loss or a delete; remove()
  closes it
- add() fails when the node is not created within the ZooKeeper connection timeout
- TestZkAMRegistry: after the ZooKeeper server is replaced, the AMRecord node is back
@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 3m 25s master passed
+1 💚 compile 2m 37s master passed
+1 💚 checkstyle 0m 31s master passed
+1 💚 javadoc 0m 21s master passed
+0 🆗 spotbugs 1m 2s tez-dag in master has 537 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 13s the patch passed
+1 💚 codespell 0m 50s No new issues.
+1 💚 compile 2m 34s the patch passed
+1 💚 javac 2m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 23s the patch passed
+1 💚 javadoc 0m 20s the patch passed
+1 💚 spotbugs 1m 9s the patch passed
_ Other Tests _
+1 💚 unit 60m 51s root in the patch passed.
+1 💚 asflicense 0m 25s The patch does not generate ASF License warnings.
77m 56s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-543/2/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux fb13607be12e 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / ae2756e
Default Java Eclipse Adoptium-21.0.12.1+1-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-543/2/testReport/
Max. process+thread count 1396 (vs. ulimit of 5500)
modules C: tez-dag U: tez-dag
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-543/2/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants