Hazelcast impl#1011
Open
ironcerocloudbees wants to merge 44 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a Hazelcast-backed coordination mode to allow the Gerrit Trigger Plugin’s BuildMemory state (and related coordination concerns) to work in distributed Jenkins/CloudBees HA/HS deployments.
Changes:
- Added a Hazelcast-based
BuildMemoryStorageimplementation plus supporting Hazelcast serialization/config/entry-processors. - Extended the coordination SPI/factory to support additional strategies (event + notification claiming) and provider lifecycle (initialize/shutdown).
- Added/updated test infrastructure and integration tests, plus a Maven profile to run the test suite with Hazelcast enabled.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest/common/gerrit-trigger.xml | Adds LocalData config fixture for Hazelcast cancellation integration tests. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/VoteSameTopicTest.java | Clears Hazelcast maps in teardown to avoid cross-test pollution. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/SpecGerritTriggerHudsonTest.java | Clears Hazelcast maps in teardown to avoid cross-test pollution. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spec/ParameterModeJenkinsTest.java | Adds an @After hook to clear Hazelcast state between tests. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/replication/ReplicationQueueTaskDispatcherTest.java | Stabilizes time-based assertions and clears Hazelcast state before/after the class. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationIntegrationTest.java | Clears Hazelcast state in teardown to prevent test pollution when Hazelcast is active. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/BuildCancellationHazelcastIntegrationTest.java | Adds Hazelcast-mode-specific build cancellation integration tests and Hazelcast test rule usage. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestRule.java | Adds a JUnit rule to initialize/shutdown Hazelcast and set coordination-mode system property. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestListener.java | Adds a surefire listener to initialize Hazelcast once for the entire Hazelcast test profile run. |
| src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastTestHelper.java | Adds helper methods to clear Hazelcast maps and inspect BuildMemory map size for tests. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/storage/LocalBuildMemoryStorage.java | Adds setCancelling(...) and eventsMatch(...) for local storage semantics. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/NotificationClaimStrategy.java | Refactors notification claiming to a fluent withClaim(...) API with chainable handlers. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/EventClaimStrategy.java | Introduces event-claiming SPI for distributed duplicate-prevention. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/CoordinationModeProvider.java | Extends SPI to support configured-mode lookup plus provider lifecycle and event claim strategy creation. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/spi/BuildMemoryStorage.java | Extends storage SPI with setCancelling(...) and storage-defined event equivalence (eventsMatch). |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/PluginImpl.java | Initializes/shuts down coordination providers during plugin start/stop. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java | Delegates cancelling flag persistence to storage; delegates event comparison to storage via eventsMatch. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalNotificationClaimStrategy.java | Implements the new fluent notification claim API for local mode. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/LocalEventClaimStrategy.java | Implements event claiming for local mode using the new fluent API. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/LocalCoordinationProvider.java | Adds event claim strategy and provider lifecycle methods to local provider. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/TriggeredProcessor.java | Adds Hazelcast EntryProcessor for atomic “triggered” updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/SetUnsuccessfulMessageProcessor.java | Adds Hazelcast EntryProcessor for atomic unsuccessful-message updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/SetCustomUrlProcessor.java | Adds Hazelcast EntryProcessor for atomic custom-URL updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/SetCancellingProcessor.java | Adds Hazelcast EntryProcessor for atomic “cancelling” flag updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/PolymorphicEventTypeAdapter.java | Adds Gson adapter to serialize/deserialize polymorphic GerritTriggeredEvent payloads. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/MemoryImprintDataSerializer.java | Adds Hazelcast Compact serializer for MemoryImprintData. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/MemoryImprintData.java | Adds DTO for storing memory imprints in Hazelcast via Compact serialization. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastNotificationClaimStrategy.java | Adds Hazelcast-backed distributed notification claim strategy with TTL. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastManager.java | Adds lifecycle management for embedded Hazelcast member initialization/shutdown. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastInstanceProvider.java | Adds a singleton provider for the Hazelcast instance. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastEventClaimStrategy.java | Adds Hazelcast-backed event claiming to avoid duplicate event processing across replicas. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastCoordinationProvider.java | Adds Hazelcast coordination provider implementation (storage + claim strategies + lifecycle). |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastConfig.java | Adds Hazelcast network/discovery/serialization configuration builder. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/HazelcastBuildMemoryStorage.java | Adds Hazelcast-backed BuildMemoryStorage implementation using IMap + EntryProcessors. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventIdentifier.java | Adds deterministic event ID generation for distributed keys/claims (with fallback). |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaimSerializer.java | Adds Hazelcast Compact serializer for EventClaim. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EventClaim.java | Adds DTO representing an event claim stored in Hazelcast. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EntryDataSerializer.java | Adds Hazelcast Compact serializer for EntryData. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/EntryData.java | Adds DTO representing per-job build entry data for Hazelcast storage. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/BuildStartedProcessor.java | Adds Hazelcast EntryProcessor for atomic “build started” updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/BuildMemoryKey.java | Adds Hazelcast map key based on deterministic event IDs. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/BuildCompletedProcessor.java | Adds Hazelcast EntryProcessor for atomic “build completed” updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/hazelcast/BuildCancelledProcessor.java | Adds Hazelcast EntryProcessor for atomic “build cancelled” updates. |
| src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/coordination/CoordinationModeFactory.java | Extends factory to create event claim strategy in addition to storage + notification claim strategy. |
| pom.xml | Adds Hazelcast dependency and a Maven profile to run tests with Hazelcast coordination enabled. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 22, 2026
rsandell
reviewed
May 24, 2026
| * | ||
| */ | ||
| public final class HazelcastConfig { | ||
|
|
Member
There was a problem hiding this comment.
Is the use of system properties because it is easier to inject into the pod? When this much coinfiguration is needed I would maybe consider a configuration form and CasC?
Member
There was a problem hiding this comment.
But I'm not sure if the rest of the gerrit trigger is CasC compatible yet :)
Collaborator
Author
There was a problem hiding this comment.
Is the use of system properties because it is easier to inject into the pod?
We were discussing (Steve and I) a lot about this topic at the beginning of the project. Initially, I developed a form in the UI to enable or disable the different modes and the configuration of the same. However, we thought that having these configurations at the level of system properties could make development (and configuration) easier (a different mode hot-changer could be very complex to track). Furthermore, we can add those system properties using Helm or other automation tools for traditional installations.
So, I would recommend keeping the system properties.
There was a problem hiding this comment.
I can see both points here (especially as the list may grow as more configuration is added). I wonder if it would make sense to have the ability to point to a properties file which holds all the values. A file could be mounted in kubernetes, etc. WDYT?
Member
There was a problem hiding this comment.
There is a limit to how many bytes you can have on the commandline, we might not be near it for most systems 🤷
rsandell
reviewed
May 24, 2026
rsandell
reviewed
May 24, 2026
rsandell
reviewed
May 24, 2026
rsandell
reviewed
May 31, 2026
rsandell
reviewed
May 31, 2026
When PS2 arrives while PS1 is building on another replica, abort PS1 via a distributed abort inbox (IMap + EntryAddedListener), since executeOnMember() targets the Hazelcast sidecar JVM in CLIENT mode. Also fixes premature forget() when QueueLoadBalancer moves queue items, and abstracts load-balancing detection into QueueCancellationStrategy SPI.
Collaborator
Author
|
Hey folks. I spent the last days fixing issues discovered by the tests. After that, I was able to successfully run the majority of the tests I had for development (Number 5 also fails with the local implementation and without any replication, so it seems this issue was not introduced by our changes). Tomorrow, I will revisit your last comments to continue with the Peer Review. However, since the current code passed those 22 tests, I think this is promising. Kind regards, |
rsandell
reviewed
Jun 1, 2026
rsandell
reviewed
Jun 1, 2026
rsandell
reviewed
Jun 1, 2026
rsandell
reviewed
Jun 1, 2026
rsandell
reviewed
Jun 1, 2026
|
|
||
| private static final Logger logger = LoggerFactory.getLogger(HazelcastManager.class); | ||
|
|
||
| private static volatile boolean initialized = false; |
Member
There was a problem hiding this comment.
This is still a bit dangerous to do and should try to be avoided. But I guess hazelcast itself is already a static field in itself so perhaps nothing can be done?
Collaborator
Author
There was a problem hiding this comment.
Do you think then that we should modify?
…ggered jobs When a Gerrit event triggers multiple jobs and the server-wide BuildCurrentPatchesOnly policy has isAbortNewPatchsets=true, cancelOutdatedEvents() was incorrectly marking the triggering event itself as outdated (self-cancellation). The second job's cancelOutdatedEvents call would find the first job's entry already in memory, and since isAbortNewPatchsets=true causes shouldIgnoreEvent() to return false even for equal patchset numbers, the entry was added to outdatedEvents and setCancelling(true) was called on the second job's entry — before the build was even scheduled. When a new patchset (PS2) later arrived, the isCancelling=true flag caused hasActiveBuildsForJob to return false for that job, so the PS1 build was never aborted. Fix: skip cancellation when the running event logically matches the new event (using storage.eventsMatch()), preventing any event from cancelling itself. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Hazelcast distributed storage implementation
This PR implements distributed BuildMemory storage using Hazelcast, enabling the Gerrit Trigger Plugin for distributed build memory.
Changes
Core Implementation
BuildMemoryStorageabstract class with two implementations:LocalBuildMemoryStorage: TreeMap-based storage for standalone Jenkins (existing behavior)HazelcastBuildMemoryStorage: Hazelcast IMap-based storage for distributed JenkinsPoints
BuildMemoryStorageinterface, remaininginfrastructure-agnostic
Testing done
CoordinationModeProvider extension is present
Submitter checklist