Repository navigation
integration: Verify incremental transfers for any provider pair - #545
Open
claudiubelu wants to merge 2 commits into
Open
claudiubelu wants to merge 2 commits into
claudiubelu wants to merge 2 commits into
Conversation
deploy_replica_disks() allocated a new, empty loop device on every execution, ignoring the volumes_info from the previous one. This dropped the disks' "replica_state", turning every transfer execution into a full sync, and leaked the previously allocated loop devices. Disks allocated by a previous execution are now reused, like the real import providers do. The replicator only loads its state file at startup, and it is already running when replicate_disks() is called: deploy_replica_source_resources() starts it without any state. It was only restarted with the replica state on reused pool minions, so incremental executions on fresh source minions were always full syncs. The replicator is now always restarted with the replica state for incremental executions.
claudiubelu
force-pushed
the
integration-tests-improvements
branch
from
October 21, 2026 12:19
295b014 to
96aed4d
Compare
Disk integrity verification is now enabled for both executions of test_incremental_replica_transfer. The execution if the source and destination checksums differ, so the transferred content is verified for any provider pair, not only with the core test import provider. The test also asserts that the incremental execution wrote less data than the full one, which was not checked before: every execution was a full sync with the core test providers, and the test still passed. Test providers can now implement exec_test_vm, which runs a shell command inside the test instance, and mutate_test_instance_data is built on top of it. The same hook can then be used by other tests that need to act on the guest. Also adds ssh_utils, so providers whose test instances are reachable over SSH only have to supply the instance's address. The core test export provider's instances were raw loop devices, so the only way to change their data was a hard-coded write at an offset. Instances are now formatted with a tiny ext4 filesystem holding only the host's static busybox, and exec_test_vm chroots into it. This keeps the transfer size small, compared to writing a full OS image. Document the new busybox requirement in the integration tests README, and install it in the integration tests workflow.
claudiubelu
force-pushed
the
integration-tests-improvements
branch
from
October 21, 2026 12:57
96aed4d to
8b30fc3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test import provider's
deploy_replica_disks()allocated a new, empty loop device on every execution, ignoring the volumes_info from the previous one. This dropped the disks'replica_state, turning every transfer execution into a full sync, and leaked the previously allocated loop devices.Disks allocated by a previous execution are now reused, like the real import providers do.
The replicator only loads its state file at startup, and it is already running when
replicate_disks()is called:deploy_replica_source_resources()starts it without any state. It was only restarted with the replica stateon reused pool minions, so incremental executions on fresh source minions were always full syncs.
The replicator is now always restarted with the replica state for incremental executions.
Disk integrity verification is now enabled for both executions of
test_incremental_replica_transfer. The execution if the source and destination checksums differ, so the transferred content is verifiedfor any provider pair, not only with the core test import provider.
The test also asserts that the incremental execution wrote less data than the full one, which was not checked before: every execution was a full sync with the core test providers, and the test still passed.
Test providers can now implement exec_test_vm, which runs a shell command inside the test instance, and
mutate_test_instance_datais built on top of it. The same hook can then be used by other tests that need to act on the guest.Also adds
ssh_utils, so providers whose test instances are reachable over SSH only have to supply the instance's address.The core test export provider's instances were raw loop devices, so the only way to change their data was a hard-coded write at an offset. Instances are now formatted with a tiny ext4 filesystem holding only the host's static
busybox, andexec_test_vmchroots into it. This keeps the transfer size small, compared to writing a full OS image.Document the new busybox requirement in the integration tests README, and install it in the integration tests workflow.