Skip to content

integration: Verify incremental transfers for any provider pair - #545

Open
claudiubelu wants to merge 2 commits into
cloudbase:mainfrom
claudiubelu:integration-tests-improvements
Open

claudiubelu wants to merge 2 commits into
cloudbase:mainfrom
claudiubelu:integration-tests-improvements

Conversation

@claudiubelu

@claudiubelu claudiubelu commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Test import provider's deploy_replica_disks() allocated a new, empty loop device on every execution, ignoring the volumes_info from the previous one. This dropped the disks' replica_state, turning every transfer execution into a full sync, and leaked the previously allocated loop devices.

Disks allocated by a previous execution are now reused, like the real import providers do.

The replicator only loads its state file at startup, and it is already running when replicate_disks() is called: deploy_replica_source_resources() starts it without any state. It was only restarted with the replica state
on reused pool minions, so incremental executions on fresh source minions were always full syncs.

The replicator is now always restarted with the replica state for incremental executions.

Disk integrity verification is now enabled for both executions of test_incremental_replica_transfer. The execution if the source and destination checksums differ, so the transferred content is verified
for any provider pair, not only with the core test import provider.

The test also asserts that the incremental execution wrote less data than the full one, which was not checked before: every execution was a full sync with the core test providers, and the test still passed.

Test providers can now implement exec_test_vm, which runs a shell command inside the test instance, and mutate_test_instance_data is built on top of it. The same hook can then be used by other tests that need to act on the guest.

Also adds ssh_utils, so providers whose test instances are reachable over SSH only have to supply the instance's address.

The core test export provider's instances were raw loop devices, so the only way to change their data was a hard-coded write at an offset. Instances are now formatted with a tiny ext4 filesystem holding only the host's static busybox, and exec_test_vm chroots into it. This keeps the transfer size small, compared to writing a full OS image.

Document the new busybox requirement in the integration tests README, and install it in the integration tests workflow.

@claudiubelu claudiubelu changed the title integration: Verify incremental transfers for any provider pai integration: Verify incremental transfers for any provider pair Oct 19, 2026
deploy_replica_disks() allocated a new, empty loop device on every
execution, ignoring the volumes_info from the previous one. This dropped
the disks' "replica_state", turning every transfer execution into a full
sync, and leaked the previously allocated loop devices.

Disks allocated by a previous execution are now reused, like the real
import providers do.

The replicator only loads its state file at startup, and it is already
running when replicate_disks() is called: deploy_replica_source_resources()
starts it without any state. It was only restarted with the replica state
on reused pool minions, so incremental executions on fresh source minions
were always full syncs.

The replicator is now always restarted with the replica state for
incremental executions.
@claudiubelu
claudiubelu force-pushed the integration-tests-improvements branch from 295b014 to 96aed4d Compare October 21, 2026 12:19
Disk integrity verification is now enabled for both executions of
test_incremental_replica_transfer. The execution if the source and
destination checksums differ, so the transferred content is verified
for any provider pair, not only with the core test import provider.

The test also asserts that the incremental execution wrote less data
than the full one, which was not checked before: every execution was a
full sync with the core test providers, and the test still passed.

Test providers can now implement exec_test_vm, which runs a shell command
inside the test instance, and mutate_test_instance_data is built on top of it.
The same hook can then be used by other tests that need to act on the guest.

Also adds ssh_utils, so providers whose test instances are reachable
over SSH only have to supply the instance's address.

The core test export provider's instances were raw loop devices, so
the only way to change their data was a hard-coded write at an offset.
Instances are now formatted with a tiny ext4 filesystem holding only the
host's static busybox, and exec_test_vm chroots into it. This keeps the
transfer size small, compared to writing a full OS image.

Document the new busybox requirement in the integration tests README,
and install it in the integration tests workflow.
@claudiubelu
claudiubelu force-pushed the integration-tests-improvements branch from 96aed4d to 8b30fc3 Compare October 21, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant