Hey there! 👋 Thanks for taking the time to report this bug — we appreciate it!
Please keep in mind: PegaProx is developed and maintained entirely by volunteers in our free time. We do our best, but we can't work magic 🪄 — please be patient, we'll get to your issue as soon as we can.
Want to help keep PegaProx alive?
- ⭐ Star the project — it's free and helps a lot!
- 💖 Become a Sponsor — helps us dedicate more time to development
- 🤝 Contributing code or docs is always welcome too!
Describe the bug
ESXi to Proxmox migration fails when using the “fancy” / advanced migration methods, especially Live Mirror.
The migration tries QEMU SSH driver, HTTPS-backed boot, SSHFS-backed boot and NBD/remote boot style fallback checks. These fail with Paramiko/libssh-style SSH banner errors:
Paramiko auth failed (3 methods): M1(ki-transport): Error reading SSH protocol banner; M2(ki-client): Error reading SSH protocol banner; M3(password): Error reading SSH protocol banner
After that, PegaProx says it switches to offline copy:
=== DISK COPY: SSH → local-zfs ===
However, the offline fallback also fails immediately with the same Paramiko error. It looks like the offline fallback is still using the same failing Paramiko/libssh path instead of using a plain OpenSSH/sshpass/dd copy.
Manual SSH to the ESXi host works after cleanup/recovery. A plain OpenSSH sshpass + dd style copy also worked successfully for another VM and transferred data normally. So the basic ESXi SSH path and Proxmox target storage are functional. The problem appears to be the advanced migration/probing paths and the offline fallback implementation.
There is also a second related problem: the log hint says to install qemu-utils, but on Proxmox VE 9 this is dangerous/misleading because Debian qemu-utils would remove important Proxmox packages. On my Proxmox node, qemu-img and qemu-nbd are already provided by pve-qemu-kvm.
Steps to Reproduce
- Add an ESXi host to PegaProx.
- Start an ESXi to Proxmox V2P migration.
- Select Live Mirror transfer mode.
- Target a Proxmox node with
local-zfs storage.
- Start the migration.
- PegaProx tries QEMU SSH / HTTPS / SSHFS / NBD remote boot checks.
- These fail with
Paramiko auth failed ... Error reading SSH protocol banner.
- PegaProx switches to offline copy, but the offline copy also fails immediately with the same Paramiko error.
Additional observation: when leaving transfer mode on Auto, PegaProx also tried several methods and then fell back to offline copy. In one earlier case, a plain dd over ssh style copy did run and transferred successfully, so the simple copy method itself can work. The repeated failure appears when PegaProx’s advanced/Live Mirror path and fallback still go through the Paramiko/libssh logic.
Expected behavior
If Live Mirror / QEMU SSH / HTTPS / SSHFS / NBD probing fails, PegaProx should fall back to a simple, reliable offline copy using plain OpenSSH/sshpass, for example conceptually:
sshpass -e ssh root@ESXI_HOST "dd if=/vmfs/volumes/.../disk-flat.vmdk bs=4M" | dd of=/dev/zvol/... bs=4M status=progress
The offline fallback should not continue using the same Paramiko/libssh path that already failed.
PegaProx should also avoid repeated SSH authentication attempts against ESXi during probing, because this can trigger ESXi login throttling/lockout. In my case, ESXi remote login had to be recovered through iLO/DCUI after repeated failed attempts.
The package hint should also be adjusted for Proxmox VE. On Proxmox VE, qemu-img and qemu-nbd may already be provided by pve-qemu-kvm; telling users to install Debian qemu-utils can be dangerous.
Environment
- PegaProx Version: 0.9.13
- Installation Method: Docker
- OS: Proxmox VE 9.2.2 / Debian 13 trixie
- Browser: Firefox on macOS
- Behind Reverse Proxy? HAProxy
- UI? Corporate / Modern Layout
Target Proxmox node example:
pve-manager/9.2.2
pve-qemu-kvm 11.0.0-3
qemu-server 9.1.15
sshfs installed
sshpass installed
netcat installed
Tool check on the Proxmox node:
/usr/bin/qemu-img
/usr/bin/qemu-nbd
/usr/bin/sshfs
/usr/bin/sshpass
/usr/bin/nc
pve-qemu-kvm: /usr/bin/qemu-img
pve-qemu-kvm: /usr/bin/qemu-nbd
APT conflict seen when trying to install Debian qemu-utils on Proxmox VE 9:
Installing:
qemu-block-extra
qemu-utils
REMOVING:
proxmox-ve
pve-container
pve-ha-manager
pve-manager
pve-qemu-kvm
qemu-server
spiceterm
Logs
[15:36:23] QEMU SSH driver test failed: Paramiko auth failed (3 methods): M1(ki-transport): Error reading SSH protocol banner; M2(ki-client): Error reading SSH protocol banner; M3(password): Error reading SSH protocol banner
[15:36:23] libssh cannot connect - trying HTTPS-backed boot (ESXi datastore)...
[15:36:23] QEMU HTTPS test failed: Paramiko auth failed (3 methods): M1(ki-transport): Error reading SSH protocol banner; M2(ki-client): Error reading SSH protocol banner; M3(password): Error reading SSH protocol banner
[15:36:23] libssh cannot connect - trying SSHFS-backed boot (OpenSSH)...
[15:36:23] SSHFS disk 0: NOT accessible at /tmp/v2p-REDACTED/VM-A/VM-A-flat.vmdk; debug: Paramiko auth failed (3 methods): M1(ki-transport): Error reading SSH protocol banner; M2(ki-client): Error reading SSH protocol banner; M3(password): Error reading SSH protocol banner
[15:36:23] SSHFS not available - trying to remount...
[15:36:23] No remote boot method available (QEMU-SSH/SSHFS/NBD all failed)
[15:36:23] Hint: install sshfs and qemu-utils (apt install sshfs qemu-utils) for more transfer options
[15:36:23] Switching to offline mode: copy first, then start VM
[15:36:23] === DISK COPY: SSH → local-zfs ===
[15:36:23] (Copying disks, VM will start after copy completes)
[15:36:24] Transfer: bs=64MB, compress=gzip, nc=yes
[15:36:24] Copying disk 0 (96.0 GB) → local-zfs:vm-1014-disk-0
[15:36:24] SSH+compress × 8
[15:36:24] SSH+compress rc=255 after 0s, output tail:
[15:36:24] Paramiko auth failed (3 methods): M1(ki-transport): Error reading SSH protocol banner; M2(ki-client): Error reading SSH protocol banner; M3(password): Error reading SSH protocol banner
[15:36:24] Copy failed!
[15:36:24] Disk copy failed
[15:36:24] FAILED: Disk copy to local storage failed. Check logs above for details. Ensure qemu-utils, sshfs and sshpass are installed on the Proxmox node.
[15:36:24] Phase: failed
Earlier related failure where the node circuit breaker interfered with the fallback copy:
[NODE] cluster 'cluster01' node 'pve02' blocked for 60s after 3 consecutive failures
SSH+compress rc=1 after 0s, output tail:
node 'pve02' in circuit-breaker backoff (59s remaining)
Copy failed!
Disk copy failed
[NODE] cluster 'cluster01' observed 'pve02' back online via /nodes aggregate — clearing breaker
Manual working copy style observed on another migration:
sshpass -e ssh root@ESXI_HOST "dd if=/vmfs/volumes/.../disk-flat.vmdk bs=4M" | dd of=/dev/zvol/rpool/data/vm-XXXX-disk-0 bs=4M
That transfer actually ran and saturated the expected 1G management path, so the simple SSH copy route itself was working.
Screenshots
Screenshots show the PegaProx migration task failing with the log above.
I also had to recover ESXi remote login through iLO/DCUI after repeated failed authentication attempts, but I am not attaching sensitive screenshots with hostnames/passwords.
Checklist
Describe the bug
ESXi to Proxmox migration fails when using the “fancy” / advanced migration methods, especially Live Mirror.
The migration tries QEMU SSH driver, HTTPS-backed boot, SSHFS-backed boot and NBD/remote boot style fallback checks. These fail with Paramiko/libssh-style SSH banner errors:
After that, PegaProx says it switches to offline copy:
However, the offline fallback also fails immediately with the same Paramiko error. It looks like the offline fallback is still using the same failing Paramiko/libssh path instead of using a plain OpenSSH/sshpass/dd copy.
Manual SSH to the ESXi host works after cleanup/recovery. A plain OpenSSH
sshpass + ddstyle copy also worked successfully for another VM and transferred data normally. So the basic ESXi SSH path and Proxmox target storage are functional. The problem appears to be the advanced migration/probing paths and the offline fallback implementation.There is also a second related problem: the log hint says to install
qemu-utils, but on Proxmox VE 9 this is dangerous/misleading because Debianqemu-utilswould remove important Proxmox packages. On my Proxmox node,qemu-imgandqemu-nbdare already provided bypve-qemu-kvm.Steps to Reproduce
local-zfsstorage.Paramiko auth failed ... Error reading SSH protocol banner.Additional observation: when leaving transfer mode on
Auto, PegaProx also tried several methods and then fell back to offline copy. In one earlier case, a plaindd over sshstyle copy did run and transferred successfully, so the simple copy method itself can work. The repeated failure appears when PegaProx’s advanced/Live Mirror path and fallback still go through the Paramiko/libssh logic.Expected behavior
If Live Mirror / QEMU SSH / HTTPS / SSHFS / NBD probing fails, PegaProx should fall back to a simple, reliable offline copy using plain OpenSSH/sshpass, for example conceptually:
The offline fallback should not continue using the same Paramiko/libssh path that already failed.
PegaProx should also avoid repeated SSH authentication attempts against ESXi during probing, because this can trigger ESXi login throttling/lockout. In my case, ESXi remote login had to be recovered through iLO/DCUI after repeated failed attempts.
The package hint should also be adjusted for Proxmox VE. On Proxmox VE,
qemu-imgandqemu-nbdmay already be provided bypve-qemu-kvm; telling users to install Debianqemu-utilscan be dangerous.Environment
Target Proxmox node example:
Tool check on the Proxmox node:
APT conflict seen when trying to install Debian
qemu-utilson Proxmox VE 9:Logs
Earlier related failure where the node circuit breaker interfered with the fallback copy:
Manual working copy style observed on another migration:
That transfer actually ran and saturated the expected 1G management path, so the simple SSH copy route itself was working.
Screenshots
Screenshots show the PegaProx migration task failing with the log above.
I also had to recover ESXi remote login through iLO/DCUI after repeated failed authentication attempts, but I am not attaching sensitive screenshots with hostnames/passwords.
Checklist