"Post-processor failed: unexpected EOF" vagrant post processor on libvirt image

[J-PC]

Overview of the Issue

I get an error "Post-processor failed: unexpected EOF" when trying to build a box from an existing libvirt kvm image.
I prebuild a kvm image with the qemu provider.
I tried to convert it with the vagrant post-processor but it fails. I have no problem building an image in one step but it doesn't work in two steps.
I used the example from the documentation using Artifice post-processor and overrode the provider.

Plugin and Packer version

I used :

• Packer v1.10.1
• packer-plugin-qemu v1.0.10
• packer-plugin-vagrant v1.1.2

Packer Buildfile

I used the following configuration :

packer {
  required_plugins {
    vagrant = {
      version = ">= 1.1.1"
      source  = "github.com/hashicorp/vagrant"
    }
  }
}

source "null" "fedora" {
  communicator = "none"
}

build {
  sources = [
    "source.null.fedora"
  ]

  post-processors {
    post-processor "artifice" {
      files = [
        "packer_images/fedora_workstation.qcow2"
      ]
    }

    post-processor "vagrant" {
      keep_input_artifact = true
      provider_override = "libvirt"
    }
  }
}

Operating system and Environment details

I build on Fedora 38

Logs

null.fedora: output will be in this color.

==> null.fedora: Running post-processor: (type artifice)
==> null.fedora (artifice): Using these artifact files: packer_images/fedora_workstation.qcow2
==> null.fedora: Running post-processor: (type vagrant)
==> null.fedora (vagrant): Creating a dummy Vagrant box to ensure the host system can create one correctly
==> null.fedora (vagrant): Creating Vagrant box for 'libvirt' provider
Build 'null.fedora' errored after 42 milliseconds 317 microseconds: 1 error(s) occurred:

• Post-processor failed: unexpected EOF

==> Wait completed after 42 milliseconds 507 microseconds

==> Some builds didn't complete successfully and had errors:
--> null.fedora: 1 error(s) occurred:

• Post-processor failed: unexpected EOF

==> Builds finished but no artifacts were created.
panic: interface conversion: interface {} is nil, not string
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin:
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: goroutine 9 [running]:
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant.(*LibVirtProvider).Process(0x11215ee?, {0x1447508, 0xc0007be6f0}, {0x1447550, 0xc0006c2720}, {0xc0006a8000, 0x15})
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant/libvirt.go:71 +0x736
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant.(*PostProcessor).PostProcessProvider(0xc0006aa968?, {0xc0006aa968, 0x7}, {0x1441010, 0x1c6d1c8}, {0x1447508, 0xc0007be6f0}, {0x1447550, 0xc0006c2720})
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant/post-processor.go:176 +0xa7a
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant.(*PostProcessor).PostProcess(0xc0001b38c0, {0xc00019c010?, 0x0?}, {0x1447508, 0xc0007be6f0}, {0x1447550?, 0xc0006c2720?})
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-vagrant/post-processor/vagrant/post-processor.go:251 +0x154
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-sdk/rpc.(*PostProcessorServer).PostProcess(0xc0003ca100, 0x19cdc8?, 0xc00079a990)
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: github.com/hashicorp/packer-plugin-sdk@v0.5.2/rpc/post_processor.go:111 +0x197
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: reflect.Value.call({0xc0001a0a80?, 0xc00019e728?, 0x13?}, {0x1121686, 0x4}, {0xc0000b6ef8, 0x3, 0x3?})
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: reflect/value.go:586 +0xb07
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: reflect.Value.Call({0xc0001a0a80?, 0xc00019e728?, 0x8000000000000000?}, {0xc0002076f8?, 0x9?, 0x0?})
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: reflect/value.go:370 +0xbc
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: net/rpc.(*service).call(0xc0003ca140, 0x1128408?, 0x9?, 0xc00019c0a0, 0xc0006b0300, 0x9?, {0xf55f80?, 0xc00019cdb8?, 0x0?}, {0xf23e60, ...}, ...)
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: net/rpc/server.go:382 +0x226
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: created by net/rpc.(*Server).ServeCodec
2024/03/01 10:15:03 packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64 plugin: net/rpc/server.go:479 +0x3fe
2024/03/01 10:15:03 /home/jorge/.config/packer/plugins/github.com/hashicorp/vagrant/packer-plugin-vagrant_v1.1.2_x5.0_linux_amd64: plugin process exited
2024/03/01 10:15:03 [INFO] (telemetry) ending vagrant
2024/03/01 10:15:03 ui error: Build 'null.fedora' errored after 42 milliseconds 317 microseconds: 1 error(s) occurred:

• Post-processor failed: unexpected EOF
  2024/03/01 10:15:03 ui:
  ==> Wait completed after 42 milliseconds 507 microseconds
  2024/03/01 10:15:03 machine readable: error-count []string{"1"}
  2024/03/01 10:15:03 ui error:
  ==> Some builds didn't complete successfully and had errors:
  2024/03/01 10:15:03 machine readable: null.fedora,error []string{"1 error(s) occurred:\n\n* Post-processor failed: unexpected EOF"}
  2024/03/01 10:15:03 ui error: --> null.fedora: 1 error(s) occurred:

• Post-processor failed: unexpected EOF
  2024/03/01 10:15:03 ui:
  ==> Builds finished but no artifacts were created.
  2024/03/01 10:15:03 [INFO] (telemetry) Finalizing.
  2024/03/01 10:15:03 waiting for all plugin processes to complete...
  2024/03/01 10:15:03 /usr/bin/packer: plugin process exited
  2024/03/01 10:15:03 /usr/bin/packer: plugin process exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Packer crashed! This is always indicative of a bug within Packer.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Packer1 so that we can fix this.

!!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

[lbajolet-hashicorp]

Hi @J-PC,

Thanks for the report.

Looking at the code, while the Packer Crash is definitively an error on our part, the core issue here is that your approach will not work at all.
The reason is the post-processor for libvirt expects something that was made by the libvirt plugin, so Artifice won't cut it as its state is not filled-in, and trying to access the data from it will crash.
To be fair, that crash is on us as the plugin shouldn't assume everything is properly set, and therefore the cast to a string should only be done once we've validated that what we expected from the artifact is indeed here.

Not sure how we can fix your config at first glance, may I ask though if you've considered building this with the qemu builder given that you have a qcow2 image to process by the vagrant post-processor?

I believe something like the following might work:

source "qemu" "fedora" {
  iso_path                 = "packer_images/fedora_workstation.qcow2"
  iso_checksum             = "none"
  disk_image               = true
  ssh_username             = "<username>"
  ssh_password             = "<password>"
  cpu_model                = "host"
  cpus                     = 4
  ssh_file_transfer_method = "sftp"
  boot_wait                = "3s"
}

build {
  sources = ["qemu.fedora"]

  post-processor "vagrant" {}
}

I tested this locally with a debian image, I do get a box as the output for it, please let me know if this would be a valid workaround for your issue here.
To be fair the current model for vagrant as a post-processor is brittle, and you're not the first one to report behaviours that are confusing, it is a plugin we should spend time on to fix it, but at the moment time is a scarce resource unfortunately.

I'll keep this open while we discuss what to do with this plugin, and how we can address scenarios like yours efficiently.