Notes to self

Inception: running Vagrant inside Vagrant with KVM

Since I package and maintain Vagrant and vagrant-libvirt for Fedora, I have a need to test new builds. But since I run just one version of Fedora and I don’t really feel like testing it directly on my host system, I actually use Vagrant to test Vagrant. In other words I run Vagrant with KVM and inside I download the new builds and run Vagrant with KVM again. This is possible because KVM supports nested virtualization. In fact I already wrote about setting that up with virt-manager. But today I show you how to do it using Vagrant itself.

First we need to configure vagrant-libvirt on the host system to enable KVM nested virtualization:

config.vm.provider :libvirt do |libvirt|
  # Enable KVM nested virtualization
  libvirt.nested = true
  libvirt.cpu_mode = "host-model"
end

I have the above lines in my user’s Vagrantfile so these options are always applied (read more on that). If we would try to spin up a new machine with Vagrant and try to run Vagrant with libvirt inside it, we would get something like the following.

$ vagrant up
Bringing machine 'default' up with 'libvirt' provider...
Error while activating network: Call to virNetworkCreate failed: internal error: Network is already in use by interface eth0.

This happens because Vagrant already used the default network on eth0 interface. We can fix it by specifying a different one in our Vagrantfile:

config.vm.provider :libvirt do |p|
   p.management_network_name = 'vagrant-libvirt-new'
   p.management_network_address = '192.168.124.0/24'
end

This helps but trying vagrant up again still fails. This time Vagrant complains about memory allocation:

$ vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Starting domain.
There was an error talking to Libvirt. The error message is shown
below:

Call to virDomainCreateWithFlags failed: internal error: early end of file from monitor: possible problem:
2015-05-26T10:30:10.991227Z qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory

The thing is that we allocated a default 512 MB for our virtual machine and then again trying to allocate another 512 MB inside for our nested environment. Lucky for us, this is easy to fix by increasing the number in the Vagrantfile:

config.vm.provider :libvirt do |libvirt|
  # Enable KVM nested virtualization
  libvirt.nested = true
  libvirt.cpu_mode = "host-model"
  # Increase memory allocation
  libvirt.memory = 1024
end

That’s it! As you can see Vagrant with vagrant-libvirt provider makes these things really simple.

Check out my book
Deployment from Scratch is unique Linux book about web application deployment. Learn how deployment works from the first principles rather than YAML files of a specific tool.
by Josef Strzibny
RSS