Make sure only a single build runs using VMs

This makes sure there are no race conditions on our workers or during deployment.
We can only use one hypervisor at a time.