Routed networks in VirtualBox

I recently had a project at work where the project's requirements forced me to think outside the box. They were:

• Spin up hundreds or thousands of VM's dynamically without much intervention from netops. These VM's would exist across many hypervisors and be provisioned dynamically.
• Allow inbound traffic to the guest VM's from networks outside the hypervisor.

The built-in VirtualBox networking options really wouldn't fit the bill. Here's why:

• Bridged networking would require a lot of prior planning and work from netops. Would all the hypervisors be in the right VLAN to get the various subnets required? Some of the hypervisors might reside in smaller existing networks that would be /22-/24. Nope, too much work to support that across all of our locations.
• Internal networks can only communicate with VM's residing on the same hypervisor. External access is a must.
• Host-only networks only allow the VM to communicate with the hypervisor and other VM's. Similar to internal but not the outside world.
• NAT networking would be a huge pain to map all of the inbound ports to VM ports. Reducing work was a goal here.

So what's the solution? Routed networking of course! What's that? There's no routed network option? Sure there is. It's called host-only networking with a local DHCP server. Next, you enable routing at the OS level. In Linux, set net.ipv4.ip_forward to 1 (for v4).

That solves the problem with VM's getting outside the hypervisor. The next problem is the return traffic knowing how to get back to the VM. Which hypervisor sent it? It's not bridged so it can't ARP. It's not NAT'd so it's not the hypervisor MAC. How do you point return traffic to the right server?

Your answer here is a dynamic routing protocol. We use OSPF, and bird is the word. A simple bird instance can advertise the IP network from vboxnet0 to the rest of your network over OSPF. Simple enough way to bring up and tear down a lot of networks with little effort. You don't have to use OSPF; you could use something else like IS-IS (which we also use but not here).

In summary, host-only network + DHCP server on hypervisor + routing + OSPF = routed VM networking in VirtualBox. It works really well and is being used for our in-house Selenium testing (taste.weebly.com).