How to force a DHCP server to hand over /32s, while running on a different /32 subnet.
What? Why?
These are all very valid questions. Let me explain.
At my job, I designed and built a virtualization platform for creating hourly billed virtual machines. One of the challenges was building a robust and scalable network. I wanted to maximize IP space utilization efficiency as well, so allocating a fixed subnet per each hypervisor was out of the question.
All this led me to design a Layer 3-only network, where routes to each virtual machine are redistributed over iBGP.
However, this somewhat complicates things. Since each virtual machine is basically only connected to the hypervisor, one would typically need a /31 for each virtual machine. One address for the VM and one for the default gateway, isn’t that right?
It turns out that it is possible to overcome this limitation with the following design:
VM interface IP (on hypervisor): 172.16.0.1/32 VM interface IP (in the VM): 192.168.1.26/32 Routes in the VM: 172.16.0.1 dev eth0 scope link default via 172.16.0.1 dev eth0 Routes on the hypervisor: 192.168.1.26 via 172.16.0.1 dev virbr13 metric 1
This solution actually works fantastically.
However, there is one issue. How do you assign addresses and routes to the VMs?
Hacking DHCP
Manual IP assignment per each virtual machine is painful and slow. Would it be possible to assign addresses automatically? Maybe even with a standard protocol like DHCP?
After spending lots of hours researching possible solutions and a hefty amount of trial and error, I found the following configuration works reliably:
option domain-name-servers 9.9.9.10, 8.8.8.8; option subnet-mask 255.255.255.255; option routers 172.16.0.1; shared-network "virbr13" { subnet 172.16.0.1 netmask 255.255.255.255 { } subnet 192.168.1.26 netmask 255.255.255.255 { range 192.168.1.26; } } lease-file-name "/var/lib/dhcpd/virbr13.leases"; deny declines; deny duplicates; one-lease-per-client true; ignore-client-uids true; authoritative; allow unknown-clients;
dhcpd
will listen for DHCP requests on 172.16.0.1
. Once such request arrives, it will however advertise 192.168.1.26/32
with 172.16.0.1
as the default gateway, as well as an interface route for 172.16.0.1/32
.
Note that only few DHCP servers allow this sort of configuration.
ISC dhcpd
works, dnsmasq
does not.
harisha
December 21, 2022 — 7:04 pm
Thanks for the writeup. Very useful.
I’m trying to get /32 addresses using this technique for all clients. First client gets 192.168.1.26 with 255.255.255.255, but all other clients get “no free leases” error. If I try to add a range in the conf file (instead of just 192.168.1.26), I get error that the range isn’t in the netmask (which of course only has 1 IP).
Any way to get multiple clients get different IPs, but force netmask to 255.255.255.255 ?
Filip Hruška
December 23, 2022 — 4:36 am
Hi, generally this approach only works if you have a point-to-point type network with only one client in it. In my case, each VM gets its own dedicated network bridge and runs its own DHCP instance.
In case you have multiple hosts in your network, you would generally want to use a more suitable subnet mask that’s not a /32