I continue to make use of the docker macvlan network support as it allows me to treat some of my containers as if they are virtual machines (VMs). Using this feature I can assign an IP address that is distinct from my host, but is still just a container running on the host. I’ve written about creating one, and expanding it.
As I’m now building out a new server and have selected NixOS as my base, I need to make some changes to how I’ve setup the docker macvlan. This blog post captures those changes.
While NixOS supports the declaration of containers, I’m not doing that right now by choice. It’ll make my migration easier and I can always go back and refactor. Thus there are just two things I need to include in my NixOS configuration:
- Enable docker support
- Modify the host network to route to the macvlan network
The first (enable docker support) is so very easy with NixOS. You need a single line added to your /etc/nixos/configuration.nix
|
1 |
virtualisation.docker.enable = true; |
You probably want to modify your user to be in the “docker” group allowing direct access to docker commands vs. needing to sudo each time.
There is a third thing we need to do, create the docker macvlan network. I don’t have this baked into my NixOS configuration because I was too lazy to write an idempotent version of doing it and figuring out where in the start up sequence to make it run. This turns out to be just a one line script:
|
1 2 3 4 5 6 |
$ docker network create -d macvlan -o parent=enp3s0 \ --subnet 192.168.1.0/24 \ --gateway 192.168.1.1 \ --ip-range 192.168.1.64/30 \ --aux-address 'host=192.168.1.67' \ myNewNet |
Docker will persist this network configuration across reboots.
If you stop here, you will be able to create containers with their own IP address. I pass along these two docker command line options to create a container with it’s own IP:
|
1 2 |
--network myNewNet \ --ip 192.168.1.64 \ |
The docker macvlan network I’ve defined has 4 IPs reserved, but you can specify a larger ip-range if you want when you create the docker mavlan network.
However, if you did stop here, you would not be able to reach the container running on 192.168.1.64 from the host. This is the second change to our Nix configuration (modify the host network to route to the macvlan network). In my original post I used a script to create the route from host to container, as this wasn’t persistent I needed to run that script after every boot.
One way to do a similar thing in NixOS is to create a systemd service. I was exploring this and did figure it out. However, I was wrong in my approach. While this worked, it wasn’t the best way to do it. NixOS has networking.macvlans which is a more NixOS-y way to solve the problem. The very helpful community helped me discover this.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
{ networking.macvlans."myNewNet-shim" = { mode = "bridge"; interface = "enp3s0"; }; networking.interfaces."myNewNet-shim".ipv4 = { addresses = [{ address = "192.168.1.67"; prefixLength = 32; }]; routes = [{ address = "192.168.1.64"; prefixLength = 30; }]; }; } |
If you dig into the implementation (createMacvlanDevice, configureAddrs), you can get some insight into how this maps onto basically the same thing my boot time script did.
This feels a lot less of a hack than using a script. Both work, but using the networking.macvlans approach is nice and clean. I should probably do the work to declare the docker macvlan inside my NixOS configuration to make this complete, but that’s a task for another day.