My load balancing software vc5 has recently aquired support for IPv6 and layer 3 DSR backend servers. As well as bringing a 21st century internetworking protocol to the table this means that the restriction of having to keep all backend servers on VLANs which are directly attached to the load balancer has been removed.
To prevent the routing loop that would occur if the original ingress packets were fed back into the router, the incoming packets need to be encapsualated with a tunneling protocol. This adds an IP header (v4 or v6 depending on the backend server’s address family) and an encapsulation protocol header (unless a simple IP-in-IP scheme is being used).
Supported protocols are IP-in-IP (plus the 6in4, 4in6 and 6in6 variants), GRE, and FOU/GUE. Each has pros and cons, different levels of support by backend server operating systems features such as receive side scaling benefits.
Because packets need to be extended to support the extra headers, provision needs to be made for hitting up against the Maximum Transmission Unit (MTU) limits of the network segments. ICMP notifications may need to be sent to the client to advise it to send smaller packets, as well as dealing with similar notifications from intermediate systems as described on the excellent Cloudflare blog.
I think that I have catered for the various scenarios, but now it is time to soak test the new release and see where there might be deficiencies.
Using encapsulation for routing packets to backend servers makes vc5 (and the lower level xvs library that it uses) more similar to Facebook’s Katran, although unlike Katran it still has the ability to employ layer 2 DSR balancing as needed, which is still a requirement for my own use cases.
Due to limitations in the eBPF/XDP implementation in older kernels, development work is now carried out on Ubuntu 24.04 (kernel 6.14), and older releases probably won’t run the balancer. vc5 will compile and run on my Raspberry Pi running Debian Bookworm with the 6.12.34+rpt-rpi-v8 kernel, but the size of flow tracking tables needs to be set very low (eg. 100) as there is precious little memory available. Building the eBPF object file on the Pi wasn’t working for me, but as the binary is built for a virtual architecture, then the one that’s built in to each xvs release works out of the box.
Whilst testing driver-mode support in newer kernels for VMware’s vmxnet3 virtual network adapter, I came across — I think — CVE-2025-37799, which caused an awful lot of head scratching, and not a little panic, until I upgraded the kernel and the problem went away. Only afterwards did I find this CVE, which is what I suspect was the issue. If you’re having problems then it might be worth checking that you’re not affected.
An example configuration file follows which has an IPv4 only service (“foo”) which is backed by a set of remote servers (which could be at redundant data centres) accessible by IP-in-IP, the variant for each backend determined by the address family of the VIP and real server address.
There is also a service (“bar”) which has both IPv4 and IPv6 VIPs, each of which have one IPv4 and one IPv6 real server address. In this case the servers are all on the locally attached VLAN, so layer 2 DSR could be employed, but I’ve chosen to use GRE encapsualation for development/testing purposes. More documentation to follow (yeah, I know, I know).
I hope that this release is useful, and if you do happen to be using it and have any problems (or questions) please raise an issue on GitHub.
---
bgp:
as_number: 65001
next_hop_6: fd6e:eec8:76ac:ac1d:100::10
peers:
- 10.1.10.254
vlans:
1: 10.1.10.254/24
vlans6:
1: fd6e:eec8:76ac:ac1d:100::254/64
services:
- name: foo
virtual:
- 192.168.101.1
tunnel-type: ipip
servers:
- 10.10.10.100
- 10.20.20.100
- 10.30.30.100
- 10.40.40.100
host: foo.example.net
path: /loadbalancer.htm
policy:
http:
- name: bar
virtual:
- 192.168.101.2
- fd6e:eec8:76ac:1337::2
tunnel-type: gre
servers:
- 10.1.10.20
- fd6e:eec8:76ac:ac1d:100::20
host: bar.example.net
path: /alive
policy:
http: