In part one of this series, we explored the dynamic IP address management (IPAM) capabilities of Open Virtual Network. We covered the subnet, ipv6_prefix, and exclude_ips options on logical switches. We then saw how these options get applied to logical switch ports whose addresses have been set to the special "dynamic" value. OVN, a subproject of Open vSwitch, is used for virtual networking in a number of Red Hat products like Red Hat OpenStack Platform, Red Hat Virtualization, and Red Hat OpenShift Container Platform in a future release.
In this part, we're going to explore some of the oversights and downsides in the feature, how those have been corrected, and what's in store for OVN in future versions.
Subnet changes
Let's start by creating a simple logical switch with a couple of logical switch ports that use dynamic addresses:
ovn-nbctl ls-add sw ovn-nbctl set Logical_Switch sw other_config:subnet=192.168.1.0/24 ovn-nbctl lsp-add sw sw-p1 ovn-nbctl lsp-set-addresses sw-p1 "dynamic" ovn-nbctl lsp-add sw sw-p2 ovn-nbctl lsp-set-addresses sw-p2 "dynamic"
This creates a logical switch sw with ports sw-p1 and sw-p2. Port sw-p1 is assigned address 192.168.1.2, and port sw-p2 is assigned 192.168.1.3.
But wait: we made a mistake! We actually meant to set the subnet of the switch to 192.168.0.0/24. Let's correct it.
ovn-nbctl set Logical_Switch sw other_config:subnet=192.168.0.0/24
All right, let's see how that's affected the logical switch ports' addresses. If you are running the Open vSwitch (OVS) 2.9 series or earlier, then you'll see the following:
$ ovn-nbctl --columns=name,dynamic_addresses,addresses list logical_switch_port name : "sw-p2" dynamic_addresses : "0a:00:00:00:00:02 192.168.1.3" addresses : [dynamic] name : "sw-p1" dynamic_addresses : "0a:00:00:00:00:01 192.168.1.2" addresses : [dynamic]
Huh? The dynamic addresses didn't update. Prior to OVS version 2.10, the dynamic addresses will not automatically update if the subnet, ipv6_prefix, or exclude_ips is updated. If you want the dynamic addresses to update, you need to clear the dynamic_addresses from the affected logical switch ports. The easiest way to clear the dynamic_addresses on all switch ports on switch sw is the following:
for port in $(ovn-nbctl --bare --columns=port find logical_switch name=sw) ; do ovn-nbctl clear logical_switch_port $port dynamic_addresses ; done
Now let's take another look at the logical switch ports:
$ ovn-nbctl --columns=name,dynamic_addresses,addresses list logical_switch_port name : "sw-p2" dynamic_addresses : "0a:00:00:00:00:03 192.168.0.2" addresses : [dynamic] name : "sw-p1" dynamic_addresses : "0a:00:00:00:00:04 192.168.0.3" addresses : [dynamic]
There; that's better. There are a couple of things to note here. First, the order in which IP addresses get assigned to the switch ports is not always predictable. The final octet of the IP addresses assigned to the switch ports was swapped from what it had previously been. Also, the MAC addresses have been updated on each switch port. When we cleared the dynamic_addresses, the MAC address assignments on the switch port got lost. As a result, ovn-northd assigns new MAC addresses to the ports. Unfortunately, if you are using dynamic MAC addresses, this is unavoidable.
The good news is that starting with OVS 2.10.0, this is no longer necessary. Updating subnet, ipv6_prefix, or exclude_ips on a logical switch will automatically update the dynamic_addresses on all logical switch ports. The even better news is that only the affected values are updated, so in this particular case, the MAC addresses on each switch port stay the same.
Conflicting addresses
Let's take our switch from the previous section and add a third switch port to it:
ovn-nbctl lsp-add sw sw-p3 ovn-nbctl lsp-set-addresses sw-p3 "00:00:00:00:00:03 192.168.0.3"
And let's have a look at our switch ports at this point:
name : "sw-p3" dynamic_addresses : [] addresses : ["00:00:00:00:00:03 192.168.0.3"] name : "sw-p2" dynamic_addresses : "0a:00:00:00:00:03 192.168.0.2" addresses : [dynamic] name : "sw-p1" dynamic_addresses : "0a:00:00:00:00:04 192.168.0.3" addresses : [dynamic]
Oops—our new switch port has an address that conflicts with one of our dynamic addresses. This will result in errors when packets are sent. There are a couple of ways to clear this up.
One way to fix this is by clearing the dynamic_addresses of sw-p2, and then sw-p2 will get a new dynamic address assigned to it. As mentioned in the previous section, this also means that sw-p2 will get assigned a new MAC address.
The other way is to use ovn-nbctl lsp-set-addresses on sw-p3 so that it has an address that doesn't conflict.
Starting with OVS version 2.10.0, this conflict can no longer occur. Instead, sw-p2 will automatically have its IP address updated to the next available address in the subnet. The code makes the assumption that statically assigned addresses are always correct and that dynamic addresses are "wrong" and need to be updated in the case of a conflict.
Starting with OVS version 2.11.0, it will be more difficult to cause this type of conflict. Watch what happens when we try the following with the current master of OVS:
$ ovn-nbctl lsp-set-addresses sw-p3 "00:00:00:00:00:03 192.168.0.3" ovn-nbctl: Error on switch sw: duplicate IPv4 address 192.168.0.3
The message above indicates that the conflict is detected by ovn-nbctl and the conflicting address is not set on sw-p3. It still is possible to set a conflicting address on sw-p3 by using the following command:
# Don't do this! $ ovn-nbctl set Logical_Switch_Port sw-p3 "00:00:00:00:00:03 192.168.0.3"
Doing this will still result in the conflicting address being set in the northbound database, and it will result in sw-p2 being assigned a new IP address.
Other fixed problems
In this final section, we'll examine some more minor things that are fixed in the 2.10 series of OVS. These are much less likely to happen than the issues explored in the previous two sections, and they're similar. Here's a brief summary:
- Prior to 2.10, if the MAC address on a switch port changes from being statically assigned to dynamically assigned, the MAC address would not be updated. In 2.10+, the MAC address is dynamically assigned.
- Prior to 2.10, if the IPv6 address is dynamically assigned and the MAC address on the port changes, then the IPv6 address is not updated. In 2.10+, when the MAC address is changed, the IPv6 address is recalculated too.
The future of IPAM in OVN
IPAM offers a handy way to have IP addresses and MAC addresses automatically get assigned to your logical switch ports. In part 1, we explored the basics of enabling IPAM in OVN, and in this part, we saw some downsides that have been fixed recently. But what is still to come? New developments are focused not so much on fixing issues as on adding features.
One improvement in the pipe is to allow for the pool of assignable MAC addresses to be configured. As we have seen in these posts, OVN will assign MAC addresses that start with "0a." But what about deployments where you want OVN to assign MAC addresses but you want to pick the range of MAC addresses to be assigned? This is currently being developed. One idea is to provide a start and end address, allowing OVN to assign addresses from that range. Another idea is to allow for an Organizational Unique Identifier (OUI) to be configured and assign OVN addresses using this OUI as a prefix.
Another improvement is to provide consistent pairings of IPv4 addresses and MAC addresses. Currently, OVN assigns MAC and IPv4 addresses independently of each other. However, it would be more friendly on ARP tables to try to assign the same IPv4 address with the same MAC address each time.
Both of the above ideas are currently in development, with a target of being available in the 2.11 series of OVS. I'm sure those of you reading these blog posts have ideas for further features that could be added. If you do, feel free to leave a comment on this post with your suggestion.
Last updated: February 17, 2022