Non-root Open vSwitch in RHEL

In a few weeks, the Fast Datapath Production channel will update the Open vSwitch version from the 2.7 series to the 2.9 series. This is an important change in more ways than one. A wealth of new features and fixes all related to packet movement will come into play. One that will surely be blamed for all your troubles will be the integration of the `--ovs-user` flag to allow for an unprivileged user to interact with Open vSwitch.

Running as root can solve a lot of pesky problems. Want to write to an arbitrary file? No problem. Want to load kernel modules? Go for it! Want to sniff packets on the wire? Have a packet dump. All of these are great when the person commanding the computer is the rightful owner. But the moment the person in front of the keyboard isn't the rightful owner, problems occur.

There's probably an astute reader who has put together some questions about why even bother with locking down the OvS binaries to non-root users. After all, the OvS switch uses netlink to tell the kernel to move ports, and voila! It happens! That won't be changing. But, that's expected.

On the other hand, it would be good to restrict Open vSwitch as much as possible. As an example, there's no need for Open vSwitch to have the kinds of privileges which allow writing new binaries to /bin. Additionally, Open vSwitch should never need access to write to Qemu disk files. These sorts of restrictions help to keep Open vSwitch confined to a smaller area of impact.

Since Open vSwitch version 2.5, the infrastructure has been available to run as a non-root user, but it always seemed a bit scary to turn it on. There were concerns about interaction with Qemu, libvirt, and DPDK. Even further, issues would really crop up with selinux. Lots of background work has been going on to address these, and after running this way for a while in Fedora, we think we've worked out the worst of the kinks.

So what do you need to do to ensure your Open vSwitch instance runs as a non-root user? Ideally nothing; a fresh install of the openvswitch rpm will automatically ensure that everything is configured properly to run as a non-root user. This is evident when checking with ps:

$ ps aux | grep ovs
 openvsw+ 15169 0.0 0.0 52968 2668 ? S<s 10:30 0:00 ovsdb-s
 openvsw+ 15214 200 0.3 5840636 229332 ? S<Lsl 10:30 809:16 ovs-vs

For new installs, this should be sufficient. Even DPDK devices will work when using a vfio-based PMD (most PMDs support vfio, so you really should use it).

Users who upgrade their Open vSwitch versions may find that the Open vSwitch instances run as root. This is intentional; we didn't want to break any existing setups. Yet all of the fancy infrastructure is there allowing you to switch if you so desire. Just a few simple steps to take:

1. Edit /etc/sysconfig/openvswitch and modify the OVS_USER_ID variable to openvswitch:hugetlbfs (or whatever user you desire)
2. Make sure that the directories (/etc/openvswitch, /var/log/openvswitch, and /dev/vfio) have the correct ownership/permissions. This includes files and sub-directories.
3. Start the daemon (systemctl start openvswitch).

If something goes wrong in this step, usually it will be evident in either journalctl (use journalctl -xe -u ovsdb-server for example), or in the log files.

Once the non-root changes are in effect, you could still encounter some permissions issues that aren't evident from journalctl. The most common one is when using libvirtd to start your VMs. In that case, the default libvirt configuration (either Group=root, or Group=qemu) may not grant the correct groupid to access vhost-user sockets. This can be configured by editing the Group= setting in the /etc/libvirt/qemu.conf configuration file to match with Open vSwitch's group (again, default is hugetlbfs).

I hope that was helpful!