This article is about debugging out-of-memory issues with Open vSwitch with the Data Plane Development Kit (OvS-DPDK). It explains the situations in which you can run out of memory when using OvS-DPDK and it shows the log entries that are produced in those circumstances. It also shows some other log entries and commands for further debugging.
When you finish reading this article, you will be able to identify that you have an out-of-memory issue and you’ll know how to fix it. Spoiler: Usually having some more memory on the relevant NUMA node works. It is based on OvS 2.9.
Continue reading “Debugging Memory Issues with Open vSwitch DPDK”
In a few weeks, the Fast Datapath Production channel will update the Open vSwitch version from the 2.7 series to the 2.9 series. This is an important change in more ways than one. A wealth of new features and fixes all related to packet movement will come into play. One that will surely be blamed for all your troubles will be the integration of the `–ovs-user` flag to allow for an unprivileged user to interact with Open vSwitch.
Running as root can solve a lot of pesky problems. Want to write to an arbitrary file? No problem. Want to load kernel modules? Go for it! Want to sniff packets on the wire? Have a packet dump. All of these are great when the person commanding the computer is the rightful owner. But the moment the person in front of the keyboard isn’t the rightful owner, problems occur.
Continue reading “Non-root Open vSwitch in RHEL”
In order to maximize performance of the Open vSwitch DPDK datapath, it pre-allocates hugepage memory. As a user you are responsible for telling Open vSwitch how much hugepage memory to pre-allocate. The question of exactly what value to use often arises. The answer is, it depends.
There is no simple answer as it depends on things like the MTU size of the ports, the MTU differences between ports, and whether those ports are on the same NUMA node. Just to complicate things a bit more, there are multiple overheads, and alignment and rounding need to be accounted for at various places in OVS-DPDK. Everything clear? OK, you can stop reading then!
However, if not, read on.
Continue reading “Open vSwitch-DPDK: How Much Hugepage Memory?”
Open vSwitch is growing every day and being used in large-scale deployments. Usually, that means there are few ports configured in the vswitch that will be always available, like physical Ethernet ports and several other ports providing networking connectivity to virtual machines or containers. Those other ports are software devices and very often they cannot be reused after a reboot or a system crash for example.
This blog post will talk about how to make sure the vSwitch comes up clean after a system crash or bad shutdown. The idea is that once vSwitch is up, there is no need for another component (usually a remote controller) to iterate over a large number of stale ports and clean them up.
Continue reading “Open vSwitch without stale ports”
This blog describes how a script can be used to automate Open vSwitch PVP testing. The goal for this PVP script was to have a quick (and dirty) way to verify the performance (change) of an Open vSwitch (DPDK) setup. This script either works with a Xena Networks traffic generator or the T-Rex Realistic Traffic Generator. For details on what the PVP test does, please refer to the following blog post, Measuring and comparing Open vSwitch performance.
Continue reading “Automated Open vSwitch PVP testing”
In a previous post, we introduced QinQ support for Open vSwitch. This post will investigate how QinQ performs relative to alternatives (VXLAN, GENEVE) in both throughput and CPU utilization. This will give us some understanding why we might consider QinQ over VXLAN or GENEVE.
Continue reading “Open vSwitch: QinQ Performance”
Typically, users will interact with the Open vSwitch kernel datapath by way of the ‘ovs-ofctl’ utility to program OpenFlow rules into the ‘ovs-vswitchd’. However, this isn’t the only mechanism for forwarding packets via the openvswitch kernel module. An additional direct flow-programming interface is available using the ‘ovs-dpctl’ utility to add flows to the kernel. This post will cover influencing the movement of packets through the openvswitch kernel module using the ‘ovs-dpctl’ utility.
Continue reading “Direct Kernel Open vSwitch Flow Programming”