This article is about debugging out-of-memory issues with Open vSwitch with the Data Plane Development Kit (OvS-DPDK). It explains the situations in which you can run out of memory when using OvS-DPDK and it shows the log entries that are produced in those circumstances. It also shows some other log entries and commands for further debugging.
When you finish reading this article, you will be able to identify that you have an out-of-memory issue and you’ll know how to fix it. Spoiler: Usually having some more memory on the relevant NUMA node works. It is based on OvS 2.9.
Continue reading “Debugging Memory Issues with Open vSwitch DPDK”
In a few weeks, the Fast Datapath Production channel will update the Open vSwitch version from the 2.7 series to the 2.9 series. This is an important change in more ways than one. A wealth of new features and fixes all related to packet movement will come into play. One that will surely be blamed for all your troubles will be the integration of the `–ovs-user` flag to allow for an unprivileged user to interact with Open vSwitch.
Running as root can solve a lot of pesky problems. Want to write to an arbitrary file? No problem. Want to load kernel modules? Go for it! Want to sniff packets on the wire? Have a packet dump. All of these are great when the person commanding the computer is the rightful owner. But the moment the person in front of the keyboard isn’t the rightful owner, problems occur.
Continue reading “Non-root Open vSwitch in RHEL”
In order to maximize performance of the Open vSwitch DPDK datapath, it pre-allocates hugepage memory. As a user you are responsible for telling Open vSwitch how much hugepage memory to pre-allocate. The question of exactly what value to use often arises. The answer is, it depends.
There is no simple answer as it depends on things like the MTU size of the ports, the MTU differences between ports, and whether those ports are on the same NUMA node. Just to complicate things a bit more, there are multiple overheads, and alignment and rounding need to be accounted for at various places in OVS-DPDK. Everything clear? OK, you can stop reading then!
However, if not, read on.
Continue reading “Open vSwitch-DPDK: How Much Hugepage Memory?”
A few months ago, on this blog, we talked about MACsec. In this post, I want to introduce the work we’ve done since then. Since that work revolves around methods to configure MACsec, this will also act as a guide to configure it by two methods: wpa_supplicant alone, or NetworkManager with wpa_supplicant.
If you read the previous MACsec post, you probably thought that this whole business of generating keys and creating “secure associations” isn’t very convenient, especially given that you then have to monitor your associations and generate new keys manually. And you’re right: it’s not.
Besides, if you run RHEL or Fedora, you’re probably used to configuring your network with NetworkManager, so you would expect to be able to configure MACsec with NetworkManager as well. We’re going to describe this below. First, let’s go a little bit behind the scenes.
Continue reading “What’s new in MACsec: setting up MACsec using wpa_supplicant and (optionally) NetworkManager”
In Network Function Virtualization, there is a need to scale functions (VNFs) and infrastructure (NFVi) across multiple NUMA nodes in order to maximize resource usage.
In this blog, we’ll show how to configure Open vSwitch using DPDK datapath (OVS-DPDK) parameters for multiple NUMA systems, based on OVS 2.6/2.7 using DPDK 16.11 LTS.
Continue reading “OVS-DPDK Parameters: Dealing with multi-NUMA”
Networking hardware is becoming crazily fast, 10Gbs NICs are entry-level for server h/w, 100Gbs cards are increasingly popular and 200Gbs are already surfacing. While the Linux kernel is striving to cope with such speeds with large packets and all kind of aggregation, ISPs are requesting much more demanding workload with NFV and line rate packet processing even for 64 bytes packets.
Is everything lost and are we all doomed to rely on some kernel bypass solution? Possibly, but let’s first inspect what is really the current status for packet processing in the kernel data path, with a perspective look at the relevant history and the recent improvements.
We will focus on UDP packets reception: UDP flood is a common tool to stress the networking stack allowing arbitrary small packets and defeating packet aggregation (GRO), in place for other protocols.
Continue reading “The need for speed and the kernel datapath – recent improvements in UDP packets processing”
Open vSwitch (OVS) recently gained support for 802.1ad (QinQ). It can be used as a lightweight alternative to tunnel technologies such as; VXLAN, GENEVE, GRE. A key advantage of QinQ is that it can make use of hardware offload features common in network interface cards (NICs). Only newer NICs support hardware offload for VXLAN and GENEVE. QinQ also incurs less frame processing and has a smaller encapsulation overhead.
Continue reading “Open vSwitch: Overview of 802.1ad (QinQ) Support”
MACsec is an IEEE standard for security in wired ethernet LANs. This blog , will give an overview of what MACsec is, how it differs from other security standards, and present some ideas about how it can be used.
- MACsec is a Layer 2 protocol that relies on GCM-AES-128 to offer integrity and confidentiality, and operates over ethernet.
- It can secure all traffic within a LAN, including DHCP and ARP, as well as traffic from higher layer protocols.
- It is an extension to 802.1X provides secure key exchange and mutual authentication for MACsec nodes.
- IPsec (a Layer 3 security protocol) and TLS (a Layer 4 security protocol) offer different guarantees and can be a better fit, depending on the use case.
Continue reading “MACsec: a different solution to encrypt network traffic”
by Jakub Hrozek and Andreas Schneider
Software testing is already a hard business. It gets even harder if you need to test software that is networked, requires custom users on the system or resolve DNS queries.
Consider software such as a file server — it needs to listen for incoming connections on a certain port, often a privileged one in case of well-known protocols. The file server also requires the ability to switch to different user accounts and act on their behalf to create files owned by these users. Finally, a client of this hypothetical file server might want predefined SRV records to be present in DNS for autodiscovery to work properly. All these cases should be tested on every build.
And it gets even harder if your unit tests can’t run as the root user to set up the environment. The use case of testing the full stack, including network, users or DNS with only regular user privileges is exactly what the cwrap.org project is aiming to solve.
Continue reading “Testing your software stack without root privileges using cwrap”