Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Network testing with testpmd and noisy_vnf

September 21, 2023
Mike Pattrick
Related topics:
Linux
Related products:
Red Hat Enterprise Linux

Share:

    Developers of software-defined network (SDN) frameworks and applications often use the DPDK utility testpmd to test DPDK features and benchmark network hardware performance. In the upcoming DPDK release 23.07, we've extended the functionality of the noisy_vnf module in testpmd to allow better simulations of heavily loaded servers or complex workloads. This can allow for more realistic benchmarks.

    The default DPDK forwarding mode is io, which reads a batch of packets from an Rx queue and writes those packets directly to a Tx queue. Because the packets aren't processed or modified, the throughput achieved with this forwarding mode could be considered a theoretical maximum and not likely to be reproduced by real-world applications. Other forwarding modes like macswap and 5tswap process and modify layer 2 and 3, respectively; however, they do not perform any other per packet or batch processing. These forwarding modes are described in more detail in the DPDK testpmd documentation.

    The noisy_vnf module has existed in DPDK since 18.11, but its usefulness has been limited by the inability to combine its use with other forwarding modes like macswap or 5tswap. Instead, it has only forwarded packets unmodified like the io module does. This has hindered its inclusion in complex simulations. With DPDK 23.07, noisy_vnf users will be able to select various forwarding modes, including mac, macswap, and 5tswap.

    Network configuration

    To illustrate how this can be used, I've configured a small network with a simple application that sends and receives UDP packets; this application measures the round trip time (RTT) between send and receive by embedding a timestamp in the packet on send and comparing that timestamp to the current time receipt. The network path takes traffic out of a physical network interface into a second interface through Open vSwitch, and finally, to testpmd. See Figure 1.

    Network topology
    Figure 1: Network topology

    The use of Open vSwitch could allow for a variety of overlay networks to quickly be swapped in or out. For example, a VLAN or GENEVE VPN could easily be configured. But in this example, the default Normal action is left in place; this causes Open vSwitch to act like a traditional switch.

    The client application was configured to write a timestamp into the packet that it sent and then compare that with the current time when received to get an accurate measure of end-to-end RTT.

    Software configuration

    We ran three separate tests to show how noisy_vnf parameters, as described below, can affect transmission rates.

    • noisy-forward-mode: The noisy_vnf forwarding mode; currently only io, mac, macswap, and 5tswap are supported.
    • noisy-tx-sw-buffer-size: Allocates a FIFO packet queue in number of packets. Ingress packets fill this buffer until it is full or until a flush time has expired.
    • noisy-tx-sw-buffer-flushtime: Flush time, in milliseconds, for the FIFO packet buffer
    • noisy-lkup-memory: The amount of memory to allocate – in MB – for random read/write activity.
    • noisy-lkup-num-reads, noisy-lkup-num-writes, noisy-lkup-num-reads-writes: The number of reads and/or writes to conduct in the memory buffer allocated from noisy-lkup-memory per packet.

    The first test only used the 5tswap forwarding mode. Next, noisy_vnf was configured to perform 200 random reads and 20 random writes across 400 MB of memory. Finally, noisy_vnf was configured to perform 32 random reads and 8 random writes across a 200 MB buffer and to buffer 128 packets in a FIFO pipeline. The actual testpmd command-line invocations are included below.

    The first set of noisy_vnf parameters attempts to simulate an application that requires a lot of processing per packet, including accessing memory that may have fallen out of the CPU cache; for example, traversing a large data structure like nested hash tables. The second set of parameters attempts to simulate an application that has to queue up multiple packets before processing them all at once. The per-packet read and write rates are much lower but, in aggregate, add up to a larger amount of memory activity per batch of packets.

    # 5tswap run
    
    dpdk-testpmd --in-memory --single-file-segment --no-pci --vdev \
    'net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/vhost0,server=1,queues=4' \
    -- -i --rxq 4 --forward-mode=5tswap
    # noisy_vnf run 1
    
    dpdk-testpmd --in-memory --single-file-segment --no-pci --vdev \
    'net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/vhost0,server=1,queues=4' \
    -- -i --noisy-tx-sw-buffer-size=0 --noisy-tx-sw-buffer-flushtime=0 \
    --noisy-lkup-memory=400 --noisy-lkup-num-writes=20 --noisy-lkup-num-reads=200 \
    --noisy-forward-mode=5tswap --rxq 4 --forward-mode=noisy
    # noisy_vnf run 2
    
    dpdk-testpmd --in-memory --single-file-segment --no-pci --vdev \
    'net_virtio_user0,mac=00:01:02:03:04:05,path=/tmp/vhost0,server=1,queues=4' \
    -- -i --noisy-tx-sw-buffer-size=128 --noisy-tx-sw-buffer-flushtime=20 \
    --noisy-lkup-memory=200 --noisy-lkup-num-writes=8 --noisy-lkup-num-reads=32 \
    --noisy-forward-mode=5tswap --rxq 4 --forward-mode=noisy
    # OVS configurations
    
    34a5a4cc-69e2-46e5-abed-8a73e5b08dd7
        Bridge noisytest
            datapath_type: netdev
            Port dpdk0
                Interface dpdk0
                    type: dpdkvhostuserclient
                    options: {n_rxq="4", vhost-server-path="/tmp/vhost0"}
            Port noisytest
                Interface noisytest
                    type: internal
            Port dpdk1
                Interface dpdk1
                    type: dpdk
                    options: {dpdk-devargs="0000:03:00.1", n_rxq="4", n_rxq_desc="4096", n_txq_desc="4096"}
        ovs_version: "3.0.90"

    The client application used for this test is available on GitHub.

    Results

    The table below shows performance results in packets per second and microsecond round trip time while only changing the testpmd invocation.

    5tswap test

    Packet source and destination addresses swapped.

    • Packets per second: 145,000
    • Round-trip time: 134

    noisy_vnf 1 test

    Source and destination swapped; 220 memory operations over 400 MB of memory per batch.

    • Packets per second: 40,000
    • Round-trip time: 235

    noisy_vnf 2 test

    Source and destination swapped; 40 memory operations over 200 MB of memory per batch; and 128 packets queued in a FIFO buffer.

    • Packets per second: 992
    • Round-trip time: 20,225

    Conclusion

    These results demonstrate how substantially a packet buffer or a lot of random memory access can impact packet throughput. Merely inserting a few hundred memory operations with each packet saw a near doubling of round trip time. Using a FIFO packet buffer had a 40x reduction in packets per second throughput. These results make it clear that a network speed test that doesn't involve any per packet processing will not give a complete picture of the performance capabilities of the network setup.

    Related Posts

    • Using virtual functions with DPDK on Red Hat OpenShift

    • Improve multicore scaling in Open vSwitch DPDK

    • Tracing hardware offload in Open vSwitch

    • Investigating the cost of Open vSwitch upcalls in Linux

    • Why you should use io_uring for network I/O

    Recent Posts

    • Create and enrich ServiceNow ITSM tickets with Ansible Automation Platform

    • Expand Model-as-a-Service for secure enterprise AI

    • OpenShift LACP bonding performance expectations

    • Build container images in CI/CD with Tekton and Buildpacks

    • How to deploy OpenShift AI & Service Mesh 3 on one cluster

    What’s up next?

    Network automation with Ansible validated content - Share Image

    The Network Border Gateway Protocol (BGP) validated content collection focuses on platform-agnostic network automation and supports the BGP management experience by providing production-ready content. This cheat sheet covers basic commands and tasks for using Network BGP-validated content so you can automate faster and more efficiently.

    Get the cheat sheet
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue