Monitor OVN networking events using Network Observability

Network security and performance maintenance crucially depend on monitoring network events triggered by Open Virtual Network (OVN), including network policies, admin network policies, and egress firewalls. You can achieve this using Network Observability eBPF agent, which runs in the Linux kernel and can trace various network activities with minimal performance overhead, allowing you to observe and capture detailed information about network traffic and events in real-time.

Key components

The following are key components for Network Observability with eBPF:

1. eBPF network events monitoring kprobe eBPF hook: Network events monitoring using eBPF kernel probes (kprobes) provides deep, kernel-level insights into network stack behavior. The NetObserv eBPF agent leverages the entry point to efficiently capture packet metadata and identify policy violations with minimal overhead. The hook implemented in the NetObserv eBPF agent can capture multiple events within the same network flow and generate a list of network events that applied to that flow, with a limit of up to four events per flow.
2. ovn-kubernetes observability library: The eBPF agent captures network events as an array of bytes, which is not very user-friendly. This library provides functionality to convert these events into human-readable strings, making them easily understandable for customers.

Use cases for Network Observability with eBPF

Below are specific use cases for Network Observability with eBPF.

Monitor network policies

When you apply OVN network policies (like Kubernetes NetworkPolicy), eBPF agent can monitor allowed and/or blocked traffic, detecting whether packets are allowed or blocked based on network policies.

An example of these network policies is as follows:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: iperf3-server-access-ingress
  namespace: iperf
spec:
  podSelector:
    matchLabels:
      app: iperf3-server
  ingress:
  - ports:
    - port: 5501
    from:
    - podSelector:
        matchLabels:
          app: iperf3-client
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: iperf3-server-access-egress
  namespace: iperf
spec:
  podSelector:
    matchLabels:
      app: iperf3-client
  egress:
  - ports:
    - port: 5201
      protocol: TCP
  podSelector: {}
  policyTypes:
  - Egress

Figure 1 depicts the traffic flows in the NetObserv console plug-in. A network events column shows traffic allowed or denied by network policy rules.

Monitor AdminNetworkPolicies

An AdminNetworkPolicy is a high-level security policy designed to provide cluster administrators with more control over network traffic in Kubernetes environments.
eBPF agent can monitor allowed/blocked traffic, detecting whether packets are allowed or blocked based on admin network policies.

An example of AdminNetworkPolicy and egress NetworkPolicy to show multiple events use case:

apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
  name: allow-egress-iperf
spec:
  egress:
  - action: Pass
    to:
    - namespaces: {}
  priority: 10
  subject:
    namespaces:
      matchLabels:
        kubernetes.io/metadata.name: iperf
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: iperf3-server-access-egress
  namespace: iperf
spec:
  podSelector:
    matchLabels:
      app: iperf3-client
  egress:
  - ports:
    - port: 5201
      protocol: TCP
  podSelector: {}
  policyTypes:
  - Egress

Figure 2 shows the traffic flows in the NetObserv console plug-in, with a network events column showing the delegation by the admin policy resulting in traffic allowed or denied.

Monitor egress firewall traffic

For egress firewall monitoring, the eBPF agent can trace outbound traffic and see whether it adheres to egress firewall rules, detecting unauthorized outbound connections and flag outbound traffic that violates egress firewall rules.

In the following example, there are three events, one for NetworkPolicy, another for AdminNetworkPolicy, and the third for EgressFirewall:

apiVersion: k8s.ovn.org/v1
kind: EgressFirewall
metadata:
  name: default
  namespace: iperf
spec:
  egress:
  - to:
      cidrSelector: 8.8.8.8/24
    type: Allow
  - to:
      cidrSelector: 0.0.0.0/0
    type: Deny
---
apiVersion: policy.networking.k8s.io/v1alpha1
kind: AdminNetworkPolicy
metadata:
  name: allow-egress-iperf
spec:
  egress:
  - action: Pass
    to:
    - namespaces: {}
  priority: 10
  subject:
    namespaces:
      matchLabels:
        kubernetes.io/metadata.name: iperf
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: iperf3-server-access-egress
  namespace: iperf
spec:
  podSelector:
    matchLabels:
      app: iperf3-client
  egress:
  - ports:
    - port: 5201
      protocol: TCP
  podSelector: {}
  policyTypes:
  - Egress

Figure 3 displays the traffic flows in the NetObserv console plug-in. A network events column shows traffic allowed or denied by egress firewall rules.

Integration with the Network Observability stack

The following are integrated into the Network Observability stack:

• eBPF agent: The eBPF agent manages a new kprobe hook through a configuration option. It also registers with the OVN observability library, allowing eBPF-generated events to be enriched into meaningful strings.
• Agent user space: The agent user space handles the aggregation of the network events.
• Flowlogs pipeline: Generated flows are transmitted over gRPC to the flowlogs-pipeline, where the flow's content is enriched with Kubernetes metadata.
• Loki storage and console plug-in: The enriched flows are then sent to Loki storage, making them available for consumption by the NetObserv console plug-in. The new events are displayed in the console.
• New filtering capability: A new filter is introduced to filter flows based on network events enrichment.

Figure 4 depicts the Network Observability stack.

Configuration

Enable the OVNObservability feature in the FeatureGate custom resource:

$ oc edit featuregates.config.openshift.io cluster
spec:
 featureSet: TechPreviewNoUpgrade
enabled:
- name: OVNObservability

Enable NetworkEvents in the Network Observability FlowCollector custom resource:

apiVersion: flows.netobserv.io/v1beta2
kind: FlowCollector
metadata:
  name: cluster
spec:
  agent:
    type: eBPF
    ebpf:
      sampling: 1
      privileged: true
      features:
      - "NetworkEvents"

Availability

The networking events monitoring feature is available in Developer Preview for Red Hat OpenShift Container Platform 4.17 z-stream and NetObserv release 1.7.

Conclusion

Relying on eBPF technology for network observability, particularly in environments governed by OVN network policies and egress firewalls, offers deep, real-time insights into network traffic. Users can monitor policy compliance, detect violations, and track detailed network flows. This helps you maintain a secure, high-performing network environment.

Feedback

We hope you liked this article! NetObserv is an open source project available on GitHub. Feel free to share your ideas ideas, use cases, or ask the community for help.