Logging in Open vSwitch

Open vSwitch (OVS) is a virtual switch used in production from small to large scale deployments. It is designed to provide network automation along with support for a number of management interfaces and protocols.

However, the developer or the administrator might need to understand more of what is going on under the hoods and OVS does a good job providing a very good logging facility which is the subject of this article.

Editor’s note: Red Hat Knowledgebase – “What Red Hat products provide support for Open vSwitch?

Introduction

The OVS logging facility provides not only flexibility but granularity as well. Different from other software packages that you have a single logging output and a set of few message levels, OVS provides control over three possible outputs and message levels per module. It comes ready out-of-box and it can be controlled at runtime.

Controlling the outputs

One common scenario is to log to a file. The ovs-vswitchd(8) accepts the option –log-file[=file] to enable logging to a specific file. The file argument is actually optional, so if it is specified, it is used as the exact name for the log file. The default is used if file is not specified. Usually the default is /var/log/openvswitch/ovs-vswitch.log but it can change depending on compile time options. The correct information is available in the man-page.

Another common scenario is to log to syslog. The ovs-vswitchd(8) provides an option to specify a remote syslog server (–syslog-target=host:port) and an option to specify the method to be used (–syslog-method) which can be libc syslog(), unix:file or udp:ip:port.

The last method is the console output. It doesn’t require specific configuration parameters. However, logging to console will have no effect if –detach is used because ovs-vswitchd will close its standard file descriptors.

Controlling the logging levels

Usually increasing log level is a concern on production environments because it can cause undesired side effects. OVS has internal modules and the logging facility is capable of setting specific message levels for each of them. This granularity helps to reduce the overall impact and in turn enables its usage in production.

The list of modules can change for different reasons, so use the command below to see the list of available modules at runtime.

# ovs-appctl vlog/list
                 console    syslog    file
                 -------    ------    ------
backtrace          OFF        ERR       INFO
bfd                OFF        ERR       INFO
bond               OFF        ERR       INFO
bridge             OFF        ERR       INFO
bundle             OFF        ERR       INFO
bundles            OFF        ERR       INFO
cfm                OFF        ERR       INFO
collectors         OFF        ERR       INFO
command_line       OFF        ERR       INFO
connmgr            OFF        ERR       INFO
conntrack          OFF        ERR       INFO
...

The list above is far from being complete because for example OVS 2.5 provides more than 90 modules. The screenshot above shows four columns. The module’s name used in the commands is in the first column, the second column is the minimum message level required for the message to be sent on console output. The third column is the same but for syslog output and the last column controls the file log output.

The logging levels can be off, emer, err, warn, info or dbg. Messages of the given severity or higher will be logged while others will be filtered out.

Those levels can be set using command line parameter –verbose=[spec] where the argument can set a level for everything, or to specific output, or even to specific module on specific output. Using the command line is specially useful to debug during the initialization, otherwise the levels can be easily modified at runtime.

The levels for each output can be modified using -v<output>:<level> syntax where the <output> is either console, syslog or file. For example, -vconsole:emer would set the minimum required log level to be emer.

The defaults usually are -vconsole:emer -vsyslog:err -vfile:info.

What about runtime? There is a tool called ovs-appctl(8) to manage logging. Actually the ‘vlog’ is the only subcommand documented in its man-page because other subcommands are registered during initialization of the modules. Anyway, this is probably a topic for another future blog post maybe.

Coming back to ovs-appctl, there is the vlog/list to list the modules and their levels for each output as shown already. And there is the vlog/set to manage the list. See below some interesting examples with outputs.

This is an example setting all modules on all output to dbg.
# ovs-appctl vlog/set dbg
# ovs-appctl vlog/list | head -n 10
                 console    syslog    file
                 -------    ------    ------
backtrace          DBG        DBG        DBG
bfd                DBG        DBG        DBG
bond               DBG        DBG        DBG
bridge             DBG        DBG        DBG
bundle             DBG        DBG        DBG
bundles            DBG        DBG        DBG
cfm                DBG        DBG        DBG
collectors         DBG        DBG        DBG


This is an example of turning off all messages to console.
# ovs-appctl vlog/set console:off
# ovs-appctl vlog/list | head -n 10
                 console    syslog    file
                 -------    ------    ------
backtrace          OFF        DBG        DBG
bfd                OFF        DBG        DBG
bond               OFF        DBG        DBG
bridge             OFF        DBG        DBG
bundle             OFF        DBG        DBG
bundles            OFF        DBG        DBG
cfm                OFF        DBG        DBG
collectors         OFF        DBG        DBG


This example sets cfm module logging level to emer only for
the console output.
# ovs-appctl vlog/set console:cfm:emer
# ovs-appctl vlog/list | head -n 10
                 console    syslog    file
                 -------    ------    ------
backtrace          OFF        DBG        DBG
bfd                OFF        DBG        DBG
bond               OFF        DBG        DBG
bridge             OFF        DBG        DBG
bundle             OFF        DBG        DBG
bundles            OFF        DBG        DBG
cfm               EMER        DBG        DBG
collectors         OFF        DBG        DBG

The tool accepts multiple arguments so the following example
changes cfm log level for different outputs at the same time.
# ovs-appctl vlog/set console:cfm:emer syslog:cfm:info file:cfm:off
# ovs-appctl vlog/list | grep cfm
cfm               EMER       INFO        OFF

There is more. The tool ovs-appctl provides things like changing the log pattern, or reopen the file log, or change the syslog facility (RFC5424). The reader can find the details in the tool’s man-page.

Conclusion

OVS is more than a simple virtual switch. It is programmable and has many features built-in. The fact that OVS provides a great deal of flexibility and granularity managing logging helps to support it, specially in production environments.


Join the Red Hat Developer Program (it’s free) and get access to related cheat sheets, books, and product downloads.

 

Share

Leave a Reply