Logging in Open vSwitch
Open vSwitch (OVS) is a virtual switch used in production from small to large scale deployments. It is designed to provide network automation along with support for a number of management interfaces and protocols.
However, the developer or the administrator might need to understand more of what is going on under the hoods and OVS does a good job providing a very good logging facility which is the subject of this article.
Editor’s note: Red Hat Knowledgebase – “What Red Hat products provide support for Open vSwitch?“
The OVS logging facility provides not only flexibility but granularity as well. Different from other software packages that you have a single logging output and a set of few message levels, OVS provides control over three possible outputs and message levels per module. It comes ready out-of-box and it can be controlled at runtime.
Controlling the outputs
One common scenario is to log to a file. The ovs-vswitchd(8) accepts the option –log-file[=file] to enable logging to a specific file. The file argument is actually optional, so if it is specified, it is used as the exact name for the log file. The default is used if file is not specified. Usually the default is /var/log/openvswitch/ovs-vswitch.log but it can change depending on compile time options. The correct information is available in the man-page.
Another common scenario is to log to syslog. The ovs-vswitchd(8) provides an option to specify a remote syslog server (–syslog-target=host:port) and an option to specify the method to be used (–syslog-method) which can be libc syslog(), unix:file or udp:ip:port.
The last method is the console output. It doesn’t require specific configuration parameters. However, logging to console will have no effect if –detach is used because ovs-vswitchd will close its standard file descriptors.
Controlling the logging levels
Usually increasing log level is a concern on production environments because it can cause undesired side effects. OVS has internal modules and the logging facility is capable of setting specific message levels for each of them. This granularity helps to reduce the overall impact and in turn enables its usage in production.
The list of modules can change for different reasons, so use the command below to see the list of available modules at runtime.
# ovs-appctl vlog/list console syslog file ------- ------ ------ backtrace OFF ERR INFO bfd OFF ERR INFO bond OFF ERR INFO bridge OFF ERR INFO bundle OFF ERR INFO bundles OFF ERR INFO cfm OFF ERR INFO collectors OFF ERR INFO command_line OFF ERR INFO connmgr OFF ERR INFO conntrack OFF ERR INFO ...
The list above is far from being complete because for example OVS 2.5 provides more than 90 modules. The screenshot above shows four columns. The module’s name used in the commands is in the first column, the second column is the minimum message level required for the message to be sent on console output. The third column is the same but for syslog output and the last column controls the file log output.
The logging levels can be off, emer, err, warn, info or dbg. Messages of the given severity or higher will be logged while others will be filtered out.
Those levels can be set using command line parameter –verbose=[spec] where the argument can set a level for everything, or to specific output, or even to specific module on specific output. Using the command line is specially useful to debug during the initialization, otherwise the levels can be easily modified at runtime.
The levels for each output can be modified using -v<output>:<level> syntax where the <output> is either console, syslog or file. For example, -vconsole:emer would set the minimum required log level to be emer.
The defaults usually are -vconsole:emer -vsyslog:err -vfile:info.
What about runtime? There is a tool called ovs-appctl(8) to manage logging. Actually the ‘vlog’ is the only subcommand documented in its man-page because other subcommands are registered during initialization of the modules. Anyway, this is probably a topic for another future blog post maybe.
Coming back to ovs-appctl, there is the vlog/list to list the modules and their levels for each output as shown already. And there is the vlog/set to manage the list. See below some interesting examples with outputs.
This is an example setting all modules on all output to dbg. # ovs-appctl vlog/set dbg # ovs-appctl vlog/list | head -n 10 console syslog file ------- ------ ------ backtrace DBG DBG DBG bfd DBG DBG DBG bond DBG DBG DBG bridge DBG DBG DBG bundle DBG DBG DBG bundles DBG DBG DBG cfm DBG DBG DBG collectors DBG DBG DBG This is an example of turning off all messages to console. # ovs-appctl vlog/set console:off # ovs-appctl vlog/list | head -n 10 console syslog file ------- ------ ------ backtrace OFF DBG DBG bfd OFF DBG DBG bond OFF DBG DBG bridge OFF DBG DBG bundle OFF DBG DBG bundles OFF DBG DBG cfm OFF DBG DBG collectors OFF DBG DBG This example sets cfm module logging level to emer only for the console output. # ovs-appctl vlog/set console:cfm:emer # ovs-appctl vlog/list | head -n 10 console syslog file ------- ------ ------ backtrace OFF DBG DBG bfd OFF DBG DBG bond OFF DBG DBG bridge OFF DBG DBG bundle OFF DBG DBG bundles OFF DBG DBG cfm EMER DBG DBG collectors OFF DBG DBG The tool accepts multiple arguments so the following example changes cfm log level for different outputs at the same time. # ovs-appctl vlog/set console:cfm:emer syslog:cfm:info file:cfm:off # ovs-appctl vlog/list | grep cfm cfm EMER INFO OFF
There is more. The tool ovs-appctl provides things like changing the log pattern, or reopen the file log, or change the syslog facility (RFC5424). The reader can find the details in the tool’s man-page.
OVS is more than a simple virtual switch. It is programmable and has many features built-in. The fact that OVS provides a great deal of flexibility and granularity managing logging helps to support it, specially in production environments.