Configuring NGINX to log HTTP POST data on Linux / RHEL

NGINX is a powerful web server that can easily handle high volumes of HTTP traffic. Each time NGINX handles a connection, a log entry is generated to store some information this connection like remote IP address, response size and status code, etc. The complete set of logged information with more details can be found here.

In some cases, you may be more interested in storing the body of requests, specifically POST requests. Lucky, the NGINX ecosystem is rich, and includes quite a few handy modules. One such module is the Echo module, which provides useful functionality for things like: echo, time, and sleep commands.

In our use case, to log a request body, what we need is to use the echo_read_request_body command and the $request_body variable (contains the request body of the Echo module). However, this module is not distributed with NGINX by default, and to be able to use it we have to build NGINX from its source code by building it with the source code of the Echo module included. 

The following steps detail how to build NGINX so that the Echo module is included (here is the complete build bash file):

Download the source codes for NGINX and Echo using the following commands:

curl -L -O 'https://github.com/openresty/echo-nginx-module/archive/v0.58.tar.gz'
tar -xzvf v0.58.tar.gz && rm v0.58.tar.gz
mv echo-nginx-module-0.58 /tmp/echo-nginx-module
curl -O 'http://nginx.org/download/nginx-1.9.7.tar.gz'
tar -xzvf nginx-1.9.7.tar.gz && rm nginx-1.9.7.tar.gz

Create an nginx user, which we will use to run the NGINX process:

groupadd nginx
useradd -G nginx nginx

Install NGINX from source code:

cd nginx-1.9.7/ && ./configure \
 --user=nginx \
 --group=nginx \
 --prefix=/etc/nginx \
 --sbin-path=/usr/sbin/nginx \
 --conf-path=/etc/nginx/nginx.conf \
 --pid-path=/var/run/nginx.pid \
 --lock-path=/var/run/nginx.lock \
 --error-log-path=/var/log/nginx/error.log \
 --http-log-path=/var/log/nginx/access.log \
 --with-http_gzip_static_module \
 --with-http_stub_status_module \
 --with-http_ssl_module \
 --with-pcre \
 --with-file-aio \
 --with-http_realip_module \
 --add-module=/tmp/echo-nginx-module

make -j2
make install

Now NGINX is ready to start, but before doing so we need to modify the default NGINX configuration file under /etc/nginx/nginx.conf to enable logging of the HTTP request body.

NGINX writes quite a bit of information about incoming requests to the location indicated in the access_log directive. What is and is not logged in each log event can be customized in the log_format directive. Further, the Echo module stores the request body in the request_body variable by calling the echo_read_request_body method. We have to modify these directives as follows (here is the complete nginx.conf):

http {
 ...
 log_format custom '$request_body';
 access_log /var/log/nginx/access.log custom;
 ...
 server {
  location / {
   echo_read_request_body;
    ...
  } 
 }
}

If you are in a high-throughput environment, you may need to increase the allowed limit of linux open files, for user nginx in order to be able to handle thousands of requests per second. You can do so by using the following commands to append this setting to your configurations (you can also edit the files directly):

echo "fs.file-max = 1073741824" >> /etc/sysctl.conf
echo "nginx soft nofile 40960" >> /etc/security/limits.conf
echo "nginx hard nofile 81920" >> /etc/security/limits.conf

Now, we are ready to start NGINX and log incoming HTTP requests' body. Run NGINX:

nginx

I hope this helps give an understanding of how to control some of the NGINX logging functionality, and how to build, install, and utilize the Echo module. The complete script and details can be found in this gist.