Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

ELK Exploration Companion

May 5, 2017
ryan birmingham
Related topics:
Open source
Related products:
Red Hat OpenShift

Share:

    ELK

    ELK (or Elastic stack) is the name for the Elasticsearch/Logstash/Kibana stack. Logstash gets log information, reports it to Elasticsearch for searching, and Kibana lets you analyze it. While the tools work independently, and with other software, they play together especially well. To understand what’s going on, let’s look at each one individually. This guide is meant to be a bit of a guided tour to each of these services.

    Elasticsearch

    Elasticsearch is a real-time search and analytics engine. It’s distributed and built on Apache’s Lucene. Curl it requests to default port 9200 like:

    (curl -X<VERB> '<PROTOCOL>://<HOST>:<PORT>/<PATH>?<QUERY_STRING>' -d '<BODY>');

    it will respond with a json; alternatively, you can use sense (a Kibana app) to query directly from the browser. (More on Kibana later, just keep this in the back of your head)

    Insert data with PUT /{index name}/{type}/{id}; get with GET /{index name}/{type}/{id}.

    Use GET /{index name}/{type}/_search to get all, append with ?q={field}:{value} to search more Domain-specific language (DSL); can use a get body to /{index name}/{type}/_search.

    DSL allows for queries that are more specific and makes them easier to read. Look at some examples! Searches will often provide relevance scores. You can use constant_score to avoid scoring and exact matches.

    Filters let you narrow down your searches; they can (and should) be nested:

    • Bool has four fields, must, should, must_not, and filter; filter runs the query in non-scoring mode.
      • Many filters use bool to deal with multi-part/multi-word queries.
    • Terms can take an array instead of a value, which searches for any that match; use tags and tag_count within to specify more specific matches.
    • Use range with GT, LT, gte, and/or LTE to search on ranges.
      • For dates, can use things like now-1m, {timestamp}||+1M.
        • Be aware of date format marks though!
      • Strings search in lexicographic order.
    • Dealing with null.
      • Exists deals with null values (if a value is not stored in post, it’s null).
      • Missing does the opposite of exists.
      • For both of the above, note that you may need to choose a null_value to store an actual/intentional null value.
      • Exists/missing will reduce to bool -> should on subfields if applicable.
    • Full-text has an analysis phase, unlike term-based.
      • Match is the general “find this thing” query.
      • Multi-word match queries are bool -> should; can use operator: and to change to must.
      • Can boost: # to add a weight (default 1).

    Aggregations are an important part of Elasticsearch. Buckets are documents that meet a criterion; you set those using aggs in a query. Metrics are statistics on a bucket’s documents, like SQL COUNT, AVG, SUM, etc. You can nest both of these.

    You can monitor the Elasticsearch status in kibana using the x-pack plugin. Alternatively, you can use curl directly:

        • GET _cluster/health returns status, nodes, shards, etc
        • GET _nodes/stats to check individual nodes, their indices
        • GET _cluster/stats to get similar info to node stats but aggregated for the whole cluster
        • GET {index}/_stats to get stats about that index, or GET _all/_stats for all
        • GET _cluster/pending_tasks to see if any tasks are pending
        • GET /_cat to see a list of endpoints which return more linux-like tabular info, instead of json

    Logstash

    Logstash is a log formatter/aggregator and operates on a pipeline. Make a pipeline configuration file to describe what you want to happen to the logs. A pipeline includes inputs, filters, and outputs (and codecs). To use a pipeline, run logstash like “{path}bin/logstash -f {pipeline file} --config.reload.automatic”, which allows auto config changes without stopping.

    Inputs are ways that data enters the pipeline. Some common ones:

    • Beat: Filebeat collects logs from server files
    • File, like unix tail -0F
    • Syslog listens on port 514
    • Redis for Redis server

    Filters are actions taken on the logs during processing.  Some common ones:

    • Grok to parse and structure arbitrary text (built in)
    • Mutate for specific changes (rename, remove, replace, modify, etc)
    • Drop / clone
    • GeoIP to add go ip info

    Outputs are destinations from the pipeline. Some common ones:

    • Elasticsearch (you want this if you’re doing ELK)
    • File
    • Graphite - another open source tool
    • Statsd, which listens for statistics over udp

    Codecs allow you to change the output format (like json, or multiline).

    For all parts of the pipeline, you can add more plugins of any of these types to match your needs.

    Monitoring

    • Can use ?human={true|false} or ?pretty={true|false} to tune for debugging
    • GET commands
      • / for general
      • /_node_/pipeline for pipeline info
      • /_node/os for node os info
      • /_node/jvm for jvm info
      • /_node/plugins for any plugins
      • /_node/stats/{jvm|process|mem|pipeline} to get some useful statistics
      • /_node/hot_threads to see what threads are running with high cpu for long

    Kibana

    Honestly, the best way to get to know Kibana is to try it. Elastic (who makes all ELK components) hosted a demo here: demo.elastic.co/.

    Alternatively, set up your own: Install from a package (deb or rpm) and run sudo -i service kibana start to start it. It listens on localhost:5601 by default.

    Double alternatively, you can get a docker image with kibana and x-pack already there, so it can easily connect to elasticsearch.

    • Quick guide to Kibana
      • Discover lets you see what fields are available, and do quick searches.
      • Visualize lets you make charts out of saved searches. Be sure to click the “play” button to update the visualization with changes.
      • A dashboard can be created out of visualizations to get a sense of what’s going on. I highly recommend making at least something similar to a “server/app status” one.
      • Management just lets you manage saved searches/dashboards, kibana/search configuration, and reporting to another source.
      • Dev tools help you develop for or with the ELK stack.

    X-Pack

    X-pack is not part of the ELK stack, but it’s useful in using it. It adds a sign on and additional security, alerting conditions, monitoring for the components of ELK, graph visualization to Kibana, and pdf reports.


    Developers can now get a no-cost Red Hat Enterprise Linux® Developer Suite subscription for development purposes by registering and downloading through developers.redhat.com.

    Last updated: February 24, 2024

    Recent Posts

    • Meet the Red Hat Node.js team at PowerUP 2025

    • How to use pipelines for AI/ML automation at the edge

    • What's new in network observability 1.8

    • LLM Compressor: Optimize LLMs for low-latency deployments

    • How to set up NVIDIA NIM on Red Hat OpenShift AI

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue