Automate and deploy a JBoss EAP cluster with Ansible

In my series introducing WildFly server configuration with Ansible collection for JCliff, I described how developers can use Ansible to manage a standalone Red Hat JBoss Enterprise Application Platform (JBoss EAP) instance. I've also written about using Ansible to automate Apache Tomcat and Red Hat JBoss Web Server deployments. In this article, we'll go a bit deeper and use Ansible to deploy a fully operational cluster of JBoss EAP instances. I'll show you how to automate the setup of each JBoss EAP instance and how to configure the network requirements—notably, fault tolerance and high availability—using features provided by the WildFly Ansible collection.

Note: This article assumes that you have prior knowledge of both Ansible and basic JBoss EAP/WildFly installation. Visit the Ansible courses page to learn the fundamentals of using Ansible.

Use case: Deploying a JBoss EAP cluster with Ansible

For this demonstration, we want to set up and run three JBoss EAP instances in a cluster. In this context, the application servers must communicate with each other to synchronize the content of the application's session. This configuration guarantees that, if one instance fails while processing a request, another one can pick up the work without any data loss.

We'll use a multicast to discover the members of the cluster and ensure that the cluster's formation is fully automated and dynamic.

Step 1: Install Ansible collection for WildFly

To follow this example, you need to install the Ansible collection that provides support for JBoss EAP. Named after JBoss EAP's upstream project, Ansible collection for WildFly is part of the middleware_automation collections and supplies a set of roles to simplify automation:

$ ansible-galaxy collection install middleware_automation.wildfly
Process install dependency map 
Starting collection install process 
Installing 'middleware_automation.wildfly:0.0.2' to '/root/.ansible/collections/ansible_collections/middleware_automation/wildfly' Installing 'middleware_automation.redhat_csp_download:1.2.1' to '/root/.ansible/collections/ansible_collections/middleware_automation/redhat_csp_download

Note the following:

The content of Ansible collection for WildFly comes from the Ansible collection for JCliff, which is used to help deploy applications and fine-tune server configurations. To keep things simple, we won't use JCliff's features in this example. The part of the WildFly collection that we are using has been separated from the JCliff collection so that developers can use the WildFly features without having to install JCliff. Features of JCliff are not required for all use cases involving JBoss EAP.

Additionally, note that the middleware_automation collections are provided through Ansible Galaxy and are not certified for Red Hat Ansible Automation Platform. Those certified collections are provided by Red Hat Ansible Automation Hub.

Step 2: Set up the JBoss EAP cluster

A typical JBoss EAP cluster has several machines, each operating a dedicated instance. In this case, for the simplicity of testing and reproducibility on a development system, we are going to use just one machine running several instances of JBoss EAP. The WildFly collection for Ansible makes it relatively easy to set up this architecture and provides all the required plumbing.

There are two parts to setting up the JBoss EAP cluster:

1. Install JBoss EAP on the hosts. This installation involves authenticating against the Red Hat Network (RHN), downloading the archive, and decompressing the archive in the appropriate directory (JBOSS_HOME). These tasks are handled by the wildfly_install role supplied by wildfly collection.
2. Create the configuration files to run several instances of JBoss EAP. Because we're running multiple instances on a single host, you also need to ensure that each instance has its own subdirectories and set of ports, so that the instances can coexist and communicate. Fortunately, this functionality is provided by a role within the Ansible collection called wildfly_systemd.

Note that if the variables rhn_username and rhn_password are defined, the collection automatically downloads the latest available version of JBoss EAP. If not, the wildfly_install role fetches the latest upstream WildFly version. To avoid adding the credentials in our playbook, we incorporate them into a separate file named rhn-creds.yml:

---
rhn_username:  
rhn_password:  
jboss_eap_rhn_id: 

Note: We could use Ansible's vault feature to safely encrypt the credential values, but doing that is out of the scope of this article.

Ansible playbook to install JBoss EAP

Here is our Ansible playbook for installing and configuring JBoss EAP:

---
- name: "JBoss EAP installation and configuration"
  hosts: "{{ hosts_group_name | default('localhost') }}"
  become: yes
  vars:
    wildfly_install_workdir: '/opt'
    wildfly_version: '7.4'
    install_name: jboss-eap
    wildfly_archive_filename: "{{ install_name }}-{{ wildfly_version }}.zip"
    wildfly_user: "{{ install_name }}"
    wildfly_config_base: standalone-ha.xml
    wildfly_home: "{{ wildfly_install_workdir }}/{{ install_name }}-{{ wildfly_version }}"
    jboss_eap_rhn_id: 99481

    instance_http_ports:
      - 8180
      - 8280
      - 8380
    app:
      name: 'info-1.1.war'
      url: 'https://drive.google.com/uc?export=download&id=1w9ss5okctnjUvRAxhPEPyC7DmbUwmbhb'
  collections:
    - middleware_automation.redhat_csp_download
    - middleware_automation.wildfly
  roles:
    - redhat_csp_download
    - wildfly_install
  tasks:

    - name: "Set up for WildFly instance {{ item }}"
      include_role:
        name: wildfly_systemd
      vars:
        wildfly_config_base: 'standalone-ha.xml'
        wildfly_basedir_prefix: "/opt/{{ inventory_hostname }}"
        wildfly_config_name: "{{ install_name }}"
        wildfly_port_range_offset: -1
        wildfly_instance_name: "{{ install_name }}"
        instance_id: "{{ item }}"
        service_systemd_env_file: "/etc/eap-{{ item }}.conf"
        service_systemd_conf_file: "/usr/lib/systemd/system/jboss-eap-{{ item }}.service"
      loop: "{{ range(0,3) | list }}"
  post_tasks:    

    - set_fact:
            instance_http_ports:
                - 8180
                - 8280
                - 8380
    - wait_for:
        port: "{{ item }}"
      loop: "{{ instance_http_ports }}"

    - name: "Checks that WildFly server is running and accessible"
      get_url:
        url: "http://localhost:{{ item }}/"
        dest: '/dev/null'
      loop: "{{ instance_http_ports }}" 

Run the playbook

Now, let's run our Ansible playbook and check the resulting output:

# ansible-playbook -e @creds.yml jboss_eap.yml

PLAY [WildFly installation and configuration] ******************************************************************

TASK [Gathering Facts] *****************************************************************************************
ok: [localhost]

TASK [middleware_automation.redhat_csp_download.redhat_csp_download : assert]
…
TASK [Checks that WildFly server is running and accessible] ****************************************************
ok: [localhost] => (item=8180)
ok: [localhost] => (item=8280)
ok: [localhost] => (item=8380)

PLAY RECAP *****************************************************************************************************
localhost                  : ok=90   changed=0    unreachable=0    failed=0    skipped=14   rescued=0    ignored=0   

Although the playbook is quite short, it performs almost 100 tasks. First, it automatically installs the dependencies for middleware_automation.redhat_csp_download by adding the associated role. Then, the wildfly_install role uses the provided credentials to connect to RHN and download the jboss-eap-7.4.zip file. Finally, once those steps have been completed successfully, the wildfly_systemd role sets up three distinct services, each with its own set of ports and directory layout to store instance-specific data.

Note that the JBoss EAP installation is not duplicated. All of the binaries live under the /opt/jboss-eap-74 directory. The separate directories simply store the runtime data for each instance.

On top of everything, we configured the instances to use the standalone-ha.xml configuration as the baseline, so they are already set up for clustering.

Step 3: Confirm the JBoss EAP instance and services are running

The playbook confirms that each instance can be reached through its own HTTP port. We can also verify that the services are running by using the systemctl command:

# systemctl status jboss-eap-*
● jboss-eap-0.service - JBoss EAP (standalone mode)
   Loaded: loaded (/usr/lib/systemd/system/jboss-eap-0.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-12-23 12:31:41 UTC; 1min 55s ago
 Main PID: 1138 (standalone.sh)
    Tasks: 70 (limit: 1638)
   Memory: 532.3M
   CGroup: /system.slice/jboss-eap-0.service
           ├─1138 /bin/sh /opt/jboss-eap-7.4/bin/standalone.sh -c jboss-eap-0.xml -b 0.0.0.0 -Djboss.server.con>
           └─1261 java -D[Standalone] -server -verbose:gc -Xloggc:/opt/localhost0/log/gc.log -XX:+PrintGCDetail>
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31:44,548 INFO  [org.jboss.as.patching] (MSC service thread)
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31:44,563 WARN  [org.jboss.as.domain.management.security] >
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31
…

Step 4: Deploy an application to the JBoss EAP cluster

At this point, the three JBoss EAP instances are configured for clustering. However, no applications were deployed, so the cluster is not active (there is nothing to keep synchronized between all the instances).

Let's modify our Ansible playbook to deploy a simple application to all three JBoss EAP instances. To achieve this, we'll leverage another role provided by the wildfly collection: jboss_eap. This role includes a set of tasks generally focused on features specific to JBoss EAP.

In our case, we will use the jboss_cli.yml task file, which encapsulates the running of JBoss command-line interface (CLI) queries:

…
    - name: "Ensures webapp {{ app.name }} has been retrieved from {{ app.url }}"
      get_url:
        url: "{{ app.url }}"
        dest: "{{ wildfly_install_workdir }}/{{ app.name }}"

    - name: "Deploy webapp"
      include_role:
        name: jboss_eap
        tasks_from: jboss_cli.yml
      vars:
        jboss_home: "{{ wildfly_home }}"
        query: "'deploy --force {{ wildfly_install_workdir }}/{{ app.name }}'"
        jboss_cli_controller_port: "{{ item }}"
      loop:
        - 10090
        - 10190
        - 10290
…

Now, we will once again execute our playbook so that the web application is deployed on all instances. Once the automation completes successfully, the deployment will trigger the formation of the cluster.

Step 5: Verify the JBoss EAP cluster and application deployment

You can verify the JBoss EAP cluster formation by looking at the log files of any of the three JBoss EAP instances:

…

021-12-23 15:02:08,252 INFO  [org.infinispan.CLUSTER] (thread-7,ejb,jboss-eap-0) ISPN000094: Received new cluster view for channel ejb: [jboss-eap-0] (3) [jboss-eap-0, jboss-eap-1, jboss-eap-2]
…

To be thorough, you can also check that the application is properly deployed and accessible. To validate the application's operation, we can simply add a separate Ansible playbook called validate.yml. We can then import the new playbook into our playbook.yml:

post_tasks: 
  - include_tasks: validate.yml 
    loop: "{{ instance_http_ports }}"

The validate.yml file contains the following:

---

- assert:
    that:
      - item is defined

- wait_for:
    port: "{{ item }}"

- name: "Checks that WildFly server is running and accessible on port {{ item }}"
  get_url:
    url: "http://localhost:{{ item }}/"
    dest: '/dev/null'
  changed_when: False

- include_tasks: info.yml

You might have noticed that we include another playbook, info.yml, which is here:

---
- assert:
    that:
      - item is defined
    quiet: true

- set_fact:
    result_file: "/tmp/info-{{ item }}.txt"

- get_url:
    url: "http://localhost:{{ item }}/info/"
    dest: "{{ result_file }}"
  changed_when: False

- slurp:
    src: "{{ result_file }}"
  register: info_res

- debug:
    msg: "{{ info_res['content'] | b64decode }}

To complete the exercise, we can run the validation playbook and see whether it confirms that our setup is fully functional:

TASK [get_url] ********************************************************************************
changed: [localhost]

TASK [slurp] **********************************************************************************
ok: [localhost]

TASK [debug] **********************************************************************************
ok: [localhost] => {
    "msg": "Request received<br/>Requested URL:\t\t\thttp://localhost:8380/info/<br/>Runs on node:\t\t\tda953ac17443 [IP: 10.0.2.100 ]<br/>Requested by:\t\t\t127.0.0.1 [IP: 127.0.0.1, port: 40334 ]<br/>JBOSS_ID:\t\t\tnull<br/>"
}

Next steps with Ansible and JBoss EAP

In this article, we've fully automated the setup and configuration of a three-instance cluster of JBoss EAP, along with an example web application as a workload. The playbook that we created for the automation is simple and straightforward. Most importantly, we were able to focus primarily on deploying the application. The WildFly collection for Ansible provided all the plumbing we needed to set up the JBoss EAP cluster.

You can find the source code for the example in the WildFly cluster demo GitHub repository. For a more complex scenario, see the Flange project demo, which adds to the JBoss EAP cluster an instance of Red Hat's single sign-on technology, using Red Hat Data Grid as a cache and Red Hat Middleware Core Services Collection as a load balancer.

See these resources to learn more about using Ansible with JBoss EAP:

• Automate Red Hat JBoss Web Server deployments with Ansible
• Set up mod_cluster for Red Hat JBoss Web Server with Ansible
• WildFly server configuration with Ansible collection for JCliff (three-part series)

For more hands-on learning, in short, interactive courses, see the Ansible courses page. Also, be sure to check out Red Hat Ansible Automation Platform for provisioning, deploying, and managing IT infrastructure across cloud, virtual, and physical environments.