A fully automated setup of a JBoss EAP cluster using Ansible

The objective of this demonstration is to set up and run three JBoss EAP instances forming a cluster. In this context, the application servers must communicate with each other to synchronize the content of the applications' session. This configuration guarantees that, if one instance fails while processing a request, another one can pick up the work without any data loss. In short, an EAP cluster is strategy to ensure high availability of the apps executed by the JEE server.

Use case: Deploying a JBoss EAP cluster with Ansible

Note that, for simplicity’s sake, we’ll use multicast discovery to identify members of the cluster and assure that the cluster’s formation is fully automated and dynamic. This will allow the cluster to configure itself without additional configuration. It’s worth mentioning that this setup can also work without multicast, but it is not the intended focus of this article.

Prerequisites

The first step is to install Ansible on the workstation, which will be referred to from now on as the Ansible control node along with ensuring that the tool can connect to the target systems. The process for setting up Ansible is out of scope for this article, but do refer to the Ansible documentation to perform this installation (keeping in mind that Ansible version 2.14 or above is recommended).

The second prerequisite is to provide the appropriate token to Ansible so that it can interact with Ansible automation hub to download the redhat.eap collection. Refer to the Red Hat Ansible documentation for this step.

Once the content of the ansible.cfg has been properly configured to use Ansible automation hub, the last step consists of installing the Red Hat Ansible Certified Content Collection for JBoss EAP:

# ansible-galaxy collection install redhat.eap
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://console.redhat.com/api/automation-hub/v3/plugin/ansible/content/published/collections/artifacts/redhat-eap-1.4.3.tar.gz to /root/.ansible/tmp/ansible-local-37nv77u54f/tmp3t3je__p/redhat-eap-1.4.3-wo61xl4t
Installing 'redhat.eap:1.4.3' to '/root/.ansible/collections/ansible_collections/redhat/eap'
redhat.eap:1.4.3 was installed successfully
'ansible.posix:1.5.4' is already installed, skipping.
'redhat.runtimes_common:1.1.3' is already installed, skipping.

For the collection to be able to download the JBoss EAP archives from the Red Hat Customer Portal, we need to supply the credentials tied a Red HAT service account. One way to provide those values parameters is to create a service_account.yml which can be passed to Ansible as an extra source of variables:

---
rhn_username: <service_account_id>
rhn_password: <service_account_password>

Note: We could use Ansible's vault to safely encrypt the credential values, but doing that is out of the scope of this article.

Set up the JBoss EAP cluster

A typical JBoss EAP cluster has many machines, each operating a dedicated instance. For the simplicity of testing and ensuring reproducibility on a development system, we are going to use just one server to run several instances of JBoss EAP. Using the Red Hat Ansible Certified Content Collection for JBoss EAP, constructing this type of architecture is relatively easy, as the collection provides all the required plumbing.

The process we are going to automate thanks to Ansible (and the redhat.eap collection) consists of two parts:

1. Install JBoss EAP on the hosts. This installation involves authenticating against the Red Hat Customer portal and downloading the EAP 7.4 archive prior to decompressing it in the appropriate directory (JBOSS_HOME). All of this is facilitated by the eap_install role within the redhat.eap collection.
2. Create the configuration files required to run several instances of JBoss EAP on the same host. Because we're executing multiple instances on a single machine, we need to ensure that each instance has its own subdirectories and set of ports so that they can coexist and communicate with each other. Fortunately, this functionality is also provided by a role within the Ansible collection called eap_systemd.

Ansible playbook to install JBoss EAP

Here is our Ansible playbook for installing and configuring JBoss EAP:

---
- name: "JBoss EAP installation and configuration"
  hosts: "{{ hosts_group_name | default('localhost') }}"
  become: yes
  vars:
    eap_install_workdir: '/opt'
    eap_version: '7.4.0'
    install_name: jboss-eap
    eap_user: "{{ install_name }}"
    eap_config_base: standalone-ha.xml
    eap_home: "{{ eap_install_workdir }}/{{ install_name }}-{{ (eap_version.split('.'))[0:2] | join('.') }}"
    instance_http_ports:
      - 8080
      - 8180
      - 8280
    app:
      name: 'info-1.1.war'
      url: 'https://drive.google.com/uc?export=download&id=1w9ss5okctnjUvRAxhPEPyC7DmbUwmbhb'
  collections:
    - redhat.eap
  roles:
    - eap_install
  tasks:
    - name: "Set up for WildFly instance {{ item }}"
      include_role:
        name: eap_systemd
      vars:
        eap_config_base: 'standalone-ha.xml'
        eap_basedir_prefix: "/opt/{{ inventory_hostname }}"
        eap_config_name: "{{ install_name }}"
        eap_instance_name: "{{ install_name }}"
        eap_instance_id: "{{ item }}"
        service_systemd_env_file: "/etc/eap-{{ item }}.conf"
        service_systemd_conf_file: "/usr/lib/systemd/system/jboss-eap-{{ item }}.service"
      loop: "{{ range(0,3) | list }}"
  post_tasks:
    - set_fact:
            instance_http_ports:
                - 8080
                - 8180
                - 8280
    - wait_for:
        port: "{{ item }}"
      loop: "{{ instance_http_ports }}"
    - name: "Checks that WildFly server is running and accessible"
      get_url:
        url: "http://localhost:{{ item }}/"
        dest: '/dev/null'
      loop: "{{ instance_http_ports }}"

Run the playbook

Now, let’s run our Ansible playbook and check the resulting output:

$ ansible-playbook -i inventory -e @service_account.yml eap-cluster.yml
PLAY [JBoss EAP installation and configuration] ********************************
TASK [Gathering Facts] *********************************************************
ok: [localhost]
TASK [redhat.eap.eap_install : Validating arguments against arg spec 'main'] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure prerequirements are fullfilled.] *********
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_install/tasks/prereqs.yml for localhost
TASK [redhat.eap.eap_install : Validate credentials] ***************************
ok: [localhost]
TASK [redhat.eap.eap_install : Validate existing zipfiles jboss-eap-7.4.0.zip for offline installs] ***
skipping: [localhost]
TASK [redhat.eap.eap_install : Validate patch version for offline installs] ****
skipping: [localhost]
TASK [redhat.eap.eap_install : Validate existing additional zipfiles jboss-eap-7.4.0.zip for offline installs] ***
skipping: [localhost]
TASK [redhat.eap.eap_install : Check that required packages list has been provided.] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Prepare packages list] **************************
skipping: [localhost]
TASK [redhat.eap.eap_install : Add JDK package java-11-openjdk-headless to packages list] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Install required packages (5)] ******************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure required local user exists.] *************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_install/tasks/user.yml for localhost
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Set eap group] **********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure group jboss-eap exists.] *****************
changed: [localhost]
TASK [redhat.eap.eap_install : Ensure user jboss-eap exists.] ******************
changed: [localhost]
TASK [redhat.eap.eap_install : Ensure workdir /opt exists.] ********************
changed: [localhost]
TASK [redhat.eap.eap_install : Ensure archive_dir /opt exists.] ****************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure server is installed] *********************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_install/tasks/install.yml for localhost
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Check local download archive path] **************
ok: [localhost]
TASK [redhat.eap.eap_install : Set download paths] *****************************
ok: [localhost]
TASK [redhat.eap.eap_install : Check target archive: /opt/jboss-eap-7.4.0.zip] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Retrieve archive from website: https://github.com/eap/eap/releases/download] ***
skipping: [localhost]
TASK [redhat.eap.eap_install : Retrieve archive from RHN] **********************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_install/tasks/install/rhn.yml for localhost
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [Download JBoss EAP from CSP] *********************************************
TASK [redhat.eap.eap_utils : Check arguments] **********************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Retrieve product download using JBoss Network API] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Determine install zipfile jboss-eap-7.4.0.zip from search results.] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Verify that filtered products has been properly populated.] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Download Red Hat JBoss Enterprise Application Platform] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Install server using RPM] ***********************
skipping: [localhost]
TASK [redhat.eap.eap_install : Check downloaded archive] ***********************
ok: [localhost]
TASK [redhat.eap.eap_install : Copy archive to target nodes] *******************
changed: [localhost]
TASK [redhat.eap.eap_install : Check target archive: /opt/jboss-eap-7.4.0.zip] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Verify target archive state: /opt/jboss-eap-7.4.0.zip] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Read target directory information: /opt/jboss-eap-7.4] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Extract files from /opt/jboss-eap-7.4.0.zip into /opt.] ***
changed: [localhost]
TASK [redhat.eap.eap_install : Note: decompression was not executed] ***********
skipping: [localhost]
TASK [redhat.eap.eap_install : Read information on server home directory: /opt/jboss-eap-7.4] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Check state of server home directory: /opt/jboss-eap-7.4] ***
ok: [localhost]
TASK [redhat.eap.eap_install : Set instance name] ******************************
ok: [localhost]
TASK [redhat.eap.eap_install : Deploy custom configuration] ********************
skipping: [localhost]
TASK [redhat.eap.eap_install : Deploy configuration] ***************************
changed: [localhost]
TASK [Apply latest cumulative patch] *******************************************
TASK [redhat.eap.eap_utils : Check installation] *******************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Set patch directory] ******************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Set download patch archive path] ******************
skipping: [localhost]
TASK [redhat.eap.eap_utils : Check download patch archive path] ****************
skipping: [localhost]
TASK [redhat.eap.eap_utils : Check local download archive path] ****************
ok: [localhost]
TASK [redhat.eap.eap_utils : Check local downloaded archive: {{ patch_bundle }}] ***
skipping: [localhost]
TASK [redhat.eap.eap_utils : Retrieve product download using JBossNetwork API] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Determine patch versions list] ********************
ok: [localhost]
TASK [redhat.eap.eap_utils : Determine latest version] *************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Determine install zipfile from search results] ****
ok: [localhost]
TASK [redhat.eap.eap_utils : Determine selected patch from supplied version: {{ eap_patch_version }}] ***
skipping: [localhost]
TASK [redhat.eap.eap_utils : Check remote downloaded archive: /opt/jboss-eap-7.4.13-patch.zip] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Download Red Hat EAP patch] ***********************
ok: [localhost]
TASK [redhat.eap.eap_utils : Update patch archive path after download] *********
ok: [localhost]
TASK [redhat.eap.eap_utils : Check remote download patch archive path] *********
ok: [localhost]
TASK [redhat.eap.eap_utils : Copy patch archive to target nodes] ***************
changed: [localhost]
TASK [redhat.eap.eap_utils : Check patch state] ********************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Set checksum file path for patch] *****************
ok: [localhost]
TASK [redhat.eap.eap_utils : Check /opt/jboss-eap-7.4/.applied_patch_checksum_6427fed0a0fab256a058232c1aa02efcdb32919d.txt state] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Print when patch has been applied already] ********
skipping: [localhost]
TASK [redhat.eap.eap_utils : Check if management interface is reachable] *******
fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 1, "msg": "Timeout when waiting for localhost:9990"}
TASK [redhat.eap.eap_utils : Set instance name] ********************************
skipping: [localhost]
TASK [redhat.eap.eap_utils : Deploy configuration] *****************************
ok: [localhost]
TASK [redhat.eap.eap_utils : Start eap for patching] ***************************
changed: [localhost]
TASK [redhat.eap.eap_utils : Wait for management interface is reachable] *******
ok: [localhost]
TASK [redhat.eap.eap_utils : Set apply CP conflict default strategy to default (if not defined): --override-all] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Apply patch /opt/jboss-eap-7.4.13-patch.zip to server installed in /opt/jboss-eap-7.4] ***
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_utils/tasks/jboss_cli.yml for localhost
TASK [redhat.eap.eap_utils : Ensure required params for JBoss CLI have been provided] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Ensure server's management interface is reachable] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Execute CLI query 'patch apply --override-all /opt/jboss-eap-7.4.13-patch.zip'] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Display patching result] **************************
ok: [localhost] => {
    "msg": "Apply patch operation result: {\n    \"outcome\" : \"success\",\n    \"response-headers\" : {\n        \"operation-requires-restart\" : true,\n        \"process-state\" : \"restart-required\"\n    }\n}"
}
TASK [redhat.eap.eap_utils : Set checksum file] ********************************
changed: [localhost]
TASK [redhat.eap.eap_utils : Set latest patch file] ****************************
changed: [localhost]
TASK [redhat.eap.eap_utils : Restart server to ensure patch content is running] ***
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_utils/tasks/jboss_cli.yml for localhost
TASK [redhat.eap.eap_utils : Ensure required params for JBoss CLI have been provided] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Ensure server's management interface is reachable] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Execute CLI query 'shutdown --restart'] ***********
ok: [localhost]
TASK [redhat.eap.eap_utils : Wait for management interface is reachable] *******
ok: [localhost]
TASK [redhat.eap.eap_utils : Stop service if it was started for patching] ******
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_utils/tasks/jboss_cli.yml for localhost
TASK [redhat.eap.eap_utils : Ensure required params for JBoss CLI have been provided] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Ensure server's management interface is reachable] ***
ok: [localhost]
TASK [redhat.eap.eap_utils : Execute CLI query 'shutdown'] *********************
ok: [localhost]
TASK [redhat.eap.eap_utils : Display resulting output] *************************
skipping: [localhost]
TASK [redhat.eap.eap_install : Ensure required parameters for elytron adapter are provided.] ***
skipping: [localhost]
TASK [Install elytron adapter] *************************************************
skipping: [localhost]
TASK [redhat.eap.eap_install : Install server using Prospero] ******************
skipping: [localhost]
TASK [redhat.eap.eap_install : Check eap install directory state] **************
ok: [localhost]
TASK [redhat.eap.eap_install : Validate conditions] ****************************
ok: [localhost]
TASK [Ensure firewalld configuration allows server port (if enabled).] *********
skipping: [localhost]
TASK [Set up for WildFly instance {{ item }}] **********************************
TASK [redhat.eap.eap_systemd : Validating arguments against arg spec 'main'] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check current EAP patch installed] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments for yaml configuration] *********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in WildFly] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in EAP] ***
skipping: [localhost]
TASK [Ensure required local user and group exists.] ****************************
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Set eap group] **********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure group jboss-eap exists.] *****************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure user jboss-eap exists.] ******************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set destination directory for configuration] ****
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance destination directory for configuration] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set base directory for instance] ****************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set bind address] *******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Create basedir /opt/localhost0 for instance: jboss-eap-0] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Create deployment directories for instance: jboss-eap-0] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy custom configuration] ********************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy configuration] ***************************
changed: [localhost]
TASK [redhat.eap.eap_systemd : Include YAML configuration extension] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check YAML configuration is disabled] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd envfile destination] ****************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd unit file destination] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy service instance configuration: /etc/eap-0.conf] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy Systemd configuration for service: /usr/lib/systemd/system/jboss-eap-0.service] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Perform daemon-reload to ensure the changes are picked up] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Ensure service is started] **********************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_systemd/tasks/service.yml for localhost
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance jboss-eap-0 state to started] ******
changed: [localhost]
TASK [redhat.eap.eap_systemd : Validating arguments against arg spec 'main'] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check current EAP patch installed] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments for yaml configuration] *********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in WildFly] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in EAP] ***
skipping: [localhost]
TASK [Ensure required local user and group exists.] ****************************
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Set eap group] **********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure group jboss-eap exists.] *****************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure user jboss-eap exists.] ******************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set destination directory for configuration] ****
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance destination directory for configuration] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set base directory for instance] ****************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set bind address] *******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Create basedir /opt/localhost1 for instance: jboss-eap-1] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Create deployment directories for instance: jboss-eap-1] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy custom configuration] ********************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy configuration] ***************************
changed: [localhost]
TASK [redhat.eap.eap_systemd : Include YAML configuration extension] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check YAML configuration is disabled] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd envfile destination] ****************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd unit file destination] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy service instance configuration: /etc/eap-1.conf] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy Systemd configuration for service: /usr/lib/systemd/system/jboss-eap-1.service] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Perform daemon-reload to ensure the changes are picked up] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Ensure service is started] **********************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_systemd/tasks/service.yml for localhost
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance jboss-eap-1 state to started] ******
changed: [localhost]
TASK [redhat.eap.eap_systemd : Validating arguments against arg spec 'main'] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check current EAP patch installed] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments for yaml configuration] *********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in WildFly] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check if YAML configuration extension is supported in EAP] ***
skipping: [localhost]
TASK [Ensure required local user and group exists.] ****************************
TASK [redhat.eap.eap_install : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Set eap group] **********************************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure group jboss-eap exists.] *****************
ok: [localhost]
TASK [redhat.eap.eap_install : Ensure user jboss-eap exists.] ******************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set destination directory for configuration] ****
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance destination directory for configuration] ***
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set base directory for instance] ****************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance name] ******************************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set bind address] *******************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Create basedir /opt/localhost2 for instance: jboss-eap-2] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Create deployment directories for instance: jboss-eap-2] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy custom configuration] ********************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy configuration] ***************************
changed: [localhost]
TASK [redhat.eap.eap_systemd : Include YAML configuration extension] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Check YAML configuration is disabled] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd envfile destination] ****************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
ok: [localhost]
TASK [redhat.eap.eap_systemd : Determine JAVA_HOME for selected JVM] ***********
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Set systemd unit file destination] **************
skipping: [localhost]
TASK [redhat.eap.eap_systemd : Deploy service instance configuration: /etc/eap-2.conf] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Deploy Systemd configuration for service: /usr/lib/systemd/system/jboss-eap-2.service] ***
changed: [localhost]
TASK [redhat.eap.eap_systemd : Perform daemon-reload to ensure the changes are picked up] ***
ok: [localhost]
TASK [redhat.eap.eap_systemd : Ensure service is started] **********************
included: /root/.ansible/collections/ansible_collections/redhat/eap/roles/eap_systemd/tasks/service.yml for localhost
TASK [redhat.eap.eap_systemd : Check arguments] ********************************
ok: [localhost]
TASK [redhat.eap.eap_systemd : Set instance jboss-eap-2 state to started] ******
changed: [localhost]
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [wait_for] ****************************************************************
ok: [localhost] => (item=8080)
ok: [localhost] => (item=8180)
ok: [localhost] => (item=8280)
TASK [Checks that WildFly server is running and accessible] ********************
ok: [localhost] => (item=8080)
ok: [localhost] => (item=8180)
ok: [localhost] => (item=8280)
PLAY RECAP *********************************************************************
localhost                  : ok=145  changed=28   unreachable=0    failed=0    skipped=52   rescued=1    ignored=0

Although the playbook is quite short, it performs quite a lot of tasks (around one hundred and fifty tasks). First, the eap_install role uses the provided credentials to connect to Red Hat Customer Portal and download the jboss-eap-7.4.zip archive. Ansible then ensures that the target system has all the required prerequisite (user accounts and dedicated group, system dependencies, …). And, finally, the product files are installed in the appropriate folders by decompressing the archive.

After that, the eap_systemd role that will take care of setting up three distinct systemd services, one for each JBoss EAP instance running on the target system. A nice benefit of this approach is that the binary file of JBoss EAP are not duplicated. All the product files live in the /opt/jboss-eap-74 directory. The only files or directories replicated are the ones that are tied to one of the instances. It also means that, in order to update all the instances (to a newly released minor version of EAP), one only needs to update the files located in this folder.

On top of everything, the instances have already been configured to use the standalone-ha.xml configuration as the baseline, so they are, already, set up for clustering.

Verify that JBoss EAP instances and associated services are running

The playbook confirms that each instance can be reached through its own HTTP port. We can also verify that the services are running by using the systemctl command:

# systemctl status jboss-eap-*
● jboss-eap-0.service - JBoss EAP (standalone mode)
   Loaded: loaded (/usr/lib/systemd/system/jboss-eap-0.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-12-23 12:31:41 UTC; 1min 55s ago
 Main PID: 1138 (standalone.sh)
    Tasks: 70 (limit: 1638)
   Memory: 532.3M
   CGroup: /system.slice/jboss-eap-0.service
           ├─1138 /bin/sh /opt/jboss-eap-7.4/bin/standalone.sh -c jboss-eap-0.xml -b 0.0.0.0 -Djboss.server.con>
           └─1261 java -D[Standalone] -server -verbose:gc -Xloggc:/opt/localhost0/log/gc.log -XX:+PrintGCDetail>
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31:44,548 INFO  [org.jboss.as.patching] (MSC service thread)
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31:44,563 WARN  [org.jboss.as.domain.management.security] >
Dec 23 12:31:44 7b38800644ee standalone.sh[1138]: 12:31
…

Deploy an application to the JBoss EAP cluster

At this point, the three JBoss EAP instances are running and configured to form a cluster. As no application has deployed, however, the cluster is not yet active (there is no data that would require some sort of synchronization between the instances).

We are going to modify our Ansible playbook to deploy of an application in each of the three JBoss EAP instances. To do so, we’ll leverage another feature of the Red Hat Ansible Certified Content Collection for JBoss EAP (redhat.eap) that integrate the use of the JBoss command-line tool (jboss-cli):

…
    - name: "Ensures webapp {{ app.name }} has been retrieved from {{ app.url }}"
      get_url:
        url: "{{ app.url }}"
        dest: "{{ wildfly_install_workdir }}/{{ app.name }}"
    - name: "Deploy webapp"
      include_role:
        name: jboss_eap
        tasks_from: jboss_cli.yml
      vars:
        jboss_home: "{{ wildfly_home }}"
        query: "'deploy --force {{ wildfly_install_workdir }}/{{ app.name }}'"
        jboss_cli_controller_port: "{{ item }}"
      loop:
        - 10090
        - 10190
        - 10290
…

Let’s execute again the playbook so that the web application is deployed on all instances. Once the automation completes successfully, the deployment of this application will trigger the formation of the cluster.

Verify the JBoss EAP cluster and application deployment

You can verify the JBoss EAP cluster formation by looking at the log files of any of the three JBoss EAP instances:

$ grep -e 'Received new cluster view for channel ejb:' /opt/jboss-eap-7.4/standalone/log/server.log
…
2023-11-23 15:02:08,252 INFO  [org.infinispan.CLUSTER] (thread-7,ejb,jboss-eap-0) ISPN000094: Received new cluster view for channel ejb: [jboss-eap-0] (3) [jboss-eap-0, jboss-eap-1, jboss-eap-2]
…

To be thorough, you can also check that the application is properly deployed and accessible. To validate the application's operation, we can simply add a separate Ansible playbook called validate.yml. We can then use this new playbook into our playbook.yml:

post_tasks:
  - include_tasks: validate.yml
    loop: "{{ instance_http_ports }}"

The validate.yml file contains the following:

---
- assert:
    that:
      - item is defined
- wait_for:
    port: "{{ item }}"
- name: "Checks that WildFly server is running and accessible on port {{ item }}"
  get_url:
    url: "http://localhost:{{ item }}/"
    dest: '/dev/null'
  changed_when: False
- include_tasks: info.yml

This playbook includes another one, called info.yml:

---
- assert:
    that:
      - item is defined
    quiet: true
- set_fact:
    result_file: "/tmp/info-{{ item }}.txt"
- get_url:
    url: "http://localhost:{{ item }}/info/"
    dest: "{{ result_file }}"
  changed_when: False
- slurp:
    src: "{{ result_file }}"
  register: info_res
- debug:
    msg: "{{ info_res['content'] | b64decode }}

To complete the exercise, we can run the validation playbook and see whether it confirms that our setup is fully functional:

TASK [get_url] ********************************************************************************
changed: [localhost]
TASK [slurp] **********************************************************************************
ok: [localhost]
TASK [debug] **********************************************************************************
ok: [localhost] => {
    "msg": "Request received<br/>Requested URL:\t\t\thttp://localhost:8380/info/<br/>Runs on node:\t\t\tda953ac17443 [IP: 10.0.2.100 ]<br/>Requested by:\t\t\t127.0.0.1 [IP: 127.0.0.1, port: 40334 ]<br/>JBOSS_ID:\t\t\tnull<br/>"
}

Conclusion

We've illustrated how to fully automated the setup and configuration of a three-instance cluster of JBoss EAP, along with the deployment of an application. The playbook is simple and straightforward. Most importantly, we were able to focus primarily on deploying the application. The Red Hat Ansible Certified Content Collection for JBoss EAP included all the required plumbing needed and did the heavy lifting, with little input by the end user.