Cross site and cross applications with Red Hat OpenShift and Red Hat Data Grid

One of the biggest problems in distributed applications is data replication. This task becomes even more complex when different technologies are involved, and even more so when we are dealing with applications distributed in different clusters. In this learning path, we will demonstrate how an application developed in three different technologies (Quarkus, .NET Core 7, and Golang) share data with each other through the Red Hat Data Grid and are distributed in two different Red Hat OpenShift clusters, thus demonstrating the power of the functionality cross-site replication offered by Data Grid.

The environment

We will work with two independent OpenShift clusters, site-1 and site-2. In each OpenShift cluster, we will use the facilities provided by the Operators to deploy and configure Data Grid and cache instances.

In this learning path created by Marcelo Daniel Sales, we will not go into detail about each of the tools used. However, there is a vast collection of documents and articles about OpenShift and Data Grid available at developers.redhat.com.

Create the project on both sites

Create a new project: name _rhdg-xsite_ (Figure 1).

Generate certificates and Identities

First, we need to generate some identities files, certificates, and a key pair. Data Grid uses these artifacts for communication between clients and distributed clusters. For this demo, we generated certificates using the following script:

#!/bin/bash

set -e
set -x

WORK_DIR="$PWD"
CA_CRT="${WORK_DIR}/ca.crt"
CA_KEY="${WORK_DIR}/ca.key"
CA_KEYSTORE="${WORK_DIR}/ca-keystore.p12"

CA_PASS="secretCaPassword"
CA_KEYSTORE_PASS="caSecret"

KEYSTORE_PASS="secret"

C="ES"
S="Valencia"
L="Valencia"
O="infinispan.org"
OU="infinispan-server"
CN="dg"

function fail() {
    echo $@
    exit 1
}

function failIfFileExists() {
    [ -f "${1}" ] && fail "${1} already exists"
}

function generate_ca_keystore() {
    echo "Generating CA. cert=${CA_CRT} key=${CA_KEY}"
    if [[ -f "${CA_CRT}" && -f "${CA_KEY}" ]] ; then
        echo "Certificate and key already exist. Skipping"
    else
        openssl req -new -x509 -keyout ${CA_KEY} -out ${CA_CRT} -passout pass:${CA_PASS} -subj "/C=${C}/ST=${S}/L=${L}/O=${O}/CN=${CN}"
    fi
    if [ -f "${CA_KEYSTORE}" ]; then
        echo "CA keystore already exist. Skipping"
    else
        keytool -keystore ${CA_KEYSTORE} -alias CARoot -import -file ${CA_CRT} -storepass ${CA_KEYSTORE_PASS} -noprompt -trustcacerts -storetype pkcs12
    fi
}

function generate_signed_certificate() {
    local alias=$1
    local crt="${WORK_DIR}/${alias}.crt"
    local signed_crt="${WORK_DIR}/${alias}-signed.crt"
    local keystore="${WORK_DIR}/${alias}-keystore.p12"
    local keystore_pass="${KEYSTORE_PASS}"

    if [ ! -f "${crt}" ] ; then
        echo "Generating certificate ${crt}"
        keytool -genkey -alias ${alias} -keyalg RSA -keystore ${keystore} -keysize 2048 -storetype pkcs12 -storepass ${keystore_pass} -noprompt -dname "CN=${alias}, OU=${OU}, O=${O}, L=${L}, S=${S}, C=${C}"
        keytool -keystore ${keystore} -alias ${alias} -certreq -file "${crt}" -storepass ${keystore_pass} -storetype pkcs12
    fi

    if [ ! -f "${signed_crt}" ] ;  then
        echo "Sign server certificate. ca_cert=${CA_CRT} ca_key=${CA_KEY} server_cert=${crt} signed_cert=${signed_crt}"
        openssl x509 -req -CA ${CA_CRT} -CAkey ${CA_KEY} -in "${crt}" -out "${signed_crt}" -days 365 -CAcreateserial -passin pass:${CA_PASS}
    fi

    echo "Import signed certificate"
    keytool -keystore ${keystore} -alias CARoot -import -file ${CA_CRT} -storepass ${keystore_pass} -noprompt -trustcacerts -storetype pkcs12
    keytool -keystore ${keystore} -alias ${alias} -import -file ${signed_crt} -storepass ${keystore_pass} -storetype pkcs12
}

generate_ca_keystore
generate_signed_certificate dg

Save the above content in a generate_certs.sh file and then run it to generate the certificates. After generating the certificates, we will create secrets on both sites using these same certificates which will be used later by Data Grid.

Let's use the OpenShift Client to create the secrets.

Log into site-1 and copy the authentication token by clicking on the username in the upper-right corner and selecting Copy login command.
Select Display token and copy the line that starts with "oc login --token=sha256".

Create secrets in OpenShift

Run the commands below to create the secrets using the previously generated keys, using the correct project (-n rhdg-xsite in our example) and the correct files (dg-keystore.p12 and ca-keystore.p12):

oc -n rhdg-xsite create secret generic xsite-keystore "--from-file=keystore.p12=$(pwd)/dg-keystore.p12" "--from-literal=password=secret" "--from-literal=type=pkcs12"
oc -n rhdg-xsite create secret generic xsite-truststore "--from-file=truststore.p12=$(pwd)/ca-keystore.p12" "--from-literal=password=caSecret"

Now let's create a secret with identities that will be used to authenticate users who will access the Data Grid. We can use the file structure below as an example:

identities.yaml

credentials:
- username: admin
  password: password
  roles:
  - admin
- username: developer
  password: password
  roles:
  - admin
- username: operator
  password: password
  roles:
  - deployer
- username: user1
  password: password
  roles:
  - monitor

oc create secret -n rhdg-xsite generic --from-file=identities.yaml dg-identities-secret

Repeat the steps above, creating the secrets in Red Hat OpenShift, but this time, point to site-2.

Log into site-2, then copy and paste the command "oc login.." with the authentication token from site-2.

Note: Do not generate other certificates. Use the same certificates for site-2 that you created for site-1. At the end of these steps, both sites should have the following secrets:

dg-identities-secret
xsite-keystore
xsite-truststore

Data Grid installation and configuration on site-1

Access the administration menu in the upper-left corner (Figure 2).

Figure 2: Access the administration menu in the upper left corner.
Select Operators, then Operator Hub.
Find and install the Data Grid operator (Figure 3).

Figure 3: Install the Data Grid operator.
After the installation, select the operator in Installed Operators (Figure 4).

Figure 4: Select the data grid operator.
Select the Infinispan Cluster tab, then Create Infinispan. Note: Red Hat Data Grid is a distributed, in-memory, data-store built from a tried and trusted open-source technology called Infinispan.

Select configure via: yaml view and use the content below as an example:

apiVersion: infinispan.org/v1
kind: Infinispan
metadata:
  name: dg
  namespace: rhdg-xsite
spec:
  security:
    endpointAuthentication: true
    endpointSecretName: dg-identities-secret
  container:
    memory: 1Gi
  expose:
    type: Route
  service:
    container:
      storage: 1Gi
    sites:
      local:
        discovery:
          launchGossipRouter: true
          type: gossiprouter
        encryption:
          routerKeyStore:
            alias: dg
            secretName: xsite-keystore
          transportKeyStore:
            alias: dg
            secretName: xsite-keystore
          trustStore:
            filename: truststore.p12
            secretName: xsite-truststore
        expose:
          type: Route
        maxRelayNodes: 1
        name: site-1
      locations:
        - name: site-2
    type: DataGrid

Click Create.

For now, that's all we need to do on site-1. Let's perform the configuration of site-2.

Data Grid installation and configuration on site-2

Repeat steps 1 - 5 as performed during the installation and configuration of site-1.

Select configure via: yaml view and use the content below as an example:

apiVersion: infinispan.org/v1
kind: Infinispan
metadata:
  name: dg
spec:
  security:
    endpointSecretName: dg-identities-secret
  container:
    memory: 1Gi
  expose:
    type: Route
  service:
    container:
      storage: 1Gi
    sites:
      local:
        discovery:
          launchGossipRouter: false
          type: gossiprouter
        encryption:
          routerKeyStore:
            alias: dg
            secretName: xsite-keystore
          transportKeyStore:
            alias: dg
            secretName: xsite-keystore
          trustStore:
            filename: truststore.p12
            secretName: xsite-truststore
        expose:
          type: Route
        maxRelayNodes: 1
        name: site-2
      locations:
        - name: site-1
          url: infinispan+xsite://<DATAGRID-ROUTE>-<SITE1-PROJECT>.<SITE1-DOMAIN>:443
    type: DataGrid
  configListener:
    enabled: true
  replicas: 1

Substitute the values between < and >. To complete this field, you must look at site-1 for the dg-router container. Find and use the route value and replace it in this field, removing the protocol (Figure 5).

Figure 5: Get Data Grid router URL.
For example: If the value of the route configured in the dg-router is something like:
```
 https://dg-route-site-rhdg-xsite.apps.ocp4.mydomain.com
```

Then the value for url field will be:

infinispan+xsite://dg-route-site-rhdg-xsite.apps.ocp4.mydomain.com:443

Select Create.

At this point, we have a cluster configured between the two OpenShift instances. Look at the Infinispan status on the operator's page, and make sure the CrossSiteViewFormed status is present (Figure 6).

You can find more information regarding Red Hat Data Grid cross-site replication here.

Linux

Java runtimes & frameworks

Kubernetes

Integration & App Connectivity

Automation

Developer tools

Developer Sandbox for Red Hat OpenShift

Programming Languages & Frameworks

System Design & Architecture

Developer Productivity

Secure Development & Architectures

Platform Engineering

Automated Data Processing

Start exploring in the Developer Sandbox for free

Interactive Lessons and Learning Paths

Developer Sandbox Activities

E-Books

Tutorials

Cheat Sheets

Documentation

Red Hat Learning

Tech Talks

Deep Dives

Red Hat Summit 2024