Satellite webhook and Insights automation for efficient RHEL operations

Red Hat Satellite is an infrastructure management tool designed for the management and operations of Red Hat Enterprise Linux (RHEL) environments. It allows system administrators to scale the management of their datacenters to thousands of hosts at ease, while implementing and enforcing secure and compliant standard operating environments (SOE).

In this article, we investigate the use of Satellite webhooks and automation to interact with third-party tooling and react to events occurring within Satellite. This is traditionally used to improve the overall operational efficiency by either forwarding events to third-party auditing and monitoring tools (e.g., Splunk), or using events as triggers for automation (e.g., Red Hat Ansible Automation Platform, ServiceNow). To this point, we explore how this approach can be used to perform automation tasks on Red Hat Insights by integrating with Red Hat Hybrid Cloud Console (HCC).

Although our example shows simple operations automating the synchronization and assignment of hostgroups and inventory groups between Satellite and Red Hat Insights, the approach can be replicated and extended to perform any automation in your organization. For example, one could think of enforcing policies in Insights (e.g., compliance or baselines assignment) or using automation in conjunction with Satellite scheduling to generate bespoke reports from Insights data.

Red Hat Satellite webhooks and automation

Red Hat Satellite recently released a new webhook functionality to perform action(s) when events occur within Satellite. In this article, the functionality is used to automatically get alerts triggered when an errata installation is complete on RHEL hosts.

With webhooks in place, one can monitor all operations happening on their Satellite and use them to automate their operational processes more efficiently. Traditionally, webhooks are used to drive monitoring and automation with third party applications such as Splunk, ServiceNow, Ansible Automation Platform, or Event-Driven Ansible, to name a few.

In this article, we decide to call Ansible automation hosted on Satellite itself as an action to a Satellite webhook trigger. All we need to do is launch Satellite automation from its Job Template API when a webhook event is triggered.

We use data provided as part of the webhook event to drive operations tasks with Ansible. In our example, we use Ansible automation to integrate to Red Hat Hybrid Cloud Console and perform queries against Red Hat Insights API. See Figure 1.

The next sections provide relevant and useful resources on Red Hat Satellite and Red Hat Insights to learn more about each feature.

References for Red Hat Satellite

More information on Satellite webhooks can be found in the product documentation for Red Hat Satellite. A list of all available Satellite webhook events is also available.

Using Ansible automation and running a Job Template from Satellite is documented in this knowledge base article and a list of job template examples is provided in the product documentation. 

Further, the API documentation for your Satellite instance can be found by replacing <your satellite fqdn> in the following URL: https://<your satellite fqdn>/apidoc/v2.html. In particular we make use of the /apidoc/v2/job_invocations/create.en.html method in the example when configuring Satellite webhooks to launch a job template automation.

References for Red Hat Insights

Red Hat Insights APIs documentation can be found on the Hybrid Cloud Console API documentation. A cheat sheet on using the Red Hat Insights API is available in case you need help getting started interacting with the API.

In this article, we use the Inventory Groups API for CRUD (Create, Read, Update, Delete) operations, as shown in Figure 2. The documentation for the relevant groups endpoint can be found under Managed Inventory API.

Finally, the following knowledge base article documents various operations that can be performed using the API to automate the management of your inventory groups and your system assignment.

Creating a service account for the integration

In order for Red Hat Satellite to perform requests against Insights APIs, we first need to create a Service Account with appropriate permissions in Red Hat Hybrid Cloud Console.

To do so, navigate to Settings (gear icon) and select Service Accounts. Clicking Create service account and going through the creation wizard results in the creation of a new service account for your Satellite automation. 

Next, we need to provide the appropriate RBAC permissions to the service account to perform the intended operations. In our example, our automation performs CRUD operations on inventory groups and inventory hosts. As such, we need inventory:hosts:read, inventory:hosts:write (can be inherited from Inventory Hosts Administrator role) and inventory:groups:read, inventory:groups:write (can be inherited from Inventory Groups Administrator role) permissions.

We can create a new user group with the Inventory Hosts Administrator and Inventory Groups Administrator roles and assign the service account from the Groups page under User Access. Additional documentation about managing service accounts in Hybrid Cloud Console is available in the product documentation.

Last step for Satellite to use our credentials is to set those as global parameters in Satellite. To do so, log in to Satellite and navigate to Configure and Global Parameters. From there, create two new name/value pairs called hcc_client_id and hcc_client_secret (select string for validation, and hidden value option) and set their value to the credentials retrieved in HCC while creating your service account. See Figure 3.

From now on, your Satellite automation can query global parameters to retrieve the HCC service account credentials required to perform Insights API queries.

Creating a job template for the Satellite automation

We can now configure the Ansible automation in Satellite that is going to be launched when an event triggers. This can be done by creating a new job template.

Note that the code used in the article is provided in GitHub to facilitate imports. Our job template is available in custom_automation_satellite_to_insights.erb file. One can download and import it in Satellite, or create it from scratch.

From Hosts and Job Templates, click Import and select the downloaded file. The import creates a new job template with the relevant automation code, as shown in Figure 4.

Looking at the Inputs tab of our job template, we notice a set of template inputs called action, hostgroup_name, and insights_id. Those inputs are populated at runtime by the webhook template after parsing the triggered event. These inputs are then replaced in the template when generating the Ansible playbook that performs automation against Insights API.

Looking at the automation, we can see how the template performs lookups of parameters and inputs. For example, global parameters are queried using <%= host_param('hcc_client_id') %> whereas template inputs are queried using <%= input('hostgroup_name') %>. Further one may notice how the template performs different tasks based on the event being triggered according to the action input condition (e.g., hostgroup created and updated, host updated, etc).

The Job tab of our job template is set to Ansible Playbook for Job Category and Ansible for Provider Type. Figure 5 depicts this menu.

The last step we need is getting the id of our new job template, as this is a required parameter when calling Satellite API to launch a job. An easy way to find out the id is to look at the Satellite URL parameter when editing the job template in the UI. From Hosts, Job Templates, click on your job template and look at the URL. It should be in the following format: https://<your satellite fqdn>/job_templates/<job template id>-<job_template_name>/edit, where <job template id> is the id we are after. We use the id in the following section while configuring our webhook template.

Creating webhooks for triggering Satellite automation

Now that our job template is configured, we can execute it from the Satellite API by passing its job_template_id and a target host to run the automation on. In the next section, we configure a Satellite webhook and associated webhook template to build this request and call the Satellite API to launch our automation. 

Creating a new webhook template

First, we need to create a new wehbook template that is used to parse the event data and generate an appropriate payload for our Satellite API query.

The code used in the article is provided in GitHub to facilitate imports. Our webhook template is available in webhook_template_host_groups.erb file. One can download and import it in Satellite, or create it from scratch.

From Administer and Webhook Templates, click Create template. Use the code of the file provided for your template. Note that the job_template_id values (two locations in the template) must be replaced by the id of your job template you retrieved in the previous section. Similarly, the name search lookup (search_query parameter) must be replaced by your own satellite FQDN (fully qualified domain name) so that it is used as a target for running automation. The value should look like: name = <your satellite fqdn>. See Figure 6.

Note that the webhook template is conditioned based on the event name (including hostgroup_ and host_). The payload is populated according to the need of the job template automation configured earlier. This allows to grab and populate hostgroup and host related parameters (e.g. hostgroup_name and insights_id). 

The webhook template is now configured and ready to be used by Satellite webhooks. We configure them in the next section.

Creating webhooks for Satellite events

The last step of our configuration is to create webhooks in Satellite to listen for triggered events and run the appropriate action. In our case, we want to call Satellite API to launch automation when an event related to host or hostgroup is triggered. 

Configuring a webhook in Satellite can be done from Administer and Webhooks. Clicking Create new takes you to a configuration screen.

The first step consists of selecting an event to subscribe to (e.g. Hostgroup Created). The target URL consists of the Satellite API endpoint that is called to launch the job template automation. The name of your Satellite instance must be replaced in the example provided following this format: https://<your satellite fqdn>/api/job_invocations. The template can be set to the new webhook template created in the previous section. Finally, set POST as HTTP Method and select Enabled, as shown in Figure 7.

In the Credentials tab, set User and Password credentials that have the right to launch job templates on your Satellite API, as shown in Figure 8.

Once done, you should be able to replicate a similar setup for other Satellite events. We are specifically interested in ‘Host Updated’ event, and ‘Hostgroup Created/Destroyed/Updated’ events for our automation example. Other events on host can be ignored.

That’s it! We can now test our configuration end-to-end and validate that it is working as expected.

Validating our configuration and automation

We are now ready to test our configuration end-to-end. The easiest way is to create and delete hostgroups from Configure and Host Groups.

Assuming all steps are configured correctly, each individual action should trigger the execution of our job template. You can monitor the launch and execution of the automation from Monitor and Jobs. From there, you can see all job invocations and their status. You can also access the generated automation playbook that is executed for each job as this can be useful for troubleshooting.

If everything is successful, you can then validate that the new groups are created/deleted in Red Hat Insights by navigating to Inventory and Groups. 

Similarly, you can start assigning or changing the assignment of hosts to hostgroups in Satellite and validate that they get assigned accordingly in Red Hat Insights.

Discussion and limitation or our example

The example provided in this article is primarily meant to showcase how Satellite automation and webhooks can work together to perform integration to third-party applications. The code provided is an example of integrating to Red Hat Insights API. The code is not supported by Red Hat and is not meant to be used in your production environment without further testing and development to ensure it matches your requirements.

Here are some points about our automation example that one should be aware of:

• A nested Satellite hostgroup is translated to a new group in Insights (there is no hierarchy for groups in Insights) following the naming pattern: <parent name>/<nested name>.
• Name change on a Satellite hostgroup results in a new group being created in Insights (the previous group remains and must be deleted manually in Insights).
• System assignment only works if the target group is present in Insights. One may want to replicate existing Satellite hostgroups in Insights prior to enabling the automation.

Conclusion

This article described how Red Hat Satellite events, webhooks and job templates can constitute a real platform for automating management operations. We use those features to interact with Red Hat Insights API and manage inventory groups and system assignment automatically according to Satellite hostgroup configuration. The job template and webhook template files used in this example are available for download in a GitHub repository.

Although our example shows simple operations synchronizing Red Hat Satellite and Red Hat Insights, the approach can be replicated to perform any other operational tasks that can be automated in your organization.

We are very interested to get your thoughts and feedback on ways to improve and grow our product. Please share your experience with us by using the Feedback form located on the right side of the Hybrid Cloud Console.