How to migrate smart inventories to constructed inventories

So far in this article series, we talked about managing or migrating configurations of the whole Red Hat Ansible Automation Platform cluster. This article is a little different. We will talk about the migration of one specific object, inventories.

For a number of years, smart inventories were a very useful tool for running workloads on a subset of existing hosts without duplicating them. Recently, Red Hat announced plans to discontinue support of smart inventories in favor of constructed inventories. The feature will be discontinued in the future releases of Ansible Automation Platform.

The change is being made for a good reason, smart inventories are not without their challenges and are limited in the way the filters are being applied. However the announced change also raises a challenge of migrating existing Smart inventories to the new format. For large organizations with hundreds of Smart inventories manual migration would be a long and tedious process.

This article offers a semi-automated process to save time and effort during the migration.

Migration challenges

Smart inventories are defined by filtering existing hosts using one or more conditions joined together by ‘or’ and ‘and’ operators. The conditions are applied sequentially as they are written, which is not going to cause issues in most cases. However for more complex conditions and host names, the resulting list of hosts may not be the desired one. In general case:

condition1 AND condition2 OR condition3

is not the same as

condition1 OR condition3 AND condition2

We’ve seen some organizations use round brackets to control the sequence of specified conditions. This may work but it’s not something that can be configured via GUI.

On the other hand constructed inventories filter hosts from one or more existing inventories, which is a part of the configuration. In addition, conditions are always applied in the following order:

: (or) -> & (and) -> ! (not)

These differences raise two challenges for the migration we are about to do. First, converting conditions specified in a smart inventory as they are listed is likely to produce a different list of hosts as these conditions will be applied in different order in constructed inventory.

Second, if existing smart inventory is currently empty it is not possible to define the source inventory for future constructed inventory. This field is mandatory however and constructed inventory can not be created without it.

The challenges make it very difficult if not impossible to perform fully automated migration. Thus human review of the resulting configurations will be required before configurations are applied, which makes this approach semi-automatic.

The migration approach

Considering the previous migration approach is to:

1. Convert filtering conditions into the format suitable for constructed inventories:
   • The or and and operators will be replaced with : and &.
   • The conditions containing startswith, such as name__startswith=condition will be replaced with condition*
   • The conditions containing icontains, such as groups__name__icontains=condition or name__icontains=condition will be replaced with condition.
   • The conditions containing regex such as name__regex=condition or groups__name__regex=condition will be replaced with ~condition.
2. Determine the list of inventories the hosts from existing smart inventories belong to and use that list in the input field for constructed inventories.
3. Review resulting configurations, make adjustments as required, and apply them.

Migration steps

We’ve done the preparation steps described in part 1: configured Ansible Automation Platform to access configify.aapconfig collection, its dependencies, and to run playbooks from this collection. Now let’s create the following playbook and run it:

---
- name: Run playbook to export Smart inventories in Constructed inventories format
  import_playbook: configify.aapconfig.convert_smart_inventories.yml

The resulting output, after double quotes are removed, will look similar to the following:

controller_objects_inventories_constructed: [
  {'name': 'Smart1 migrated', 'descr': '', 'org': 'Default', 'input': ['Hosts'],
   'source_vars': {'plugin': 'constructed', 'strict': True}, 'limit': '*Host*', 'source': ''},
  {'name': 'Smart2 migrated', 'descr': '', 'org': 'Default', 'input': ['Servers'],
   'source_vars': {'plugin': 'constructed', 'strict': True}, 'limit': 'Server*', 'source': ''}
]

This output in the example corresponds to the following two smart inventories:

controller_objects_inventories_smart: [
  {'name': 'Smart1', 'description': '', 'org': 'Default', 'variables': {},
   'host_filter':  'name__icontains=Host'},
  {'name': 'Smart2', 'description': '', 'org': 'Default', 'variables': {},
   'host_filter': 'name__startswith=Server'}"
]

A human being must review the output produced by the playbook:

• If there are empty input fields, decide which inventory should act as a source or remove the line (which means the inventory will not be migrated).
• Search for "=" in the limit field. These are the filters that need to be converted manually.
• Search for "(" in the limit field. The brackets need to be removed unless they are a part of a regex.
• Check for any other issues.

After the review, add the resulting configuration to the existing CaC and apply it using configify.aapconfig.aap_configure.yml playbook, specifying the following:

tags:
  controller_config_inventories_apply
extra_vars:
  sync_inventory_sources: true

This will create constructed inventories with the word "migrated" in the name corresponding to existing smart inventories and populate hosts in them (synchronize them).

Next, verify that the hosts in the new constructed inventories are the same as in their corresponding smart inventories. To do that, let’s run the configify.aapconfig.compare_inventory_hosts.yml playbook. If there are any differences, troubleshoot them, adjust configurations as required, and reapply.

To finalize the migration we need to:

1. Remove smart inventories from the CaC and rerun automation with: 

   tags:   
    controller_config_inventories_cleanup 
   extra_vars: 
    delete_object: true

2. Rename constructed inventories (remove the word "migrated" from the name) and reapply configuration with:

    tags: 
     controller_config_inventories 
    extra_vars: 
     delete_object: true

3. At this point job templates and workflows that were using smart inventories are broken as these inventories have been deleted. To fix that, simply reapply configurations so that new constructed inventories with the same names are picked up. To achieve this, run automation with:

    tags: 
     controller_config_templates_apply 
     controller_config_workflows_apply

Final thoughts

While the suggested approach is not fully automated due to the differences between two kinds of inventories, playbooks available in the configify.aapconfig collection allow reducing time and effort required for such migration.

Stay tuned for the final installment, discussing migration from Ansible Automation Platform version 2.4 to version 2.5. 

Follow this series:

• Part 1: The first steps on the path of managing an existing Ansible Automation Platform instance as CaC, setting up Ansible Automation Platform accounts, collections, credentials, projects, and job templates required to run the automation, exporting configuration of some objects, handling secrets and special strings in the CaC, and managing configuration drift.

• Part 2: Completing the transition: exporting all objects, formatting configurations for readability, verification, access restrictions, and Git management.

• Part 3: Migrating configurations from AWX 24 to Ansible Automation Platform 2.5.

• Part 4: Migrating smart inventories (deprecated in future Ansible Automation Platform releases) to constructed inventories.

• Part 5: Migrating configurations from Ansible Automation Platform 2.4 to Ansible Automation Platform 2.5.