Terraform looping of nested data

In this article we will explore how to utilize the Terraform looping mechanism to manage Cisco ACI configuration based on JSON-like data. This enables us to store the configuration in a compact and meaningful data structure which greatly simplifies the operational aspect of our Infrastructure-as-Code file structure.

To do this we must combine the Cisco ACI provider-specific basic HCL arguments with more complex Terraform variable handling through functions, expressions and meta-arguments. This also goes for all other Terraform providers, but because we are dealing with a Cisco Certified DevNet Expert topic, we will be using ACI as an example.

Basic HCL syntax to create hierarchical configuration

Let's begin by creating the configuration without loops by using one HCL block per configuration element. We will be using the CiscoDevNet/aci version 0.7.0 provider, since this is what is mentioned of the DevNet Expert blueprint and software version list.

The references and relationships between configuration elements are defined by specifying the "tenant_dn" and "relation_fv_rs_ctx" as the ACI object model Distinguished Name of the element. To make Terraform manage the configuration dependencies automatically, these IDs are inferred by the .id property of the corresponding Terraform resource inside the main.tf file in this example.

The summary of the resulting resource addresses looks like this:

aci_tenant.tenant_devnet: Creation complete after 0s [id=uni/tn-DevNet]

aci_vrf.vrf_b: Creation complete after 3s [id=uni/tn-DevNet/ctx-B]

aci_vrf.vrf_a: Creation complete after 3s [id=uni/tn-DevNet/ctx-A]

aci_bridge_domain.bd_x: Creation complete after 2s [id=uni/tn-DevNet/BD-X]

aci_bridge_domain.bd_z: Creation complete after 2s [id=uni/tn-DevNet/BD-Z]

aci_bridge_domain.bd_y: Creation complete after 2s [id=uni/tn-DevNet/BD-Y]

Notice how each resource has its own unique address that matches the resource name from the HCL configuration: vrf_a, bd_x, etc.

Looping over a list to create VRFs

As you can see in the basic HCL example above, we are repeating ourselves for every VRF and Bridge Domain that we want to create. In real-world infrastructure you would have a large number of each these resources, and the .tf files would become large and overwhelming. For each of the blocks above there are many other arguments (configuration settings) that may be used, like the arp_flood or ip_learning of the Bridge Domain.

The first loop we will create is a basic for_each loop over a simple list of VRF names. Bridge Domains will be added in later examples.

We define our VRF list as a local value in the HCL block "locals" (plural, with an s at the end). In the configuration block it-self we access the values using the "local." prefix (nonplural, without an s at the end).

The for_each meta-argument ensures that the configuration is created multiple times; one per element in the loop. The required input must be either a map (equal to a Python dict) or a set of strings (equal to Python list or set with unique strings). Since our local.vrfs list is not defined as a Terraform set, we need to convert it using the toset() function.

In every iteration of the loop we have access to each.key and each.value.
each.key will be used as the Terraform resource address, and each.value is in this case used for the VRF name. Since this is a basic set/list and not a map, each.key and each.value is the same string.

The summary of the resulting resource addresses looks like this:

aci_tenant.tenant_devnet: Creation complete after 0s [id=uni/tn-DevNet]

aci_vrf.devnet_vrfs["B"]: Creation complete after 2s [id=uni/tn-DevNet/ctx-B]

aci_vrf.devnet_vrfs["A"]: Creation complete after 2s [id=uni/tn-DevNet/ctx-A]

Notice how each VRF resource is represented as an entry of a dictionary-like address: devnet_vrfs["A"] and devnet_vrfs["B"]

Looping over a map/dict to create VRFs

To get started with more complex data structure, we will now change the local value containing our VRF list to a Terraform map (equal to Python dict).

We can loop of the entries of the map, and use either the key or the value in each entry, as the basis for our configuration. The Terraform resource address will automatically be assigned by the key, but it is up to our HCL argument how to use the value or key.

Looping over a list of maps/dicts to create VRFs

Now we will start to use nested data in the form of a list where each entry is a map (dict) with an entry with the key "name" that we would like to use as our VRF name in the ACI Terraform configuration.

Because Terraform needs a unique address for each configured resource, we need to loop through the data and specify the key of the for_each meta-argument using a nested for-loop.

The result of the for-loop must be a map (dict) and for this reason we enclose the loop in curly brackets {}. Each element needs a key and a value. For each iteration this is defined on the left and right hand side of the "=>" element definition syntax respectively.

The summary of the resulting resource addresses looks like this:

aci_tenant.tenant_devnet: Creation complete after 0s [id=uni/tn-DevNet]

aci_vrf.devnet_vrfs["A"]: Creation complete after 1s [id=uni/tn-DevNet/ctx-A]

aci_vrf.devnet_vrfs["B"]: Creation complete after 1s [id=uni/tn-DevNet/ctx-B]

Notice how each VRF resource is represented as an entry of a dictionary-like address: devnet_vrfs["A"] and devnet_vrfs["B"]

These keys are derived with the for-loop and defined by the value on the left hand side of the "=>" element definition syntax.

Referring to resources created within another loop

Let us add another level of the configuration by defining the bridge domains as a new local value. Each bridge domain needs at least a name and a VRF relation that is tied to an existing configuration resource. Because the VRFs are first created from a simple list of names, we can use the same names as the resource address by referring to them dictionary-style like this: 

The summary of the resulting resource addresses looks like this:

aci_tenant.tenant_devnet: Creation complete after 1s [id=uni/tn-DevNet]

aci_vrf.devnet_vrfs["A"]: Creation complete after 0s [id=uni/tn-DevNet/ctx-A]

aci_vrf.devnet_vrfs["B"]: Creation complete after 0s [id=uni/tn-DevNet/ctx-B]

aci_bridge_domain.devnet_bds["Y"]: Creation complete after 1s [id=uni/tn-DevNet/BD-Y]

aci_bridge_domain.devnet_bds["Z"]: Creation complete after 1s [id=uni/tn-DevNet/BD-Z]

aci_bridge_domain.devnet_bds["X"]: Creation complete after 1s [id=uni/tn-DevNet/BD-X]

Notice how Terraform creates the configuration in the correct order to resolve dependencies. This is not something that happens automatically, but it is guided by the way we asign the "relation_fv_rs_ctx"-value using a Terraform resource ID. By doing it this way, we force Terraform to create the referred resources (VRFs in this case) first. This is the graph features built in to Terraform that ensures an execution plan that takes care of dependencies.

Looping over a map/dict of lists to create VRFs and Bridge Domains

As you may notice in the example above, we are using the same strings multiple times when defining the VRF name and later referring to the same names. We can avoid repeating ourselves by collapsing all of the data into a minimal structure, in this case a dict of lists.

This forced us to flatten and loop over the data to create simple maps/dicts and lists that Terraform can use in for_each loops. The flatten() function takes a multi-dimensional list and converts it to a simple list (of strings, maps or other objects). Nesting multiple for-loops within pairs of square brackets can iterate through very complex data, and at the inner-most loop create map/dict entries by using the curly brackets {}

Even though this adds complexity to the HCL configuration blocks and variable manipulation, it simplifies the actual data input structure which is what we usually have to deal with on a daily basis. All the input for the configuration are defined only in two JSON-like lines where the VRF names A and B are the map/dict keys and the list items X, Y and Z are the related bridge domain names.

The summary of the resulting resource addresses looks like this:

aci_tenant.tenant_devnet: Creation complete after 1s [id=uni/tn-DevNet]

aci_vrf.devnet_vrfs["A"]: Creation complete after 0s [id=uni/tn-DevNet/ctx-A]

aci_vrf.devnet_vrfs["B"]: Creation complete after 0s [id=uni/tn-DevNet/ctx-B]

aci_bridge_domain.devnet_bds["Y"]: Creation complete after 1s [id=uni/tn-DevNet/BD-Y]

aci_bridge_domain.devnet_bds["Z"]: Creation complete after 1s [id=uni/tn-DevNet/BD-Z]

aci_bridge_domain.devnet_bds["X"]: Creation complete after 1s [id=uni/tn-DevNet/BD-X]

The result is exactly the same as when we defined the VRF and bridge domains with separate local list-values, but the required input is highly reduced.

Conclusion

When you look at the basic usage example for a Terraform provider, it may be easy to just copy and paste directly from the documentation and then repeat the HCL code blocks for every bit of configuration that you want to manage. This is not always the best solution, as the HCL files becomes very large when you add new configuration - especially when the HCL syntax is more complex than in the examples above.

Using looping and resource references dramatically reduces the complexity of the input data, but the Terraform code itself may seem more complex. This is a trade off that I usually suggest you make, to keep your Infrastructure-as-Code repositories manageable and easy to operate.

Written by:
Andreas Baekdahl
Cisco Certified DevNet Expert #20220006
3xCCIE and CCDE