Terraform for Loop: Managing Multiple Resources

Terraform for Loop allows you to create and manage multiple resources in a more efficient and streamlined manner

Terraform is a powerful infrastructure-as-code tool that allows you to define and manage your cloud resources in a declarative way. It provides a wide range of features and functionalities that make it easy to provision and manage resources across different cloud providers. One such feature is the "for loop" construct, which allows you to create and manage multiple resources in a more efficient and streamlined manner.

However, when it came to Day 2 operations, we were running into some issues where Terraform was destroying our resources, causing us headaches that were slowing us down.  

Learn how to PerfectScale helps to master Day 2 Kubernetes operations, where optimizing costs and performance emerges as a pivotal focus.

In this article, we'll delve into the challenges we encountered with Terraform and how we plan to mitigate them effectively by utilizing Terraform for Loop and making strategic key choices.

‍

The Problem: Terraform Destroying Resources

Terraform is an Infrastructure as a Code (IaaC) tool widely used to manage infrastructure. It allows you to define required resources in a declarative manner. All our cloud infrastructure such as VPCs, Kubernetes Clusters, RDS, S3 Buckets, Service Accounts are provisioned and maintained by Terraform.

Our product is growing at a rapid pace and Terraform allows us to build our IaaC in a fast and easy manner. But on Day 2, we faced an issue that Terraform was taking unexpected actions that were destroying our resources. We were using Terraform Loops incorrectly, and were not able to change existing resources anymore, just add new ones. We refactored our code and would like to share it to help you avoid a headache.

Benefits of Terraform Loops

‍
Terraform loops bring numerous benefits to your infrastructure management process. One major advantage is the ability to automate the creation and configuration of resources that follow a similar pattern. With loops, you can define a single resource block and then dynamically generate multiple instances of that resource based on different variables or conditions. This not only saves time but also reduces the chances of errors and ensures consistency across your infrastructure.

‍
Let's dive deeper into the benefits of using Terraform loops:

‍
1. Improved Efficiency:

‍
By utilizing Terraform loops, you can significantly improve the efficiency of your infrastructure management process. Instead of manually defining each resource individually, you can leverage loops to automate the provisioning and scaling process. This allows you to quickly and easily scale the number of resources based on your needs, saving you valuable time and effort.

‍
For example, imagine you have an auto-scaling group with multiple instances. Instead of defining each instance individually, you can use a loop to generate the required number of instances based on a variable. This not only simplifies the configuration process but also ensures consistency across all instances.

‍
2. Enhanced Scalability:

‍
Terraform loops enable you to easily scale your resources based on your requirements. Whether you need to increase the number of instances in an auto-scaling group or add more virtual machines to a cluster, loops provide a flexible and efficient solution.

‍
By defining a loop, you can dynamically generate multiple instances of a resource, allowing you to scale up or down as needed. This scalability is particularly valuable in scenarios where you have a large number of similar resources. Instead of manually creating and configuring each resource, you can rely on loops to automate the process, saving you time and reducing the chances of human error.

‍
3. Consistency and Standardization:

‍
Using Terraform loops ensures consistency and standardization across your infrastructure. By defining a single resource block and generating multiple instances based on variables or conditions, you can ensure that all resources follow the same configuration and settings.

‍
Consistency is crucial in maintaining a stable and reliable infrastructure. With loops, you eliminate the risk of inconsistencies that may arise from manually configuring each resource. This not only simplifies the management process but also reduces the chances of errors and ensures that your infrastructure is always in a desired state.

‍
4. Increased Maintainability:

‍
Terraform loops contribute to increased maintainability of your infrastructure. By abstracting the creation and configuration of resources into a loop, you make it easier to manage and update your infrastructure in the future.
When you need to make changes or updates to your resources, you can simply modify the loop configuration, and the changes will be applied to all generated instances. This centralized approach to managing resources simplifies maintenance tasks, reduces the risk of errors, and improves overall efficiency.

‍
In conclusion, Terraform loops offer significant benefits in terms of efficiency, scalability, consistency, and maintainability. By leveraging loops, you can automate the creation and configuration of resources, scale your infrastructure easily, ensure consistency across all resources, and simplify maintenance tasks. Incorporating Terraform loops into your infrastructure management process can greatly enhance your overall workflow and productivity.

‍

The Basics of Terraform Loops

Sometimes we need to create multiple resources at once. So what do we do?

resource "aws_s3_bucket" "bucket0" {
  bucket = "wgk24wt55-demo0"
}
resource "aws_s3_bucket" "bucket1" {
  bucket = "wgk24wt55-demo1"
}
resource "aws_s3_bucket" "bucket2" {
  bucket = "wgk24wt55-demo2"
}

I'm kidding, of course, we would not make it like this as it would break the DRY principle.

We use loops instead.

Types of Loops:

1. Count Loops: The simplest way to make a loop based on predefined or counted numbers to create resources.
2. For_each Loops: This loop allows you to iterate over variables in maps and sets based on keys to create resources.
3. For Loops: These loops don’t allow you to directly create resources, but allow you to filter or transform lists, sets, tuples, or maps.

Terraform Count Loop

Let's look for some examples of how we can create multiple S3 buckets with the count loops:

resource "aws_s3_bucket" "bucket" {
  count = 3
  bucket = "wgk24wt55-demo${count.index}"
}

# Or lets add more controll, and iterate over tuple
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  count = length(local.buckets)
  bucket = "wgk24wt55-${local.buckets[count.index].name}"
}

After terraform apply our S3 buckets are created. We can see them listed in our state:

aws_s3_bucket.bucket[0]
aws_s3_bucket.bucket[1]
aws_s3_bucket.bucket[2]

But later, we decided that we don't need the demo1 bucket anymore, let's delete it:

locals {
  buckets = [
    { name = "demo0" },
    { name = "demo2" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  count = length(local.buckets)
  bucket = "wgk24wt55-${local.buckets[count.index].name}"
}

Running terraform apply again, and what do we see?

Plan: 1 to add, 0 to change, 2 to destroy.

But wait, why would Terraform to destroy 2 resources when we only deleted 1?

# aws_s3_bucket.bucket[1] must be replaced

~ bucket = "wgk24wt55-demo1" -> "wgk24wt55-demo2" # forces replacement

So demo1 and demo2 will be deleted, and only after that demo2 will be recreated.

The reasons is key of state objects. When we use count it marks aws_s3_bucket.bucket[X] with numbered index. And object with index 1 had a different name before.

It might be ok for some stateless resources, but not for S3 buckets with data. So how can this be avoided?

Terraform For_each Loop

Let’s try switching to another loop: for_each

locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}
# !Will not work!
resource "aws_s3_bucket" "bucket" {
  for_each = local.buckets
  bucket = "wgk24wt55-${each.value.name}"
}

But this snippet will not work, because for_each can only iterate over a map or a set of strings. Here we should use a for loop, which can help us transform tuple to map where key would be presented as bucket.name.

locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}
resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : bucket.name => bucket }
  bucket = "wgk24wt55-${each.value.name}"
}

The above for loop transformed the buckets variable:

# From:
buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]

# Into:
buckets = {
       demo0 = {
           name = "demo0"
        }
       demo1 = {
           name = "demo1"
        }
       demo2 = {
           name = "demo2"
        }
    }

State Migration

However, if we apply these updates, it still will lead to deleting all our buckets. To avoid it, we need to migrate our state first:

terraform state mv "aws_s3_bucket.bucket[0]" "aws_s3_bucket.bucket[\"demo0\"]"
terraform state mv "aws_s3_bucket.bucket[1]" "aws_s3_bucket.bucket[\"demo1\"]"
terraform state mv "aws_s3_bucket.bucket[2]" "aws_s3_bucket.bucket[\"demo2\"]"

Let’s check the state list:

aws_s3_bucket.bucket["demo0"]
aws_s3_bucket.bucket["demo1"]
aws_s3_bucket.bucket["demo2"]

Now if we apply a new code we will get:

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Change as expected

The key right now is a bucket name that we specified via the for loop, so if we delete demo1 bucket now, other buckets will stay untouched as we expected.

locals {
  buckets = [
    { name = "demo0" },
    { name = "demo2" },
  ]
}
resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : bucket.name => bucket }
  bucket = "wgk24wt55-${each.value.name}"
}

Plan: 0 to add, 0 to change, 1 to destroy.

Best Practices for Using Terraform for Loop

While Terraform loops can greatly simplify and automate the management of multiple resources, it's important to follow some best practices to ensure efficient and reliable infrastructure management.

1. Keep the loop logic simple:

Complex logic within loops can make your code harder to understand and maintain. It's recommended to keep the loop logic simple and separate any complex decision-making processes into separate modules or variables.

2. Use input variables:

Input variables allow you to parameterize your Terraform configurations and make them more flexible. By using input variables, you can easily modify loop configurations without needing to modify the underlying infrastructure code.

3. Test and validate your configurations:

As with any infrastructure code, it's crucial to test and validate your configurations before applying them to your actual infrastructure. This helps avoid any unforeseen issues and ensures that your loop configurations work as expected.

How to Use for loop in Terraform

We can also do more complex keys. For example:

locals {
  buckets = [
    { name = "demo0", namespace = "A" },
    { name = "demo2", namespace = "B" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : "${bucket.namespace}-${bucket.name}" => bucket }
  bucket = "${each.value.namespace}-wgk24wt55-${each.value.name}"
}

# Which will give us:
aws_s3_bucket.bucket["A-demo0"]
aws_s3_bucket.bucket["B-demo2"]

We are striving to keep keys simple but unique. Mostly we use a combination of name + namespace, but you can also add your domain or environment name. Keep in mind, changing the key requires either deletion or one more state movement, so I don’t recommend including frequently changing parameters such as tags or access policies to your keys.

Now that you understand the benefits and basics of Terraform loops, it's time to put them into action. Start by identifying the resources in your infrastructure that could benefit from automation and scalability. Experiment with different loop configurations and test them in a non-production environment before deploying them to your production environment.
Remember to follow best practices, keep the loop logic simple, and continuously validate and test your configurations. With Terraform loops, you can take your infrastructure management to the next level and effectively manage multiple resources with ease.

Ready to optimize your Kubernetes cluster management and take your infrastructure to the next level? With PerfectScale, you can enhance efficiency and scalability by intelligently managing your Kubernetes resources. Our advanced algorithms and machine learning techniques ensure your workloads are optimally scaled, reducing waste and cutting costs without compromising performance. Join forward-thinking companies who have already streamlined their Kubernetes environments with PerfectScale. Sign up and Book a demo to experience the immediate benefits of automated Kubernetes cost optimization and resource management. Ensure your infrastructure is always perfectly scaled and cost-efficient, even when demand fluctuates.