Terraform Ep 13: Advanced State Manipulation (Surgical Refactoring)

1. The Danger of Renaming Resources

#

Imagine you wrote the following code months ago:

resource "aws_db_instance" "my_db" {
  engine         = "postgres"
  instance_class = "db.t3.micro"
  # ...
}

Now, your company requires strict naming conventions. You decide to refactor your code and rename the block from my_db to production_postgres_primary:

resource "aws_db_instance" "production_postgres_primary" {
  engine         = "postgres"
  instance_class = "db.t3.micro"
  # ...
}

If you run terraform plan right now, Terraform will output:

• Destroy: aws_db_instance.my_db
• Create: aws_db_instance.production_postgres_primary

Terraform is preparing to delete your entire Production Database!

Why? Because Terraform only looks at the logical names in the code compared to the logical names in the .tfstate file. It sees that my_db is missing from the code (so it must be destroyed), and production_postgres_primary is new (so it must be created). It does not realize they are the exact same physical database.

2. Safe Refactoring: The moved Block

#

To solve this, Terraform v1.1 introduced the moved block. This block acts as a set of instructions directly to the Terraform engine, telling it: “Hey, I didn’t delete this resource. I just changed its name.”

Let’s safely refactor the database. In your main.tf, add the moved block:

# The newly renamed resource
resource "aws_db_instance" "production_postgres_primary" {
  engine         = "postgres"
  instance_class = "db.t3.micro"
}

# The instruction block
moved {
  from = aws_db_instance.my_db
  to   = aws_db_instance.production_postgres_primary
}

Now, when you run terraform plan, Terraform will output:

Terraform will perform the following actions:

  # aws_db_instance.my_db has moved to aws_db_instance.production_postgres_primary
    resource "aws_db_instance" "production_postgres_primary" {
        id = "db-123456789"
        # ...
    }

Plan: 0 to add, 0 to change, 0 to destroy.

Zero destructions! When you run terraform apply, Terraform will simply update the name inside the .tfstate file without touching the physical AWS database. You can safely delete the moved block from your code in the future once the state file has been updated.

3. Surgical Operations with the State CLI

#

Sometimes, moved blocks are not enough, and you must operate directly on the .tfstate file using the terraform state CLI command suite.

Scenario A: Forgetting a Resource (state rm)

#

Suppose you created an S3 bucket via Terraform, but the Security Team says they need to take over management of that bucket using their own custom tool. You need Terraform to “forget” the bucket without actually deleting the bucket from AWS.

First, list all resources in your state file to find the exact name:

terraform state list

(Output: aws_s3_bucket.legacy_data)

Now, instruct Terraform to remove the mapping:

terraform state rm aws_s3_bucket.legacy_data

Expected Terminal Output:

Removed aws_s3_bucket.legacy_data
Successfully removed 1 resource instance(s).

You can now safely delete the resource "aws_s3_bucket" "legacy_data" block from your main.tf. When you run terraform plan, Terraform will do nothing, because it no longer tracks the bucket. The physical bucket remains perfectly safe in AWS.

Scenario B: Adopting Manual Resources (import)

#

The opposite scenario is even more common. An engineer manually clicked through the AWS Console to create an EC2 instance (ID: i-0abcdef1234567890), and now your team wants to bring it under Terraform management.

Step 1: Write the empty HCL block You must first write the HCL shell in your main.tf to give the resource a logical home.

resource "aws_instance" "adopted_server" {
  # Leave this empty for now, or fill in what you know
}

Step 2: Execute the Import Command Tell Terraform to map the logical name in your code to the physical ID in AWS.

terraform import aws_instance.adopted_server i-0abcdef1234567890

Expected Terminal Output:

aws_instance.adopted_server: Importing from ID "i-0abcdef1234567890"...
aws_instance.adopted_server: Import prepared!
  Prepared aws_instance for import
aws_instance.adopted_server: Refreshing state... [id=i-0abcdef1234567890]

Import successful!

Step 3: Align the Code If you run terraform plan now, Terraform will try to change the server because your HCL block is mostly empty. You must manually inspect the state file (terraform show) and fill out the ami and instance_type arguments in your main.tf until terraform plan returns 0 changes.

4. Disaster Recovery: state pull and state push

#

What happens if someone accidentally corrupts the remote state file in S3, and the DynamoDB lock is permanently jammed?

You can surgically pull the raw JSON state file from the remote backend down to your local laptop, fix the JSON manually in VS Code, and push it back up.

# Download the state file from S3 to your laptop
terraform state pull > backup.tfstate

# Edit the backup.tfstate file carefully...

# Force upload the repaired state file back to S3, bypassing locks
terraform state push -force backup.tfstate

Troubleshooting & Common Errors

#

1. Cannot move to a non-existent resource

   • Root Cause: In your moved block, the to = destination does not actually exist in your main.tf.
   • Solution: Ensure you have written the new resource block (or module block) before running terraform plan.

2. Resource already managed by Terraform

   • Root Cause: You ran terraform import on a physical resource ID that is already mapped to a different logical name in your current state file.
   • Solution: You cannot import a resource twice. Use terraform state mv to rename it internally instead.

Conclusion & Next Steps

#

You are now a surgeon. You can adopt rogue infrastructure, refactor messy codebases, and gracefully hand over resources to other teams without causing a single second of downtime.

As you start managing hundreds of Terraform directories (one for every microservice across 5 different environments), you will notice a massive problem: You are copy-pasting the backend "s3" and provider "aws" blocks into every single directory. The codebase is no longer DRY.

In Episode 14: Achieving DRY Architecture with Terragrunt, we will introduce a revolutionary wrapper tool that solves Terraform’s most glaring architectural flaw.