Terraform Ep 11: Managing Multi-Environment Architecture (Workspaces vs Directories)

1. The Multi-Environment Problem

#

Imagine you have a beautifully written main.tf file that provisions a complete microservices architecture. You run terraform apply, and it deploys perfectly to your AWS account.

Now, your manager asks you to deploy a “Staging” clone of this exact architecture.

If you just change the variables in terraform.tfvars from prod to staging and run terraform apply in the same directory, Terraform will destroy your Production environment and replace it with Staging. Why? Because both environments are fighting over the exact same .tfstate file!

To solve this, we must isolate the state files. There are two industry-standard ways to achieve this: Workspaces and Directory Isolation.

2. Approach A: Terraform Workspaces

#

Terraform Workspaces allow you to use a single directory of HCL code, but maintain multiple, entirely separate .tfstate files underneath the hood.

Practice: Managing Workspaces

#

Create a new directory named terraform-workspaces:

mkdir terraform-workspaces
cd terraform-workspaces

Create a main.tf file:

provider "aws" {
  region = "us-east-1"
}

# Notice we use the built-in 'terraform.workspace' variable!
resource "aws_s3_bucket" "app_data" {
  bucket = "my-company-data-${terraform.workspace}-12345"
}

By default, every Terraform directory starts in the default workspace.

terraform init
terraform workspace list

Expected Terminal Output:

* default

Let’s create two new workspaces: staging and production.

terraform workspace new staging
terraform workspace new production

Verify your current workspace:

terraform workspace show

(Should output production).

Now, switch back to staging and apply the code:

terraform workspace select staging
terraform apply -auto-approve

Terraform just created a bucket named my-company-data-staging-12345.

Now, switch to production and apply the identical code:

terraform workspace select production
terraform apply -auto-approve

Terraform just created a second bucket named my-company-data-production-12345.

The Pros and Cons of Workspaces

#

3. Approach B: Directory Isolation (The Industry Standard)

#

The safer, more robust approach favored by Enterprise Platform Teams is Directory Isolation. Instead of using hidden workspaces, you physically separate the environments into different folders.

Practice: Structuring for Directory Isolation

#

Create a new directory structure like this:

terraform-environments/
├── modules/
│   └── web-app/           # Reusable child module (from Episode 9)
│       ├── main.tf
│       └── variables.tf
├── staging/               # Staging Root Module
│   ├── main.tf
│   └── backend.tf
└── production/            # Production Root Module
    ├── main.tf
    └── backend.tf

The Staging Configuration

#

In staging/main.tf, you call the module and pass Staging-specific values:

# staging/main.tf
module "web_app" {
  source        = "../modules/web-app"
  environment   = "staging"
  instance_type = "t3.micro"  # Save money in staging
  replica_count = 1
}

In staging/backend.tf, you configure an isolated state file:

# staging/backend.tf
terraform {
  backend "s3" {
    bucket = "my-company-terraform-state"
    key    = "staging/terraform.tfstate" # Isolated path!
    region = "us-east-1"
  }
}

The Production Configuration

#

In production/main.tf, you call the exact same module, but with Production values:

# production/main.tf
module "web_app" {
  source        = "../modules/web-app"
  environment   = "production"
  instance_type = "m5.large" # High performance for prod
  replica_count = 5          # High availability
}

In production/backend.tf, you ensure the state file is completely segregated:

# production/backend.tf
terraform {
  backend "s3" {
    bucket = "my-company-terraform-state"
    key    = "production/terraform.tfstate" # Completely different path!
    region = "us-east-1"
  }
}

The Pros and Cons of Directory Isolation

#

Troubleshooting & Common Errors

#

1. Workspace already exists

   • Root Cause: You ran terraform workspace new <name> but someone already created it in the remote backend.
   • Solution: Use terraform workspace select <name> instead.

2. Error configuring the backend: State data found

   • Root Cause: When setting up Directory Isolation, you copied the .terraform hidden folder from staging/ into production/.
   • Solution: Never copy the .terraform directory or local .tfstate files between environment folders. Always run a fresh terraform init in every new environment folder.

Conclusion & Next Steps

#

You now understand the architectural trade-offs between Workspaces and Directory Isolation. For enterprise deployments, Directory Isolation provides the strict blast-radius containment required for Production safety.

However, running terraform apply manually from your laptop is a massive security risk, regardless of which architecture you choose.

In Episode 12: CI/CD Automation with Terraform Cloud and GitHub Actions, we will permanently confiscate Terraform execution rights from your laptop and shift all deployments to automated GitOps pipelines!