Terraform Ep 12: CI/CD Automation with Terraform Cloud and GitHub Actions

1. The Risk of Local Execution

#

Up until this episode, you have been executing terraform plan and terraform apply directly from your terminal.

While necessary for learning, this approach introduces severe enterprise risks:

1. Credential Leaks: Every engineer needs highly privileged AWS Access Keys stored on their laptop. If a laptop is stolen or compromised, your entire AWS account is compromised.
2. Lack of Auditability: If the database goes down, how do you know who ran the terraform apply that caused the outage? There is no central log of terminal executions.
3. Peer Review Bypass: A junior engineer can run terraform apply locally and destroy a production cluster without anyone else on the team reviewing the code first.

The solution is GitOps. We shift the execution environment from the developer’s laptop to a centralized CI/CD runner (like GitHub Actions or GitLab CI).

2. Architecture of a Terraform CI/CD Pipeline

#

The industry-standard Terraform GitOps pipeline follows this strict workflow:

1. Pull Request (PR) Phase: An engineer writes new HCL code and opens a Pull Request on GitHub.
2. Automated Plan: GitHub Actions detects the PR, downloads the code, and runs terraform plan. It then automatically posts the output of the plan as a comment directly on the PR.
3. Peer Review: Senior engineers review the HCL code AND the plan output comment. If it looks dangerous, they reject the PR.
4. Merge & Apply: If approved, the PR is merged into the main branch. GitHub Actions detects the merge and automatically runs terraform apply, deploying the changes to AWS.

The Role of Terraform Cloud (TFC)

#

While you can use AWS S3 and DynamoDB (as learned in Episode 10) to store state for GitHub Actions, managing state securely in a CI pipeline can be tricky.

Terraform Cloud (TFC) is HashiCorp’s managed SaaS platform. It acts as both your Remote Backend (replacing S3/DynamoDB) and your execution environment (if you choose). For this tutorial, we will use TFC strictly as a secure Remote Backend.

3. Practice: Migrating to Terraform Cloud

#

First, you must create a free account on Terraform Cloud and create an Organization (e.g., my-company-org) and a Workspace (e.g., prod-network).

Step 3.1: Updating the Backend Configuration

#

In your Terraform code, replace the s3 backend block with the cloud block:

# main.tf

terraform {
  # The modern syntax for Terraform Cloud
  cloud {
    organization = "my-company-org"
    workspaces {
      name = "prod-network"
    }
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

Step 3.2: Generating an API Token

#

To allow GitHub Actions to authenticate with Terraform Cloud, you must generate a Team API Token in the TFC Web Console. Save this token securely; you will need it in the next step.

4. Practice: Building the GitHub Actions Pipeline

#

Navigate to your GitHub Repository Settings -> Secrets and variables -> Actions.

Add the following Repository Secrets:

• TF_API_TOKEN: The token you generated from Terraform Cloud.
• AWS_ACCESS_KEY_ID: A dedicated CI/CD AWS user key (NOT your personal key).
• AWS_SECRET_ACCESS_KEY: The corresponding secret key.

Step 4.1: The CI/CD YAML Workflow

#

Create the following file in your repository at .github/workflows/terraform.yml:

name: "Terraform GitOps Pipeline"

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  terraform:
    name: "Terraform Plan & Apply"
    runs-on: ubuntu-latest
    
    # Inject the secrets securely into the runner environment
    env:
      TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }}
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

      - name: Terraform Format Check
        run: terraform fmt -check
        continue-on-error: true

      - name: Terraform Init
        run: terraform init

      - name: Terraform Validate
        run: terraform validate

      # Run Plan ONLY on Pull Requests
      - name: Terraform Plan
        if: github.event_name == 'pull_request'
        run: terraform plan -no-color
        
      # Run Apply ONLY when merged to the main branch
      - name: Terraform Apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        run: terraform apply -auto-approve

How the Pipeline Works

#

1. setup-terraform: This official GitHub Action automatically installs the correct version of the Terraform CLI onto the Ubuntu runner and configures the TF_API_TOKEN so it can talk to Terraform Cloud.
2. fmt and validate: The pipeline acts as a strict linter. If your HCL code is improperly indented or contains syntax errors, the pipeline fails immediately, blocking the Pull Request.
3. The if Conditions: This is the core logic. terraform plan is triggered solely by Pull Requests. terraform apply is strictly reserved for code that has successfully landed on the main branch.

Troubleshooting & Common Errors

#

1. Error: Invalid legacy provider address

   • Root Cause: Usually occurs when migrating old local state to Terraform Cloud if the state file contains deprecated provider syntax (from Terraform v0.12 or older).
   • Solution: Run terraform state replace-provider locally before migrating the state to TFC.

2. Error saving plan to Terraform Cloud

   • Root Cause: The TF_API_TOKEN in GitHub Secrets is invalid, expired, or you typed the organization name incorrectly in your cloud {} block.
   • Solution: Regenerate the token in TFC and update the GitHub Secret. Ensure your organization name perfectly matches the URL slug in TFC.

Conclusion & Next Steps

#

You have now successfully automated your infrastructure. Your developers can safely propose architectural changes via Pull Requests, the plans are audited automatically, and execution is handled by machines. This is the definition of Production-Grade Platform Engineering.

However, what happens if an engineer accidentally renames a resource block in the code during a refactor? Terraform will think the old resource needs to be destroyed, and a new one created. If that resource is a database, you just lost all your data!

In Episode 13: Advanced State Manipulation (Surgical Refactoring), we will learn how to safely rename and move resources within the .tfstate file to prevent catastrophic data loss during code refactoring.