Skip to content

solution.md

Latest commit

 

History

History
249 lines (166 loc) · 9.26 KB

solution.md

File metadata and controls

249 lines (166 loc) · 9.26 KB

Mini-Project 3: Comprehensive Guide to Terraform Modules and Backends

This guide provides a detailed, beginner-friendly walkthrough for setting up a remote backend and provisioning infrastructure using Terraform modules. It addresses common issues and incorporates best practices for a successful project submission.

Table of Contents

  1. Project Overview
  2. Prerequisites
  3. Step 1: Verify AWS Authentication
  4. Step 2: Set Up the Terraform Backend Manually
  5. Step 3: Configure the Terraform Project
  6. Step 4: Provision the Infrastructure
  7. Step 5: Clean Up Resources
  8. Troubleshooting Common Errors
  9. Recommended .gitignore Configuration
  10. Side Note: terraform apply --auto-approve

1. Project Overview

The goal of this project is to demonstrate proficiency with two core Terraform concepts:

  • Remote State Management: Using an S3 bucket and a DynamoDB table as a remote backend to securely store and lock the Terraform state file. This is crucial for collaborative environments.
  • Terraform Modules: Structuring your infrastructure code into reusable modules for a VPC and an S3 bucket to promote organization and code reuse.

We will provision a new VPC and an S3 bucket with advanced features like versioning, logging, and lifecycle policies.

2. Prerequisites

Before you begin, ensure you have the following:

  • AWS Account: An active AWS account with programmatic access (Access Key ID and Secret Access Key).
  • AWS CLI: The AWS Command Line Interface installed and configured. You can test your configuration with aws configure list.
  • Terraform: Terraform installed on your local machine. You can verify the installation with terraform --version.

3. Step 1: Verify AWS Authentication

It is critical to confirm that your AWS CLI is correctly authenticated before running any commands. This ensures that Terraform can interact with your AWS account.

Run the following command:

aws sts get-caller-identity

Expected Output:

The output should show your AWS Account ID, User ID, and ARN.

{
    "UserId": "AIDA...",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/your-user-name"
}

Action Required: Take a screenshot of the command and its output. This will be part of your submission to prove you are authenticated.

aws-sts-get-caller-identity

4. Step 2: Set Up the Terraform Backend Manually

Terraform needs a backend to store its state file. When working in a team, this backend must be remote (i.e., not on your local machine). We use an S3 bucket for storage and a DynamoDB table for state locking to prevent concurrent modifications.

These resources must be created before you initialize the main Terraform project that will use them. For this project, we will create them manually via the AWS CLI.

  1. Choose a Unique S3 Bucket Name: Bucket names are globally unique. For this guide, we use daretechie-devops-tf-state-bucket, but you must replace this with your own unique name.

  2. Create the S3 Bucket:

    aws s3api create-bucket --bucket daretechie-devops-tf-state-bucket --region us-east-1

  3. Enable Versioning: This is a best practice to keep a history of your state files, allowing you to revert to a previous state if necessary.

    aws s3api put-bucket-versioning --bucket daretechie-devops-tf-state-bucket --versioning-configuration Status=Enabled

  4. Create the DynamoDB Table for State Locking:

    aws dynamodb create-table \
    --table-name terraform-lock \
    --attribute-definitions AttributeName=LockID,AttributeType=S \
    --key-schema AttributeName=LockID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

Action Required: Take a screenshot of the successful creation of these resources.

backend-setup

5. Step 3: Configure the Terraform Project

Now, navigate to the main project directory and prepare the Terraform files.

cd /home/daretechie/DevProject/GitHub/devops-training/Module-4/mini-project-03/terraform-modules-vpc-s3

a. Fix Module Paths

The error Invalid module source address occurs because Terraform requires a ./ prefix for local paths. Correct the source arguments in main.tf.

File: main.tf

module "vpc" {
  source = "./modules/vpc"
  vpc_cidr = "10.0.0.0/16"
}

module "s3" {
  source      = "./modules/s3"
  bucket_name = "daretechie-devops-unique-bucket-for-hosting"
}

b. Configure the Backend

Your backend.tf file tells Terraform where to store the state. Make sure the bucket name matches the one you created manually.

File: backend.tf

terraform {
  backend "s3" {
    bucket         = "daretechie-devops-tf-state-bucket" # Use your unique bucket name here
    key            = "terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-lock"
  }
}

6. Step 4: Provision the Infrastructure

With the configuration corrected, you can now initialize Terraform, review the plan, and apply it.

  1. Initialize Terraform: This command downloads the necessary providers and configures the backend.

    terraform init

    You should see a message "Successfully configured the backend "s3"" and "Terraform has been successfully initialized!".

  2. Plan the Changes: This command shows you what resources Terraform will create, change, or destroy.

    terraform plan

  3. Apply the Changes: This command executes the plan and builds the infrastructure.

    terraform apply

    Terraform will ask for confirmation. Type yes and press Enter.

Action Required: Take a screenshot of the successful terraform apply output, showing the resources created.

terraform-apply-output

7. Step 5: Clean Up Resources

A critical part of infrastructure management is cleaning up resources you no longer need to avoid incurring costs.

  1. Destroy Terraform-Managed Resources:

    terraform destroy

    Confirm the action by typing yes.

    Action Required: Take a screenshot of the successful terraform destroy output.

    terraform-destroy-output

  2. Manually Delete Backend Resources: The S3 bucket and DynamoDB table were created manually, so they must be deleted manually.

    • Empty and Delete the S3 Bucket:

      aws s3 rb s3://daretechie-devops-tf-state-bucket --force

    • Delete the DynamoDB Table:

      aws dynamodb delete-table --table-name terraform-lock

8. Troubleshooting Common Errors

  • Error: Invalid module source address

    • Cause: The source path for a local module in main.tf is missing the ./ prefix.
    • Solution: Change source = "modules/vpc" to source = "./modules/vpc".

  • Error: S3 bucket "..." does not exist

    • Cause: You ran terraform init before creating the S3 bucket for the backend, or there is a typo in the bucket name in backend.tf.
    • Solution: Ensure you have manually created the S3 bucket and that the name in backend.tf is correct.

  • Error: creating S3 Bucket (...): BucketAlreadyExists

    • Cause: This happens if your Terraform configuration attempts to create an S3 bucket that already exists. This is common if you try to manage your backend bucket with the same Terraform code that uses it as a backend.
    • Solution: Backend resources should be managed completely separately from your main infrastructure configuration. By creating them manually as we did, you avoid this circular dependency.

9. Recommended .gitignore Configuration

To keep your repository clean and secure, you should prevent Terraform state files and other temporary files from being committed. Your .gitignore file should contain:

# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log

# Exclude all .tfvars files, which are likely to contain sensitive data,
# unless the user explicitly commits them.
*.tfvars
*.tfvars.json

# Ignore override files as they are usually used for local testing
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# TFLint files
.tflint.hcl

# Terraform lock file
.terraform.lock.hcl

10. Side Note: terraform apply --auto-approve

The --auto-approve flag is used to skip the interactive confirmation step during terraform apply. While this is useful for automation (like in a CI/CD pipeline where no user is present to type "yes"), it should be used with caution. In a production environment, you should always run terraform plan and have a human review the changes before applying them. Manual approval is a critical safety check.