Terraform Remote State Storage and State Locking with AWS S3 and DynamoDB

Step-01: Introduction

Understand Terraform Backends
Understand about Remote State Storage and its advantages
This state is stored by default in a local file named "terraform.tfstate", but it can also be stored remotely, which works better in a team environment.
Create AWS S3 bucket to store terraform.tfstate file and enable backend configurations in terraform settings block
Understand about State Locking and its advantages
Create DynamoDB Table and implement State Locking by enabling the same in Terraform backend configuration

Step-02: Create S3 Bucket

Go to Services -> S3 -> Create Bucket
Bucket name: terraform-on-aws-for-ec2
Region: US-East (N.Virginia)
Bucket settings for Block Public Access: leave to defaults
Bucket Versioning: Enable
Rest all leave to defaults
Click on Create Bucket
Create Folder
Folder Name: dev
Click on Create Folder
Create Folder
Folder Name: dev/project1-vpc
Click on Create Folder

Step-03: Terraform Backend Configuration

Reference Sub-folder: terraform-manifests
Terraform Backend as S3
Add the below listed Terraform backend block in Terrafrom Settings block in main.tf

  # Adding Backend as S3 for Remote State Storage
  backend "s3" {
    bucket = "terraform-on-aws-for-ec2"
    key    = "dev/project1-vpc/terraform.tfstate"
    region = "us-east-1" 

    # Enable during Step-09     
    # For State Locking
    dynamodb_table = "dev-project1-vpc"    
  }

Step-04: Terraform State Locking Introduction

Understand about Terraform State Locking Advantages

Step-05: Add State Locking Feature using DynamoDB Table

Create Dynamo DB Table
Table Name: dev-project1-vpc
Partition key (Primary Key): LockID (Type as String)
Table settings: Use default settings (checked)
Click on Create

Step-06: Execute Terraform Commands

# Initialize Terraform 
terraform init
Observation: 
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

# Terraform Validate
terraform validate

# Review the terraform plan
terraform plan 
Observation: 
1) Below messages displayed at start and end of command
Acquiring state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
2) Verify DynamoDB Table -> Items tab

# Create Resources 
terraform apply -auto-approve

# Verify S3 Bucket for terraform.tfstate file
dev/project1-vpc/terraform.tfstate
Observation: 
1. Finally at this point you should see the terraform.tfstate file in s3 bucket
2. As S3 bucket version is enabled, new versions of `terraform.tfstate` file new versions will be created and tracked if any changes happens to infrastructure using Terraform Configuration Files

Step-07: Destroy Resources

Destroy Resources and Verify Bucket Versioning

# Destroy Resources
terraform destroy -auto-approve

# Clean-Up Files
rm -rf .terraform*
rm -rf terraform.tfstate*  # This step not needed as e are using remote state storage here

Step-08: Little bit theory about Terraform Backends

Understand little bit more about Terraform Backends
Where and when Terraform Backends are used ?
What Terraform backends do ?
How many types of Terraform backends exists as on today ?

References

Sample Output - During Remote State Storage Migration**

Kalyans-MacBook-Pro:project-1-networking kdaida$ terraform init
Initializing modules...

Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "s3" backend. No existing state was found in the newly
  configured "s3" backend. Do you want to copy this state to the new "s3"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes


Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v3.34.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Kalyans-MacBook-Pro:project-1-networking kdaida$