AWS EKS Ingress Name based Virtal Host Routing

Step-01: Introduction¶

Implement Host Header routing using Ingress
We can also call it has name based virtual host routing

Step-02: Review Ingress Manifests for Host Header Routing¶

Project Folder: 04-kube-manifests-ingress-nvhr

File Name: 04-ALB-Ingress-HostHeader-Routing.yml

# Annotations Reference: https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-namedbasedvhost-demo
  annotations:
    # Load Balancer Name
    alb.ingress.kubernetes.io/load-balancer-name: namedbasedvhost-ingress
    # Ingress Core Settings
    #kubernetes.io/ingress.class: "alb" (OLD INGRESS CLASS NOTATION - STILL WORKS BUT RECOMMENDED TO USE IngressClass Resource)
    alb.ingress.kubernetes.io/scheme: internet-facing
    # Health Check Settings
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP 
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    #Important Note:  Need to add health check path annotations in service level if we are planning to use multiple targets in a load balancer    
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '5'
    alb.ingress.kubernetes.io/success-codes: '200'
    alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'   
    ## SSL Settings
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:180789647333:certificate/0d86500a-08b3-4f17-8fb4-f09532ba0522
    #alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-1-2017-01 #Optional (Picks default if not used)    
    # SSL Redirect Setting
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    # External DNS - For creating a Record Set in Route53
    external-dns.alpha.kubernetes.io/hostname: default101.stacksimplify.com 
spec:
  ingressClassName: my-aws-ingress-class   # Ingress Class                  
  defaultBackend:
    service:
      name: app3-nginx-nodeport-service
      port:
        number: 80     
  rules:
    - host: app101.stacksimplify.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app1-nginx-nodeport-service
                port: 
                  number: 80
    - host: app201.stacksimplify.com
      http:
        paths:                  
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app2-nginx-nodeport-service
                port: 
                  number: 80

# Important Note-1: In path based routing order is very important, if we are going to use  "/*", try to use it at the end of all rules.                                        

# 1. If  "spec.ingressClassName: my-aws-ingress-class" not specified, will reference default ingress class on this kubernetes cluster
# 2. Default Ingress class is nothing but for which ingress class we have the annotation `ingressclass.kubernetes.io/is-default-class: "true"`

Step-03: Deploy all Application Kubernetes Manifests and Verify¶

Project Folder: 04-kube-manifests-ingress-nvhr

# Change Directory 
cd 33-EKS-Ingress-NameBasedVirtualHost-Routing

# Deploy kube-manifests
kubectl apply -f 04-kube-manifests-ingress-nvhr/

# Verify Ingress Resource
kubectl get ingress

# Verify Apps
kubectl get deploy
kubectl get pods

# Verify NodePort Services
kubectl get svc

Verify Load Balancer & Target Groups¶

Load Balancer - Listeneres (Verify both 80 & 443)
Load Balancer - Rules (Verify both 80 & 443 listeners)
Target Groups - Group Details (Verify Health check path)
Target Groups - Targets (Verify all 3 targets are healthy)

Verify External DNS Log¶

# Verify External DNS logs
kubectl logs -f $(kubectl get po | egrep -o 'external-dns[A-Za-z0-9-]+')

Verify Route53¶

Go to Services -> Route53
You should see Record Sets added for
default101.stacksimplify.com
app101.stacksimplify.com
app201.stacksimplify.com

Step-04: Access Application using newly registered DNS Name¶

Perform nslookup tests before accessing Application¶

Test if our new DNS entries registered and resolving to an IP Address

# nslookup commands
nslookup default101.stacksimplify.com
nslookup app101.stacksimplify.com
nslookup app201.stacksimplify.com

Positive Case: Access Application using DNS domain¶

# Access App1
http://app101.stacksimplify.com/app1/index.html

# Access App2
http://app201.stacksimplify.com/app2/index.html

# Access Default App (App3)
http://default101.stacksimplify.com

Negative Case: Access Application using DNS domain¶

# Access App2 using App1 DNS Domain
http://app101.stacksimplify.com/app2/index.html  -- SHOULD FAIL

# Access App1 using App2 DNS Domain
http://app201.stacksimplify.com/app1/index.html  -- SHOULD FAIL

# Access App1 and App2 using Default Domain
http://default101.stacksimplify.com/app1/index.html -- SHOULD FAIL
http://default101.stacksimplify.com/app2/index.html -- SHOULD FAIL

Step-05: Clean Up¶

# Delete Manifests
kubectl delete -f 04-kube-manifests-ingress-nvhr/

## Verify Route53 Record Set to ensure our DNS records got deleted
- Go to Route53 -> Hosted Zones -> Records 
- The below records should be deleted automatically
  - default101.stacksimplify.com
  - app101.stacksimplify.com
  - app201.stacksimplify.com

Step-06: Review Terraform Manifests¶

Project Folder: 05-ingress-nvhr-terraform-manifests
c1-versions.tf
c2-remote-state-datasource.tf
c3-providers.tf
c4-kubernetes-app1-deployment.tf
c5-kubernetes-app2-deployment.tf
c6-kubernetes-app3-deployment.tf
c7-kubernetes-app1-nodeport-service.tf
c8-kubernetes-app2-nodeport-service.tf
c9-kubernetes-app3-nodeport-service.tf
c11-acm-certificate.tf

Step-07: c10-kubernetes-ingress-service.tf¶

Project Folder: 05-ingress-nvhr-terraform-manifests

Update name based virtual host routing related changes.

# Kubernetes Service Manifest (Type: Load Balancer)
resource "kubernetes_ingress_v1" "ingress" {
  metadata {
    name = "ingress-namedbasedvhost-demo"
    annotations = {
      # Load Balancer Name
      "alb.ingress.kubernetes.io/load-balancer-name" = "ingress-namedbasedvhost-demo"
      # Ingress Core Settings
      "alb.ingress.kubernetes.io/scheme" = "internet-facing"
      # Health Check Settings
      "alb.ingress.kubernetes.io/healthcheck-protocol" =  "HTTP"
      "alb.ingress.kubernetes.io/healthcheck-port" = "traffic-port"
      #Important Note:  Need to add health check path annotations in service level if we are planning to use multiple targets in a load balancer    
      "alb.ingress.kubernetes.io/healthcheck-interval-seconds" = 15
      "alb.ingress.kubernetes.io/healthcheck-timeout-seconds" = 5
      "alb.ingress.kubernetes.io/success-codes" = 200
      "alb.ingress.kubernetes.io/healthy-threshold-count" = 2
      "alb.ingress.kubernetes.io/unhealthy-threshold-count" = 2
      ## SSL Settings
      # Option-1: Using Terraform jsonencode Function
      "alb.ingress.kubernetes.io/listen-ports" = jsonencode([{"HTTPS" = 443}, {"HTTP" = 80}])
      # Option-2: Using Terraform File Function      
      #"alb.ingress.kubernetes.io/listen-ports" = file("${path.module}/listen-ports/listen-ports.json")
      "alb.ingress.kubernetes.io/certificate-arn" =  "${aws_acm_certificate.acm_cert.arn}"
      #"alb.ingress.kubernetes.io/ssl-policy" = "ELBSecurityPolicy-TLS-1-1-2017-01" #Optional (Picks default if not used)    
      # SSL Redirect Setting
      "alb.ingress.kubernetes.io/ssl-redirect" = 443
    # External DNS - For creating a Record Set in Route53
      "external-dns.alpha.kubernetes.io/hostname" = "tfdefault101.stacksimplify.com"
    }    
  }

  spec {
    ingress_class_name = "my-aws-ingress-class" # Ingress Class        
    # Default Rule: Route requests to App3 if the DNS is "tfdefault101.stacksimplify.com"        
    default_backend {
      service {
        name = kubernetes_service_v1.myapp3_np_service.metadata[0].name
        port {
          number = 80
        }
      }
    }

    # Rule-1: Route requests to App1 if the DNS is "tfapp101.stacksimplify.com"
    rule {
      host = "tfapp101.stacksimplify.com"      
      http {
        path {
          backend {
            service {
              name = kubernetes_service_v1.myapp1_np_service.metadata[0].name
              port {
                number = 80
              }
            }
          }
          path = "/"
          path_type = "Prefix"
        }
      }        
    }

    # Rule-2: Route requests to App2 if the DNS is "tfapp102.stacksimplify.com"
    rule {
      host = "tfapp201.stacksimplify.com"      
      http {
        path {
          backend {
            service {
              name = kubernetes_service_v1.myapp2_np_service.metadata[0].name
              port {
                number = 80
              }
            }
          }
          path = "/"
          path_type = "Prefix"
        }
      }        
    }
  }
}

Step-08: Execute Terraform Commands¶

# Change Directory 
cd 05-ingress-nvhr-terraform-manifests

# Terraform Initialize
terraform init

# Terraform Validate
terraform validate

# Terraform Plan
terraform plan

# Terraform Apply
terraform apply -auto-approve

Step-09: Verify Ingress Service¶

# Verify Ingress Resource
kubectl get ingress

# Verify Apps
kubectl get deploy
kubectl get pods

# Verify NodePort Services
kubectl get svc

Step-10: Verify External DNS Log¶

# Verify External DNS logs
kubectl logs -f $(kubectl get po | egrep -o 'external-dns[A-Za-z0-9-]+')

Step-11: Verify Route53¶

Go to Services -> Route53
You should see Record Sets added for
tfapp101.stacksimplify.com
tfapp201.stacksimplify.com
tfdefault101.stacksimplify.com

Step-12: Access Application using newly registered DNS Name¶

Perform nslookup tests before accessing Application

Test if our new DNS entries registered and resolving to an IP Address

# nslookup commands
nslookup tfapp101.stacksimplify.com
nslookup tfapp201.stacksimplify.com
nslookup tfdefault101.stacksimplify.com

Step-13: Access Application¶

# Access App1
http://tfapp101.stacksimplify.com/app1/index.html

# Access App2
http://tfapp201.stacksimplify.com/app2/index.html

# Access Default App (App3)
http://tfdefault101.stacksimplify.com

Step-14: Clean-Up Ingress¶

# Change Directory 
cd 05-ingress-nvhr-terraform-manifests

# Terraform Destroy
terraform apply -destroy -auto-approve
rm -rf .terraform*

Step-15: Don't Clean-Up LBC Controller & EKS Cluster¶

Dont destroy the Terraform Projects in below two folders
Terraform Project Folder: 01-ekscluster-terraform-manifests
Terraform Project Folder: 02-lbc-install-terraform-manifests
Terraform Project Folder: 03-externaldns-install-terraform-manifests
We are going to use them for all upcoming Usecases.
Destroy Resorces Order
03-externaldns-install-terraform-manifests
02-lbc-install-terraform-manifests

01-ekscluster-terraform-manifests

##############################################################
## Destroy External DNS
# Change Directroy
cd 03-externaldns-install-terraform-manifests

# Terraform Destroy
terraform init
terraform apply -destroy -auto-approve
##############################################################
## Destroy  LBC
# Change Directroy
cd 02-lbc-install-terraform-manifests

# Terraform Destroy
terraform init
terraform apply -destroy -auto-approve
##############################################################
## Destroy EKS Cluster
# Change Directroy
cd 01-ekscluster-terraform-manifests

# Terraform Destroy
terraform init
terraform apply -destroy -auto-approve
##############################################################

References¶

https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/alb-ingress.md
https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/aws.md