Creating a Google Kubernetes Engine (GKE) Cluster with Terraform in a Custom VPC: A Comprehensive Guide

ByNakamura

In today's cloud-native landscape, orchestrating containerized applications efficiently is paramount. Google Kubernetes Engine (GKE), combined with Terraform for infrastructure as code and a custom Virtual Private Cloud (VPC), offers a robust solution for deploying scalable and secure Kubernetes clusters. This comprehensive guide will walk you through the process of creating a GKE cluster within a custom VPC using Terraform, providing insights and best practices along the way.

Understanding the Power of GKE, Custom VPCs, and Terraform

Before delving into the technical implementation, it's crucial to understand why this combination is particularly powerful for modern cloud architectures.

The Advantages of Google Kubernetes Engine

GKE, Google Cloud Platform's managed Kubernetes service, has become a go-to solution for many organizations deploying containerized applications. It abstracts away much of the complexity involved in managing a Kubernetes control plane, allowing developers and operations teams to focus on application deployment and scaling rather than infrastructure management.

According to the Cloud Native Computing Foundation's 2021 survey, Kubernetes adoption has skyrocketed, with 96% of organizations either using or evaluating Kubernetes. GKE's market share within this ecosystem has grown significantly, thanks to its robust feature set and integration with Google Cloud's broader suite of services.

The Importance of Custom VPCs

While GKE provides a managed Kubernetes environment, deploying your cluster within a custom Virtual Private Cloud offers several key benefits:

  1. Enhanced Network Isolation: Custom VPCs allow you to define your network topology, creating subnets and firewall rules that align with your security requirements.

  2. Improved Performance: By carefully designing your VPC, you can optimize network latency and throughput for your specific use case.

  3. Compliance and Governance: Many regulatory frameworks require strict network segmentation, which is more easily achieved with custom VPCs.

Terraform: Infrastructure as Code

Hashicorp's Terraform has emerged as a leading infrastructure as code (IaC) tool, with over 100 million downloads as of 2021. Its declarative approach to defining infrastructure allows teams to version control their environment configurations, leading to more consistent and reproducible deployments.

By using Terraform to define both our VPC and GKE cluster, we gain several advantages:

  1. Version Control: Infrastructure changes can be tracked and reviewed just like application code.
  2. Repeatability: The same configuration can be used to create identical environments across different projects or regions.
  3. Collaboration: Team members can work together on infrastructure definitions, leveraging familiar git workflows.

Prerequisites and Project Structure

Before we begin, ensure you have the following prerequisites in place:

  • A Google Cloud Platform (GCP) account with billing enabled
  • Google Cloud SDK installed and initialized (version 369.0.0 or later)
  • Terraform installed (version 1.0.0 or later)
  • Basic familiarity with Kubernetes concepts, GCP, and Terraform syntax

Our project will be structured as follows to promote modularity and reusability:

my-gke-tf/
├── modules/
│   ├── vpc/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   └── gke/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── main.tf
├── variables.tf
├── outputs.tf
└── terraform.tfvars

This structure allows us to encapsulate the VPC and GKE configurations in separate modules, promoting code reuse and making our main configuration more readable.

Crafting the VPC Module

Let's start by creating our custom VPC module. This will define the network and subnet where our GKE cluster will reside.

In modules/vpc/main.tf, we'll define our VPC and subnet resources:

resource "google_compute_network" "vpc" {
  name                    = var.vpc_name
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "subnet" {
  name          = var.subnet_name
  ip_cidr_range = var.subnet_cidr
  region        = var.region
  network       = google_compute_network.vpc.self_link

  private_ip_google_access = true

  secondary_ip_range {
    range_name    = "pod-range"
    ip_cidr_range = var.pod_cidr
  }

  secondary_ip_range {
    range_name    = "service-range"
    ip_cidr_range = var.service_cidr
  }
}

Note that we've added secondary IP ranges for pods and services, which is a best practice for GKE deployments. We've also enabled private Google access, allowing VMs in the subnet to reach Google APIs and services without public IP addresses.

In modules/vpc/variables.tf, we'll define our input variables:

variable "vpc_name" {
  description = "Name of the VPC"
  type        = string
}

variable "subnet_name" {
  description = "Name of the subnet"
  type        = string
}

variable "subnet_cidr" {
  description = "CIDR range for the subnet"
  type        = string
}

variable "pod_cidr" {
  description = "CIDR range for pods"
  type        = string
}

variable "service_cidr" {
  description = "CIDR range for services"
  type        = string
}

variable "region" {
  description = "Region for the subnet"
  type        = string
}

Finally, in modules/vpc/outputs.tf, we'll define our outputs:

output "vpc_self_link" {
  value       = google_compute_network.vpc.self_link
  description = "The self link of the VPC"
}

output "subnet_self_link" {
  value       = google_compute_subnetwork.subnet.self_link
  description = "The self link of the subnet"
}

output "pod_range_name" {
  value       = google_compute_subnetwork.subnet.secondary_ip_range[0].range_name
  description = "The name of the pod IP range"
}

output "service_range_name" {
  value       = google_compute_subnetwork.subnet.secondary_ip_range[1].range_name
  description = "The name of the service IP range"
}

Designing the GKE Module

With our VPC module in place, let's create our GKE module. This will define our Kubernetes cluster and its associated node pool.

In modules/gke/main.tf:

resource "google_container_cluster" "primary" {
  name     = var.cluster_name
  location = var.region

  # We can't create a cluster with no node pool defined, but we want to only use
  # separately managed node pools. So we create the smallest possible default
  # node pool and immediately delete it.
  remove_default_node_pool = true
  initial_node_count       = 1

  network    = var.vpc_self_link
  subnetwork = var.subnet_self_link

  ip_allocation_policy {
    cluster_secondary_range_name  = var.pod_range_name
    services_secondary_range_name = var.service_range_name
  }

  private_cluster_config {
    enable_private_nodes    = true
    enable_private_endpoint = false
    master_ipv4_cidr_block  = var.master_ipv4_cidr_block
  }

  master_authorized_networks_config {
    cidr_blocks {
      cidr_block   = var.authorized_ipv4_cidr_block
      display_name = "External Control Plane access"
    }
  }

  addons_config {
    http_load_balancing {
      disabled = false
    }
    horizontal_pod_autoscaling {
      disabled = false
    }
  }

  release_channel {
    channel = "REGULAR"
  }
}

resource "google_container_node_pool" "primary_nodes" {
  name       = "${var.cluster_name}-node-pool"
  location   = var.region
  cluster    = google_container_cluster.primary.name
  node_count = var.node_count

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only"
    ]

    labels = {
      env = var.project_id
    }

    machine_type = var.machine_type
    tags         = ["gke-node", "${var.project_id}-gke"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }

  autoscaling {
    min_node_count = 1
    max_node_count = 5
  }

  management {
    auto_repair  = true
    auto_upgrade = true
  }
}

This configuration creates a private GKE cluster with a separately managed node pool. We've enabled several important features:

  1. Private nodes with a public control plane
  2. Master authorized networks for added security
  3. HTTP load balancing and horizontal pod autoscaling
  4. Regular release channel for timely updates
  5. Node pool autoscaling and auto-repair/auto-upgrade

In modules/gke/variables.tf, we'll define our input variables:

variable "project_id" {
  description = "The project ID to host the cluster in"
  type        = string
}

variable "cluster_name" {
  description = "The name for the GKE cluster"
  type        = string
}

variable "region" {
  description = "The region to host the cluster in"
  type        = string
}

variable "vpc_self_link" {
  description = "The self link of the VPC"
  type        = string
}

variable "subnet_self_link" {
  description = "The self link of the subnet"
  type        = string
}

variable "pod_range_name" {
  description = "The name of the pod IP range"
  type        = string
}

variable "service_range_name" {
  description = "The name of the service IP range"
  type        = string
}

variable "node_count" {
  description = "Number of nodes in the cluster"
  type        = number
}

variable "machine_type" {
  description = "The machine type for the nodes"
  type        = string
}

variable "master_ipv4_cidr_block" {
  description = "The IP range for the master network"
  type        = string
}

variable "authorized_ipv4_cidr_block" {
  description = "The IP range authorized to access the master network"
  type        = string
}

And in modules/gke/outputs.tf:

output "kubernetes_cluster_name" {
  value       = google_container_cluster.primary.name
  description = "GKE Cluster Name"
}

output "kubernetes_cluster_host" {
  value       = google_container_cluster.primary.endpoint
  description = "GKE Cluster Host"
}

Bringing It All Together

Now that we have our modules defined, let's tie everything together in our root main.tf:

provider "google" {
  project = var.project_id
  region  = var.region
}

module "vpc" {
  source       = "./modules/vpc"
  vpc_name     = "${var.project_id}-vpc"
  subnet_name  = "${var.project_id}-subnet"
  subnet_cidr  = var.subnet_cidr
  pod_cidr     = var.pod_cidr
  service_cidr = var.service_cidr
  region       = var.region
}

module "gke" {
  source                     = "./modules/gke"
  project_id                 = var.project_id
  cluster_name               = "${var.project_id}-gke"
  region                     = var.region
  vpc_self_link              = module.vpc.vpc_self_link
  subnet_self_link           = module.vpc.subnet_self_link
  pod_range_name             = module.vpc.pod_range_name
  service_range_name         = module.vpc.service_range_name
  node_count                 = var.node_count
  machine_type               = var.machine_type
  master_ipv4_cidr_block     = var.master_ipv4_cidr_block
  authorized_ipv4_cidr_block = var.authorized_ipv4_cidr_block
}

In our root variables.tf, we'll define all the variables we need:

variable "project_id" {
  description = "The project ID to host the cluster in"
}

variable "region" {
  description = "The region to host the cluster in"
}

variable "subnet_cidr" {
  description = "The CIDR for the subnet"
}

variable "pod_cidr" {
  description = "The CIDR for pods"
}

variable "service_cidr" {
  description = "The CIDR for services"
}

variable "node_count" {
  description = "Number of nodes in the cluster"
}

variable "machine_type" {
  description = "The machine type for the nodes"
}

variable "master_ipv4_cidr_block" {
  description = "The IP range for the master network"
}

variable "authorized_ipv4_cidr_block" {
  description = "The IP range authorized to access the master network"
}

Finally, create a terraform.tfvars file to set your specific values:

project_id                 = "your-project-id"
region                     = "us-central1"
subnet_cidr                = "10.0.0.0/24"
pod_cidr                   = "10.1.0.0/16"
service_cidr               = "10.2.0.0/16"
node_count                 = 3
machine_type               = "n1-standard-2"
master_ipv4_cidr_block     = "172.16.0.0/28"
authorized_ipv4_cidr_block = "your-ip-range/32"

Deploying and Connecting to Your GKE Cluster

With our Terraform configuration complete, we can now deploy our infrastructure:

  1. Initialize Terraform:

    terraform init

  2. Plan the changes:

    terraform plan

  3. Apply the changes:

    terraform apply

After the apply completes successfully, you'll have a GKE cluster running in your custom VPC!

To connect to your new GKE cluster, use the gcloud command:

gcloud container clusters get-credentials $(terraform output -raw kubernetes_cluster_name) --region $(terraform output -raw region)

This will configure kubectl to use your new cluster.

Best Practices and Advanced Considerations

While this setup provides a solid foundation for running GKE in a custom VPC, there are several advanced topics and best practices to consider as you scale and secure your infrastructure:

  1. Workload Identity: Enable and configure Workload Identity to provide Google Cloud IAM permissions to your Kubernetes workloads securely.

  2. Binary Authorization: Implement Binary Authorization to ensure only trusted container images are deployed in your cluster.

  3. Network Policy: Use Kubernetes Network Policies to control traffic flow between pods for enhanced security.

  4. Cluster Monitoring: Set up Cloud Monitoring and Logging for comprehensive visibility into your cluster's performance and health.

  5. Cost Optimization: Implement node auto-provisioning and cluster autoscaler to optimize resource utilization and costs.

  6. Anthos: Consider adopting Anthos for a consistent Kubernetes experience across on-premises and multiple clouds.

Conclusion

In this comprehensive guide, we've walked through the process of creating a Google Kubernetes Engine cluster within a custom VPC using Terraform. This approach combines the power of GKE's managed Kubernetes offering with the network isolation and control provided by a custom VPC, all defined as infrastructure as code for easy versioning and replication.

By leveraging Terraform modules, we've created a scalable and maintainable infrastructure definition that can be easily adapted to different environments or projects. As containerized applications continue to dominate the cloud-native landscape, having a solid foundation for deploying and managing Kubernetes clusters becomes increasingly crucial.

Remember, while this setup provides a strong starting point, every organization's needs are unique. Always consider your specific requirements for security, compliance, and performance when

Similar Posts