Create GCP instances and VPC using Terraform

Despite popularity of serverless and Kubernetes, ordinary virtual machines could be handy for development, bulky workloads, small applications, databases or inherently scalable applications like Kafka. Google provides API enabling developers to quickly spin up machines of many types, ranging from micro instances to multi-processor or GPU enabled boxes. Working with Google APIs directly could be quite cumbersome, but luckily there are tools which are able to do heavy lifting for us. Terraform is one of the most popular tools, allowing to treat infrastructure as a code, and thus providing a higher level of abstraction. Its declarative configuration syntax allows to describe desirable cloud infrastructure in a concise, human readable fashion. The tool is then able to create required cloud resources and saves the recent state of infrastructure (existing infrastructure could be imported as well). Later, upon request, Terraform is able to update the state of your cloud infrastructure smartly, or can destroy it completely. Lets look at how Terraform configuration may look like for GCP compute instances.

VPC

Before creating compute instances we may wish to create our private network. VPCs allow to divide cloud infrastructure into subnets and configure external access using firewall rules. Suppose we wish to create multiple web servers. Then VPC could be configured to allow SSH access, HTTP requests and pings:

// Create VPC
resource "google_compute_network" "vpc" {
 name                    = "${var.project_name}-vpc"
 auto_create_subnetworks = false
}

// Create Subnet
resource "google_compute_subnetwork" "app" {
 name          = "${var.project_name}-app-subnet"
 ip_cidr_range = var.subnet_cidr
 network       = "${var.project_name}-vpc"
 depends_on    = [google_compute_network.vpc]
 region        = var.region
}

// VPC firewall configuration
resource "google_compute_firewall" "firewall" {
  name    = "${var.project_name}-firewall"
  network = google_compute_network.vpc.name

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }

  allow {
    protocol = "tcp"
    ports    = ["80", "8080", "1000-9000"]
  }

  source_tags = ["web"]

  source_ranges = ["0.0.0.0/0"]
}

Terraform, has the notion of variables, which are usually defined in a separate variables.tf file. Default variable values could be overridden using JSON files, environment variables or during module invocation. Variables make configuration flexible and reusable.

Google compute instances

Although Terraform configuration is declarative and human readable, it has quite powerful control flow constructs, for example, supporting arrays and iterations. Thanks to this, multiple web servers could be created (see count property assignment):

resource "google_compute_instance" "webservers" {
  count                     = length(var.webservers)
  name                      = "${var.project_name}-web-${count.index}"
  machine_type              = var.webservers[count.index].type
  zone                      = var.zone

  tags = ["http-server", "https-server", var.webservers[count.index].name]

  service_account {
    scopes = var.scopes_default_web
  }

  metadata_startup_script = "sudo apt-get update; sudo apt-get install -yq build-essential python-pip rsync; pip install flask"

  network_interface {
    subnetwork = google_compute_subnetwork.app.self_link
    access_config {
      // Ephemeral IP
    }
  }

  boot_disk {
    initialize_params {
      image = var.webservers[count.index].image
    }
  }
}

Here variable webservers (referenced as var.webservers) is an array of objects containing server parameters. Providing such an array, one may create as many servers as he wish (even none at all).

Using metadata_startup_script argument, multiple python packages could be installed. A metadata_startup_script (as any Terraform string type argument) could be a multiline string, so, using this method, a fully functional server could be provisioned using, for example, some docker containers. Or we may setup compute instances manually or using tools like Ansible.

Finally to get access to the running servers, we need their IP addresses. To obtain them define outputs:

output "webserver_ip_addresses" {
 value = {
      for webserver in google_compute_instance.webservers:
        webserver.instance_id => webserver.network_interface.0.access_config.0.nat_ip
   }
}

SSH keys

To simplify maintenance, ssh keys could be generated for later upload to GCP:

# first, generate ssh keys
ssh-keygen -t rsa -f ssh-key -C admin

Using Terraform file function, generated file now could be uploaded to GCP (for flexibility, location of the public key file is defined in ssh_pub_key_file variable):

resource "google_compute_project_metadata_item" "ssh-keys" {
  project     = var.gcp_project
  key   = "ssh-keys"
  value = "${var.ssh_user}:${file(var.ssh_pub_key_file)}"
}

When configuration is ready, it is possible to verify, apply or preview it using commands like terraform verify, terraform apply, terraform plan. After infrastructure is finally created, one may test correctness of the setup manually, using, for example, simple Hello World python application:

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_cloud():
   return 'Hello GCP!'

app.run(host='0.0.0.0')

To test our setup, login to server, save the above snippet into app.py file, start web application, and finally test it using IP address of the server from Terraform output:

# login:
ssh -i ssh-key admin@XX.XXX.249.234

# save and start simple Flask server:
python app.py

# test accessibility:
curl http://XX.XXX.249.234:5000

Source code

Thanks to Terraform, in a matter of seconds, one may easily spin up new virtual machines (terraform apply) or destroy unused cloud resources (terraform destroy). Take the code on Github and give it a spin.