Terraform for GenAI Infrastructure

What is Terraform?

Terraform is an Infrastructure as Code (IaC) tool by HashiCorp that allows you to define, provision, and manage cloud infrastructure using declarative configuration files. For GenAI projects, Terraform enables reproducible deployment of compute resources, databases, networking, and AI services across cloud providers.

Core Concepts

Basic Configuration

# main.tf - Provider configuration
terraform {
  required_version = ">= 1.0"
  
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
  }
  
  # Remote state storage (recommended for teams)
  backend "gcs" {
    bucket = "my-terraform-state"
    prefix = "genai-project"
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

# variables.tf
variable "project_id" {
  description = "GCP Project ID"
  type        = string
}

variable "region" {
  description = "GCP Region"
  type        = string
  default     = "us-central1"
}

variable "environment" {
  description = "Environment (dev, staging, prod)"
  type        = string
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

GenAI Infrastructure Examples

# cloud_run.tf
resource "google_cloud_run_v2_service" "genai_api" {
  name     = "genai-api-${var.environment}"
  location = var.region

  template {
    containers {
      image = "gcr.io/${var.project_id}/genai-api:latest"
      
      resources {
        limits = {
          cpu    = "2"
          memory = "4Gi"
        }
      }
      
      env {
        name  = "OPENAI_API_KEY"
        value_source {
          secret_key_ref {
            secret  = google_secret_manager_secret.openai_key.secret_id
            version = "latest"
          }
        }
      }
    }
    
    scaling {
      min_instance_count = var.environment == "prod" ? 1 : 0
      max_instance_count = 10
    }
  }
}

# IAM binding for public access
resource "google_cloud_run_v2_service_iam_member" "public" {
  count    = var.environment == "prod" ? 1 : 0
  location = google_cloud_run_v2_service.genai_api.location
  name     = google_cloud_run_v2_service.genai_api.name
  role     = "roles/run.invoker"
  member   = "allUsers"
}

# vertex_ai.tf - Vector Search for RAG
resource "google_vertex_ai_index" "embeddings" {
  region       = var.region
  display_name = "genai-embeddings-${var.environment}"
  
  metadata {
    contents_delta_uri = "gs://${var.bucket_name}/embeddings/"
    config {
      dimensions                  = 768
      approximate_neighbors_count = 100
      distance_measure_type       = "COSINE_DISTANCE"
      
      algorithm_config {
        tree_ah_config {
          leaf_node_embedding_count    = 1000
          leaf_nodes_to_search_percent = 10
        }
      }
    }
  }
  
  index_update_method = "STREAM_UPDATE"
}

# secrets.tf
resource "google_secret_manager_secret" "openai_key" {
  secret_id = "openai-api-key-${var.environment}"
  
  replication {
    auto {}
  }
}

resource "google_secret_manager_secret" "anthropic_key" {
  secret_id = "anthropic-api-key-${var.environment}"
  
  replication {
    auto {}
  }
}

# Grant Cloud Run access to secrets
resource "google_secret_manager_secret_iam_member" "api_access" {
  for_each  = toset(["openai_key", "anthropic_key"])
  secret_id = google_secret_manager_secret[each.key].secret_id
  role      = "roles/secretmanager.secretAccessor"
  member    = "serviceAccount:${google_service_account.cloud_run.email}"
}

Essential Commands

Project Structure

infrastructure/
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── terraform.tfvars
│   ├── staging/
│   │   └── ...
│   └── prod/
│       └── ...
├── modules/
│   ├── cloud-run/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── vector-db/
│   │   └── ...
│   └── networking/
│       └── ...
├── .terraform.lock.hcl    # Provider lock file
└── README.md

Best Practices

• Remote State: Store state in cloud storage with locking (GCS, S3, Azure Blob)
• Modules: Create reusable modules for common patterns (Cloud Run, databases)
• Environment Separation: Use workspaces or separate directories per environment
• Variables: Never hardcode values; use variables with validation
• Outputs: Export important values (URLs, IPs) for other systems
• Version Pinning: Pin provider versions to avoid breaking changes
• Plan Review: Always review terraform plan before apply
• CI/CD Integration: Run Terraform in GitHub Actions with approval gates

Terraform vs Alternatives

Learn More

Related Topics

• GitHub Actions - CI/CD for Terraform
• Cloud Run - Serverless Containers
• Docker - Containerization
• Secrets Management
• GenAI Deployment Best Practices