Secrets Management

Securely manage API keys, credentials, and sensitive configuration for GenAI applications

Why Secrets Management Matters

GenAI applications rely on multiple API keys—OpenAI, Anthropic, database credentials, and more. A single exposed key can lead to massive bills, data breaches, or service abuse. Proper secrets management ensures credentials are stored securely, accessed only when needed, and rotated regularly.

⚠️ Real Risk: Exposed API keys on GitHub have cost companies hundreds of thousands of dollars in minutes. Bots continuously scan public repos for secrets.

🤖

LLM API Keys

OpenAI, Anthropic

🗄️

Database

Passwords, URLs

☁️

Cloud Creds

GCP, AWS, Azure

🔑

JWT Secrets

Auth tokens

Secrets DO and DON'T

❌ NEVER DO

• Hardcode secrets in source code
• Commit .env files to git
• Log secrets or API keys
• Pass secrets in URLs
• Share secrets via Slack/email
• Store in frontend JavaScript
• Use same key for dev/prod

✅ ALWAYS DO

• Use secret managers (GCP, AWS, Vault)
• Add .env to .gitignore
• Rotate keys regularly
• Use IAM roles when possible
• Set API key spending limits
• Audit secret access
• Use separate keys per environment

Secret Manager Architecture

Google Secret Manager

Create & Access Secrets (gcloud)

# Create a secret
echo -n "sk-your-openai-key-here" | \
  gcloud secrets create openai-api-key --data-file=-

# Add a new version
echo -n "sk-new-key-value" | \
  gcloud secrets versions add openai-api-key --data-file=-

# Access the latest version
gcloud secrets versions access latest --secret="openai-api-key"

# Grant access to a service account
gcloud secrets add-iam-policy-binding openai-api-key \
  --member="serviceAccount:my-app@my-project.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

Access Secrets in Python

from google.cloud import secretmanager
import os

def get_secret(secret_id: str, version: str = "latest") -> str:
    """Access a secret from Google Secret Manager"""
    client = secretmanager.SecretManagerServiceClient()
    
    project_id = os.environ.get("GOOGLE_CLOUD_PROJECT")
    name = f"projects/{project_id}/secrets/{secret_id}/versions/{version}"
    
    response = client.access_secret_version(name=name)
    return response.payload.data.decode("UTF-8")

# Usage
openai_key = get_secret("openai-api-key")
database_url = get_secret("database-url")

Cloud Run with Secrets

# Deploy with secret as environment variable
gcloud run deploy my-genai-app \
  --image gcr.io/my-project/my-app:latest \
  --set-secrets "OPENAI_API_KEY=openai-api-key:latest" \
  --set-secrets "DATABASE_URL=database-url:latest" \
  --region us-central1

# Or mount as a file
gcloud run deploy my-genai-app \
  --image gcr.io/my-project/my-app:latest \
  --set-secrets "/secrets/openai=openai-api-key:latest" \
  --region us-central1

Secrets with Terraform

# Create the secret
resource "google_secret_manager_secret" "openai_key" {
  secret_id = "openai-api-key"
  
  replication {
    auto {}
  }
}

# Add a version (value from Terraform variable)
resource "google_secret_manager_secret_version" "openai_key_version" {
  secret      = google_secret_manager_secret.openai_key.id
  secret_data = var.openai_api_key  # Pass via TF_VAR_openai_api_key
}

# Grant access to Cloud Run service account
resource "google_secret_manager_secret_iam_member" "openai_access" {
  secret_id = google_secret_manager_secret.openai_key.secret_id
  role      = "roles/secretmanager.secretAccessor"
  member    = "serviceAccount:${google_service_account.app.email}"
}

# Use in Cloud Run
resource "google_cloud_run_v2_service" "app" {
  name     = "genai-app"
  location = "us-central1"

  template {
    containers {
      image = "gcr.io/${var.project_id}/genai-app:latest"
      
      env {
        name = "OPENAI_API_KEY"
        value_source {
          secret_key_ref {
            secret  = google_secret_manager_secret.openai_key.secret_id
            version = "latest"
          }
        }
      }
    }
  }
}

Local Development (.env)

Important: .env files are for local development only. Never commit them to git. Always add .env to .gitignore.

.env file

# .env (NEVER COMMIT!)
OPENAI_API_KEY=sk-dev-xxx
DATABASE_URL=postgresql://...
JWT_SECRET=local-dev-secret

.gitignore

# .gitignore
.env
.env.local
.env.*.local
*.pem
*.key
secrets/

Load .env in Python

from dotenv import load_dotenv
import os

# Load .env file (development only)
load_dotenv()

# Access secrets
openai_key = os.environ.get("OPENAI_API_KEY")
database_url = os.environ.get("DATABASE_URL")

# Fail fast if required secrets are missing
if not openai_key:
    raise ValueError("OPENAI_API_KEY environment variable is required")

Secret Rotation

Best Practice: Rotate secrets regularly (90 days recommended). Use Secret Manager versioning to maintain multiple versions during rotation periods.

1️⃣

Create New Version

Add new key to Secret Manager

2️⃣

Update Application

Deploy to use new version

3️⃣

Revoke Old Key

Disable in provider dashboard

Secret Management Tools

Tool	Provider	Best For	Features
Secret Manager	Google Cloud	GCP-native apps	Versioning, IAM, auto-rotation
Secrets Manager	AWS	AWS-native apps	RDS rotation, Lambda integration
Key Vault	Azure	Azure-native apps	HSM, certificates, keys
HashiCorp Vault	Multi-cloud	Multi-cloud, on-prem	Dynamic secrets, PKI
doppler	SaaS	Startups, teams	Easy UI, sync to platforms

Best Practices Checklist

Use secret managers: Never hardcode or commit secrets
Least privilege: Only grant access to services that need it
Separate environments: Different keys for dev, staging, prod
Set spending limits: Configure budget alerts on API providers
Enable audit logging: Track who accessed which secrets
Rotate regularly: 90-day rotation minimum, immediately if compromised
Use IAM roles: Prefer workload identity over service account keys
Scan for leaks: Use tools like gitleaks, trufflehog in CI/CD

Learn More

Essential Resources

Test Your Knowledge

Score 8/10 or higher to pass