CI/CD Pipeline - Handa Uncle Platform

Overview

This document describes the Continuous Integration and Continuous Deployment (CI/CD) pipeline for the HandaUncle Backend application. The pipeline automates building, testing, security scanning, and deploying the application across multiple environments.

The pipeline uses GitHub Actions for automation and Google Cloud Run for deployments with Workload Identity Federation for secure, keyless authentication.

Architecture

Environments

Development

Trigger: Manual (Claim → Deploy → Release)Project: Dev GCP ProjectPurpose: Feature development and testing

Staging

Branch: developProject: Staging GCP ProjectPurpose: Pre-production testing and QA

Production

Branch: mainProject: Production GCP ProjectPurpose: Live production environment

Branch Strategy

Git Flow

Branch Rules

feature/* - Development branches

Created from develop
Manual deployment to Dev environment (Claim → Deploy → Release)
Must pass CI checks before merging
Used for individual feature development

develop - Integration branch

Receives merges from feature branches via PR
Auto-deploys to Staging environment on push
Protected: Requires PR with passing checks
Used for integration testing

main - Production branch

Only accepts PRs from develop
Auto-deploys to Production on push
Protected: Requires PR from develop only
Never push directly to main

Never push directly to main. All production deployments must go through develop first.

Dev Environment Lock System

The Dev environment uses a claim/release system to prevent multiple developers from overwriting each other’s deployments during testing.

How It Works

Workflow Steps

Claim the Environment

Run 🔒 Claim Dev Environment workflow

Blocks other developers from deploying
Shows who has the environment claimed

Deploy Your Branch

Run 🚀 Deploy to Dev workflow

Enter your branch name (e.g., feature/my-feature)
Only works if you own the claim

Test Your Changes

Test as long as needed

Environment stays locked to you
Other developers see “claimed by @you”

Release When Done

Run 🔓 Release Dev Environment workflow

Frees the environment for others
Don’t forget this step!

Emergency Release

If a developer forgets to release and is unavailable:

Run 🔓 Release Dev Environment
Check the Force release checkbox
This will release even if claimed by someone else

Only use force release in emergencies. Always try to contact the developer first.

Workflow Files

CI Pipeline (`ci.yml`)

Overview
Jobs

Triggers:

Push to: main, develop, feature/**
Pull requests to: main, develop

Artifacts:

Vulnerability report (JSON) - 30-day retention

Job	Description	Duration
🔍 Lint & Type Check	ESLint + TypeScript validation	~1-2 min
🧪 Unit Tests	Bun test suite with coverage	~1-3 min
🐳 Docker Build	Multi-stage Docker build with caching	~2-4 min
🔒 Security Scan	Container vulnerability scan	~1-2 min
📦 Dependency Scan	Filesystem/dependency scan	~1 min
🔐 Secret Scan	Secret detection	~1 min
📊 CI Summary	Aggregated status report	~10 sec

Dev Environment Workflows

Workflow	File	Purpose
🔒 Claim Dev Environment	`claim-dev.yml`	Claim exclusive access to dev
🚀 Deploy to Dev	`deploy-dev.yml`	Deploy a branch to dev (requires claim)
🔓 Release Dev Environment	`release-dev.yml`	Release dev for others to use

Deploy to Staging (`deploy-staging.yml`)

Authenticate to GCP

Uses Workload Identity Federation for keyless authentication

Build Docker Image

Build with commit SHA tag for traceability

Push to Artifact Registry

Push to regional Docker registry

Deploy to Cloud Run

Deploy with all secrets injected from Secret Manager

Output URL

Display the deployment URL in workflow summary

Concurrency: Only one staging deployment runs at a time (others queue)

PR Checks (`pr-checks.yml`)

Job	Description
Branch Protection	Blocks direct PRs to main (must come from develop)
PR Summary	Displays PR information and merge path

GCP Infrastructure

Projects

Environment	Purpose
Dev	Feature development and testing
Staging	Pre-production QA
Production	Live environment

Enabled APIs

Cloud Run Admin API

Container deployment and management

Artifact Registry API

Docker image storage

Secret Manager API

Secure secrets storage

IAM API

Identity and access management

Workload Identity Federation

Enables keyless authentication from GitHub Actions to GCP - no service account keys needed!

How it works:

GitHub Actions generates an OIDC token
GCP exchanges it for a short-lived access token
No long-lived credentials stored anywhere

Service Accounts

Purpose	Description
GitHub Actions SA	Used by CI/CD to deploy
Cloud Run SA	Runtime identity for the application

IAM Roles (GitHub Actions SA):

Cloud Run Admin
Artifact Registry Writer
Service Account User
Secret Manager Accessor

Secrets Management

GitHub Repository Secrets

Category	Description
Workload Identity	Provider URLs for each environment
Service Accounts	Email addresses for GCP authentication

GitHub Repository Variables

Variable	Description
`DEV_CLAIMED_BY`	Tracks who has claimed the dev environment

GCP Secret Manager

Core Configuration

Server port, backend secrets, environment settings

Database

Database connection strings, Redis configuration

Authentication

OAuth provider credentials, JWT secrets

AI Services

API keys for various AI providers (LLMs, embeddings)

Vector DB & Memory

Vector database and memory service credentials

Observability

Monitoring and analytics service credentials

File Processing

Document processing and storage credentials

Web Search

Search API credentials

Payments

Payment gateway credentials

Feature Flags

Feature configuration values

Development Workflow

Creating a New Feature

Start from develop

git checkout develop
git pull origin develop

Create feature branch

git checkout -b feature/my-new-feature

Make changes and commit

git add .
git commit -m "feat: Add new feature"

Push your branch

git push origin feature/my-new-feature

Deploying to Dev

Claim the environment

Go to Actions → 🔒 Claim Dev Environment → Run workflow

You now have exclusive access to dev!

Deploy your branch

Go to Actions → 🚀 Deploy to Dev → Run workflow

Enter your branch name (e.g., feature/my-new-feature)
Your code is deploying to Dev!

Test your changes

Use the deployment URL shown in the workflow summary

Release when done

Go to Actions → 🔓 Release Dev Environment → Run workflow

Environment is now available for others!

Promoting to Staging

Create Pull Request

Go to GitHub and create a PR: feature/my-new-feature → develop

Wait for CI checks

All lint, tests, and security scans must pass

Get code review

Have a teammate review your changes

Merge PR

Merging automatically triggers staging deployment!

Promoting to Production

Create PR from develop to main

Go to GitHub and create a PR: develop → main

Wait for CI checks

All checks must pass

Get approval

Production deployments require approval

Merge PR

Merging automatically triggers production deployment!

Monitoring & Debugging

Viewing Workflow Runs

Go to: GitHub Actions
Select the workflow (CI Pipeline, Deploy Dev, etc.)
Click on a specific run to see logs

Viewing Deployment Logs

# View logs for a Cloud Run service
gcloud run services logs read SERVICE_NAME \
  --project=PROJECT_ID \
  --region=REGION

Getting Deployment URLs

# Get the URL for a Cloud Run service
gcloud run services describe SERVICE_NAME \
  --project=PROJECT_ID \
  --region=REGION \
  --format='value(status.url)'

Downloading Security Reports

Go to the CI Pipeline workflow run
Scroll to “Artifacts” section
Download the vulnerability report

Troubleshooting

Dev Environment Claimed by Someone Else

Error: Dev environment is claimed by @usernameSolution:

Contact the developer and ask them to release
If unavailable, use Force release option in the Release workflow

Deploy Blocked - Not Claimed

Error: Dev environment is not claimedSolution: Run 🔒 Claim Dev Environment workflow first, then deploy

Workload Identity Authentication Failed

Error: Unable to get federated tokenSolution: Verify the Workload Identity pool and provider exist in GCP

Secret Access Denied

Error: Permission denied on secretSolution: Ensure the service account has Secret Manager Accessor role

Cloud Run Deployment Failed

Error: Could not find service accountSolution: Verify the Cloud Run service account exists in the project

Docker Build Cache Miss

Symptom: Slow builds despite cachingSolution: GitHub Actions cache has a 10GB limit. Old caches are automatically evicted. This is normal behavior.

Security Considerations

Secrets Protection

All secrets stored in GCP Secret Manager (encrypted at rest)
Workload Identity Federation (no long-lived keys)
Minimal IAM permissions (principle of least privilege)
Security scanning on every build

Branch Protection

main only accepts PRs from develop
CI checks must pass before merge
Code review recommended

Container Security

Multi-stage Docker builds (minimal attack surface)
Non-root user in container
Vulnerability scanning
Secret scanning to prevent credential leaks

Network Security

HTTPS only endpoints
Cloud Run automatic TLS
No exposed ports except 443

Maintenance

Updating Secrets

# Update a secret value in GCP Secret Manager
echo -n "new-value" | gcloud secrets versions add SECRET_NAME \
  --project=PROJECT_ID \
  --data-file=-

# The next deployment will use the new value

Adding New Secrets

Create the secret in GCP

echo -n "value" | gcloud secrets create NEW_SECRET \
  --project=PROJECT_ID \
  --data-file=-

Update the deployment workflow

Add to the secrets section in the workflow file

Cleaning Up Old Images

# List images in Artifact Registry
gcloud artifacts docker images list \
  REGISTRY_PATH/REPOSITORY

# Delete old images
gcloud artifacts docker images delete \
  REGISTRY_PATH/REPOSITORY/IMAGE@DIGEST

Cost Optimization

Cloud Run is pay-per-use and scales to zero when idle, making it very cost-effective for development environments.

Service	Pricing Model
Cloud Run	Pay-per-use, scales to zero
Artifact Registry	Standard storage pricing
Secret Manager	Per-access operation pricing
GitHub Actions	Free for public repos, usage-based for private

References

GitHub Actions

Official GitHub Actions documentation

Cloud Run

Google Cloud Run documentation

Workload Identity

Keyless authentication to GCP

Trivy Scanner

Security vulnerability scanner

​Overview

​Architecture

​Environments

Development

Staging

Production

​Branch Strategy

​Git Flow

​Branch Rules

​Dev Environment Lock System

​How It Works

​Workflow Steps

​Emergency Release

​Workflow Files

​CI Pipeline (ci.yml)

​Dev Environment Workflows

​Deploy to Staging (deploy-staging.yml)

​PR Checks (pr-checks.yml)

​GCP Infrastructure

​Projects

​Enabled APIs

Cloud Run Admin API

Artifact Registry API

Secret Manager API

IAM API

​Workload Identity Federation

​Service Accounts

​Secrets Management

​GitHub Repository Secrets

​GitHub Repository Variables

​GCP Secret Manager

​Development Workflow

​Creating a New Feature

​Deploying to Dev

​Promoting to Staging

​Promoting to Production

​Monitoring & Debugging

​Viewing Workflow Runs

​Viewing Deployment Logs

​Getting Deployment URLs

​Downloading Security Reports

​Troubleshooting

​Security Considerations

Secrets Protection

Branch Protection

Container Security

Network Security

​Maintenance

​Updating Secrets

​Adding New Secrets

​Cleaning Up Old Images

​Cost Optimization

​References

GitHub Actions

Cloud Run

Workload Identity

Trivy Scanner

Overview

Architecture

Environments

Branch Strategy

Git Flow

Branch Rules

Dev Environment Lock System

How It Works

Workflow Steps

Emergency Release

Workflow Files

CI Pipeline (`ci.yml`)

Dev Environment Workflows

Deploy to Staging (`deploy-staging.yml`)

PR Checks (`pr-checks.yml`)

GCP Infrastructure

Projects

Enabled APIs

Workload Identity Federation

Service Accounts

Secrets Management

GitHub Repository Secrets

GitHub Repository Variables

GCP Secret Manager

Development Workflow

Creating a New Feature

Deploying to Dev

Promoting to Staging

Promoting to Production

Monitoring & Debugging

Viewing Workflow Runs

Viewing Deployment Logs

Getting Deployment URLs

Downloading Security Reports

Troubleshooting

Security Considerations

Maintenance

Updating Secrets

Adding New Secrets

Cleaning Up Old Images

Cost Optimization

References