Deployment Strategies Explained: Choosing the Right Approach for Your Production Systems

Why This Matters

You have a new version of your application. It is tested. The CI pipeline is green. Now comes the hard part: getting it into production without breaking things for your users.

The deployment strategy you choose determines:

• Blast radius: How many users are affected if something goes wrong?
• Rollback speed: How quickly can you recover from a bad deployment?
• Infrastructure cost: Do you need to run duplicate environments?
• Downtime: Will users experience service interruptions?

Let me walk through the most common strategies, explain how they work, and show you when to use each one.

1. Recreate Deployment

This is the simplest strategy: stop the old version, start the new version.

How It Works

1. Shut down all instances running version A
2. Wait for them to fully terminate
3. Start new instances running version B
4. Route traffic to the new version

Example Scenario

You have an internal admin dashboard that processes reports overnight. The system runs from 10 PM to 6 AM. You deploy a new version at 9 PM when no one is using it.

# Stop old version
kubectl delete deployment admin-dashboard
 
# Deploy new version
kubectl apply -f dashboard-v2.yaml
 
# Wait for pods to be ready
kubectl rollout status deployment/admin-dashboard

When to Use This

• Internal tools with scheduled downtime windows
• Non-critical services where a few minutes of downtime is acceptable
• Development and staging environments
• Systems with no active user sessions to preserve

Pros

• Simple to understand and implement
• No extra infrastructure needed
• Clean state transition (no version mixing)
• Zero cost for duplicate environments

Cons

• Downtime: Users cannot access the service during deployment
• High risk: If the new version has issues, users experience immediate impact
• Slow rollback: You need to redeploy the old version to recover

Real World Example

A fintech company deploys their batch processing system using recreate deployments. The system processes transactions from 2 AM to 5 AM. They deploy new versions at 1 AM during their maintenance window. If a deployment fails, they have an hour to rollback before the batch window opens.

2. Rolling Deployment

Replace instances gradually, one at a time or in small batches.

How It Works

1. Take one instance out of the load balancer
2. Deploy the new version to that instance
3. Wait for health checks to pass
4. Add it back to the load balancer
5. Repeat for the next instance

Example Scenario

You run an API with 10 servers behind a load balancer. You configure a rolling deployment to replace 2 servers at a time. At any point, 8 servers handle traffic while 2 are being upgraded.

# Kubernetes rolling update configuration
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 2  # Max instances down during update
      maxSurge: 2        # Max extra instances during update
  template:
    spec:
      containers:
      - name: api
        image: api:v2.0.0

When to Use This

• Stateless applications that can run mixed versions temporarily
• Systems where gradual rollout is acceptable
• When you want zero downtime but cannot afford duplicate infrastructure
• APIs and web services with horizontal scaling

Pros

• No downtime: Some instances always serve traffic
• Resource efficient: No need to double your infrastructure
• Gradual rollout: Issues appear slowly, giving you time to react
• Built-in to most platforms: Kubernetes, ECS, and Auto Scaling Groups support this natively

Cons

• Version mixing: Old and new versions run simultaneously during deployment
• Slower rollback: Must roll back one instance at a time
• Session affinity issues: Users might hit different versions on subsequent requests
• Database migration challenges: Both versions must work with the same database schema

Real World Example

An e-commerce platform uses rolling deployments for their product API. They have 50 instances and configure maxUnavailable=5. During a deployment, 45 instances always serve traffic. If a new version causes errors, they catch it when only 5 instances have upgraded and halt the rollout.

3. Blue-Green Deployment

Run two identical environments. Switch traffic from one to the other instantly.

How It Works

1. Blue environment runs the current production version
2. Deploy the new version to the green environment
3. Test the green environment thoroughly
4. Switch the load balancer to point to green
5. Keep blue running as a hot standby for quick rollback

Example Scenario

You operate a payment processing system. You cannot afford version mixing or gradual rollouts because transactions must be consistent. You deploy the new version to a completely separate environment, verify it works, then flip a switch to route all traffic to it.

# Deploy to green environment
aws ecs update-service --cluster prod \
  --service payments-green \
  --desired-count 10
 
# Wait for healthy state
aws ecs wait services-stable --cluster prod \
  --services payments-green
 
# Switch traffic
aws elbv2 modify-listener --listener-arn $LISTENER_ARN \
  --default-actions Type=forward,TargetGroupArn=$GREEN_TARGET_GROUP
 
# Scale down blue (but don't terminate yet)
aws ecs update-service --cluster prod \
  --service payments-blue \
  --desired-count 1

When to Use This

• Mission-critical systems where instant rollback is required
• Applications that cannot tolerate version mixing
• When you need comprehensive testing in a production-like environment before switching traffic
• Systems with complex state that would break during gradual rollouts

Pros

• Instant rollback: Just point the load balancer back to blue
• Zero downtime: Traffic switches instantly
• No version mixing: Only one version serves traffic at a time
• Production testing: Fully test green before users see it

Cons

• Cost: Running two full environments doubles your infrastructure during deployment
• Database synchronization: Both environments often need to share the same database
• Stateful services: Harder to implement for systems with persistent connections
• Waste during idle time: Green environment sits unused between deployments

Real World Example

A healthcare platform uses blue-green deployments for their patient records system. They cannot risk version mixing due to HIPAA compliance requirements. They deploy to green, run automated tests including real anonymized patient data queries, then switch traffic only after confirming zero errors. If issues appear, they switch back to blue in less than 30 seconds.

4. Canary Deployment

Release to a small subset of users first, then gradually increase traffic.

How It Works

1. Deploy the new version alongside the old version
2. Route 5% of traffic to the new version
3. Monitor error rates, latency, and business metrics
4. If metrics look good, increase to 25%, then 50%, then 100%
5. If metrics degrade, route all traffic back to the old version

Example Scenario

You launch a new recommendation algorithm for your video streaming platform. You route 5% of users to the new algorithm and measure watch time, completion rate, and user satisfaction. If the metrics improve, you increase the percentage. If they worsen, you abort the rollout.

# Simple canary configuration
def route_traffic(user_id):
    # Route 5% of users to canary
    if hash(user_id) % 100 < 5:
        return "canary-v2.0.0"
    else:
        return "stable-v1.9.5"
 
# Monitor metrics
canary_error_rate = get_error_rate("canary-v2.0.0")
stable_error_rate = get_error_rate("stable-v1.9.5")
 
if canary_error_rate > stable_error_rate * 1.5:
    # Abort: canary has 50% more errors
    set_canary_traffic_percentage(0)
else:
    # Success: increase traffic
    increase_canary_traffic()

When to Use This

• New features with uncertain production behavior
• Performance optimizations where metrics must be compared
• Systems where user impact can be measured in real time
• When you want to validate production behavior before full rollout

Pros

• Limited blast radius: Only a small percentage of users see issues
• Real production validation: Test with real users and real traffic
• Gradual confidence building: Increase traffic as you gain confidence
• Data-driven decisions: Compare metrics between versions

Cons

• Complex infrastructure: Requires traffic splitting and advanced monitoring
• Slower rollout: Full deployment takes hours or days instead of minutes
• Version mixing: Both versions run simultaneously
• Requires good metrics: You need robust observability to make decisions

Real World Example

Netflix uses canary deployments extensively. When they roll out a new video encoding algorithm, they route 1% of streams to the new encoder. They monitor buffering rates, playback failures, and encoding cost. If the new algorithm reduces buffering by 10% and costs 5% less, they increase traffic to 10%, then 50%, then 100% over three days.

Comparing the Strategies

Choosing the Right Strategy

Use Recreate When:

• You have a maintenance window and users expect downtime
• The system is non-critical (internal tools, batch jobs)
• Simplicity is more important than availability

Use Rolling When:

• You need zero downtime with minimal infrastructure cost
• Your application is stateless or can handle version mixing
• You have good health checks and monitoring

Use Blue-Green When:

• Instant rollback is a hard requirement
• You cannot tolerate version mixing
• You can afford the infrastructure cost
• The system is mission-critical

Use Canary When:

• You need to validate changes with real users before full rollout
• You have robust monitoring and can measure user impact
• The change involves algorithm updates, performance optimizations, or UI changes
• You want to minimize risk for uncertain deployments

Hybrid Approaches

Many production systems combine multiple strategies:

Example 1: Blue-Green + Canary

Deploy the new version to the green environment, then gradually shift traffic from blue to green using canary percentages. This gives you both instant rollback and controlled exposure.

Example 2: Rolling + Canary

Perform a rolling update but only route 10% of traffic to the newly updated instances. This limits blast radius during the gradual rollout.

Health Checks Are Non-Negotiable

Every deployment strategy depends on accurate health checks. A bad health check will cause your deployment to fail or, worse, deploy a broken version.

# Good health check configuration
livenessProbe:
  httpGet:
    path: /health/live
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10
  failureThreshold: 3
 
readinessProbe:
  httpGet:
    path: /health/ready
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5
  failureThreshold: 2

Your health check should verify:

• Application started successfully
• Database connections are established
• Required dependencies are reachable
• Critical configuration is loaded

Rollback Strategies

No deployment is complete without a rollback plan.

Rolling Rollback

# Kubernetes rollback
kubectl rollout undo deployment/api-service
 
# Rollback to specific version
kubectl rollout undo deployment/api-service --to-revision=3

Blue-Green Rollback

# Just switch the load balancer back
aws elbv2 modify-listener --listener-arn $LISTENER_ARN \
  --default-actions Type=forward,TargetGroupArn=$BLUE_TARGET_GROUP

Canary Rollback

# Set canary traffic to 0%
kubectl patch service api-service \
  -p '{"spec":{"selector":{"version":"stable"}}}'

Common Pitfalls

1. Database Schema Migrations

If you change the database schema, both the old and new versions must work with the same schema during deployment. Use backward-compatible migrations:

Bad:

-- This breaks the old version immediately
ALTER TABLE users DROP COLUMN phone;

Good:

-- Step 1: Deploy code that doesn't use phone
-- Step 2: After deployment, remove column
ALTER TABLE users DROP COLUMN phone;

2. Configuration Changes

Avoid tightly coupling deployments to configuration changes. Use feature flags to decouple code deploys from feature releases.

3. Insufficient Monitoring

You cannot roll back what you cannot measure. Ensure you have:

• Error rate metrics
• Latency percentiles (p50, p95, p99)
• Business metrics (orders, signups, conversions)
• Alerts configured for anomalies

Tools and Platforms

Kubernetes

• Native support for rolling updates
• Can implement blue-green with services and labels
• Requires additional tooling (Flagger, Argo Rollouts) for canary

AWS ECS

• Native rolling updates
• Blue-green with CodeDeploy
• Canary with AppMesh or ALB traffic weighting

Terraform

# Blue-green with target group switching
resource "aws_lb_listener_rule" "production" {
  listener_arn = aws_lb_listener.app.arn
 
  action {
    type             = "forward"
    target_group_arn = var.active_target_group  # Switch this variable
  }
}

GitOps Tools

• ArgoCD: Declarative deployments with sync waves
• Flux: Progressive delivery with Flagger
• Spinnaker: Multi-cloud canary deployments

Conclusion

There is no universal best deployment strategy. The right choice depends on your system requirements, infrastructure budget, and risk tolerance.

Start simple with rolling deployments. Add blue-green when you need faster rollbacks. Introduce canary when you need to validate changes with real users before full rollout.

The key is to have a strategy at all. Too many production incidents happen because teams deploy code without thinking about how to roll back if things go wrong.

Choose your strategy. Document it. Practice it. And when something breaks at 3 AM, you will be glad you did.