Docker Swarm provides manual service scaling through simple commands, but production workloads often experience variable demand that requires automatic scaling. When traffic increases, you need additional capacity immediately. When traffic decreases, you want to reduce costs by scaling down. This article explores strategies for implementing metric-based autoscaling in Swarm environments using external tools and automation frameworks.
The Scaling Challenge
Why Manual Scaling is Insufficient
Manual scaling works for predictable workloads with stable demand. However, modern applications face dynamic conditions:
Traffic Spikes: Sudden increases in user requests require immediate capacity Daily Patterns: Usage fluctuates between peak and off-peak hours Weekly Cycles: Weekday vs weekend traffic varies significantly Seasonal Changes: Holiday periods or special events create temporary demand Gradual Growth: User base expansion requires continuous capacity adjustments
Manual intervention cannot respond quickly enough to these changes, leading to either over-provisioning (wasted resources) or under-provisioning (degraded performance).
Metrics-Driven Decisions
Effective autoscaling requires monitoring key metrics:
CPU Utilization: Percentage of CPU capacity being used Memory Usage: Amount of RAM consumed by services Request Rate: Number of requests per second Response Time: How quickly services respond to requests Queue Depth: Number of pending tasks or messages Custom Application Metrics: Business-specific indicators
By continuously monitoring these metrics, autoscaling systems make informed decisions about when to scale up or down.
Understanding Swarm's Scaling Primitives
Manual Scaling Commands
Swarm provides basic scaling commands:
# Scale a service to specific replica count docker service scale myapp=5 # Scale using update command docker service update --replicas 10 myapp # Scale multiple services simultaneously docker service scale web=3 api=5 worker=2
These commands adjust the desired replica count. Swarm's orchestrator creates or removes tasks to match the new count.
Scaling Behavior
When you scale a service:
- Swarm updates the service's desired replica count
- The orchestrator detects the difference between desired and actual state
- For scale-up: New tasks are created and scheduled to nodes
- For scale-down: Existing tasks are gracefully stopped
The process respects update parallelism and delay settings:
docker service update \ --replicas 10 \ --update-parallelism 2 \ --update-delay 10s \ myapp
This scales to 10 replicas, updating 2 tasks at a time with 10-second delays.
Scaling Limits
Consider limits when scaling:
Node Capacity: Available CPU and memory on worker nodes Network Bandwidth: Total bandwidth across nodes Storage I/O: Disk performance limits Service Dependencies: Database connections, external API rate limits
Scaling beyond these limits degrades performance even with additional replicas.
Metrics Collection Architecture
The Monitoring Stack
Implementing autoscaling requires a monitoring infrastructure:
Metrics Collection: Gather performance data from containers and services Metrics Storage: Store time-series data for analysis Metrics Query: Retrieve current and historical metrics Decision Engine: Analyze metrics and determine scaling actions Scaling Executor: Execute scaling commands against Swarm
Prometheus for Metrics Collection
Prometheus is a popular metrics collection system that integrates well with Docker:
Pull-Based Model: Prometheus scrapes metrics from exporters Time-Series Database: Stores metrics with timestamps Flexible Query Language: PromQL enables complex metric queries Alerting: Trigger actions based on metric thresholds
Deploy Prometheus as a service:
docker service create \ --name prometheus \ --publish 9090:9090 \ --mount type=bind,source=/prometheus-config,target=/etc/prometheus \ --replicas 1 \ prom/prometheus
cAdvisor for Container Metrics
cAdvisor collects container resource usage metrics:
docker service create \ --name cadvisor \ --mode global \ --mount type=bind,source=/,target=/rootfs,readonly=true \ --mount type=bind,source=/var/run,target=/var/run,readonly=false \ --mount type=bind,source=/sys,target=/sys,readonly=true \ --mount type=bind,source=/var/lib/docker,target=/var/lib/docker,readonly=true \ --publish 8080:8080 \ google/cadvisor:latest
Using global mode deploys cAdvisor on every node, collecting metrics from all containers.
Node Exporter for System Metrics
Node Exporter collects host-level metrics:
docker service create \ --name node-exporter \ --mode global \ --mount type=bind,source=/proc,target=/host/proc,readonly=true \ --mount type=bind,source=/sys,target=/host/sys,readonly=true \ --mount type=bind,source=/,target=/rootfs,readonly=true \ --publish 9100:9100 \ prom/node-exporter \ --path.procfs=/host/proc \ --path.sysfs=/host/sys \ --collector.filesystem.ignored-mount-points="^/(sys|proc|dev|host|etc)($$|/)"
Prometheus Configuration
Configure Prometheus to scrape metrics:
# prometheus.yml
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'cadvisor'
dns_sd_configs:
- names:
- 'tasks.cadvisor'
type: 'A'
port: 8080
- job_name: 'node-exporter'
dns_sd_configs:
- names:
- 'tasks.node-exporter'
type: 'A'
port: 9100
Prometheus discovers all instances of cAdvisor and Node Exporter automatically.
Querying Metrics with PromQL
CPU Utilization Queries
Calculate average CPU usage across service replicas:
# Average CPU usage for service
avg(rate(container_cpu_usage_seconds_total{
container_label_com_docker_swarm_service_name="myapp"
}[5m])) * 100
This calculates the average CPU usage percentage over the last 5 minutes for all containers in the "myapp" service.
Memory Utilization Queries
Query memory usage:
# Average memory usage for service
avg(container_memory_usage_bytes{
container_label_com_docker_swarm_service_name="myapp"
}) / 1024 / 1024
Returns average memory usage in megabytes.
Request Rate Queries
For HTTP services with metrics endpoints:
# Requests per second
sum(rate(http_requests_total{
service="myapp"
}[5m]))
Calculate total requests per second across all replicas.
Response Time Queries
Monitor service response times:
# 95th percentile response time
histogram_quantile(0.95,
rate(http_request_duration_seconds_bucket{
service="myapp"
}[5m])
)
Tracks the 95th percentile of response times.
Implementing Autoscaling Logic
Scaling Decision Criteria
Define when to scale:
Scale Up When:
- CPU utilization > 70% for 2 minutes
- Memory usage > 80% for 3 minutes
- Request queue depth > 100 for 1 minute
- Response time > 500ms for 5 minutes
Scale Down When:
- CPU utilization < 30% for 10 minutes
- Memory usage < 40% for 10 minutes
- All metrics below thresholds for sustained period
Longer stabilization periods for scale-down prevent flapping (rapid scale up/down cycles).
Calculating Desired Replicas
Use target utilization to calculate desired replicas:
desired_replicas = current_replicas × (current_utilization / target_utilization)
Example:
- Current replicas: 5
- Current CPU utilization: 80%
- Target CPU utilization: 60%
desired_replicas = 5 × (80 / 60) = 6.67 ≈ 7
Scale to 7 replicas.
Scaling Constraints
Apply constraints to scaling decisions:
Minimum Replicas: Never scale below this count (e.g., 2 for availability) Maximum Replicas: Upper limit based on capacity or cost (e.g., 20) Step Size: Maximum replicas to add/remove per action (e.g., add max 5 at once) Cooldown Period: Wait time between scaling actions (e.g., 5 minutes)
def calculate_target_replicas(current, desired, min_replicas, max_replicas, max_step):
# Apply min/max constraints
target = max(min_replicas, min(max_replicas, desired))
# Apply step size constraint
if target > current:
target = min(target, current + max_step)
elif target < current:
target = max(target, current - max_step)
return target
Python Autoscaler Implementation
Basic Autoscaler Structure
A simple Python autoscaler:
#!/usr/bin/env python3
import time
import subprocess
import requests
from prometheus_api_client import PrometheusConnect
class SwarmAutoscaler:
def __init__(self, service_name, prometheus_url):
self.service_name = service_name
self.prom = PrometheusConnect(url=prometheus_url)
self.min_replicas = 2
self.max_replicas = 20
self.target_cpu = 60.0
self.cooldown_seconds = 300
self.last_scale_time = 0
def get_current_replicas(self):
"""Get current replica count from service"""
result = subprocess.run(
['docker', 'service', 'ls', '--filter', f'name={self.service_name}', '--format', '{{.Replicas}}'],
capture_output=True, text=True
)
replicas_str = result.stdout.strip().split('/')[0]
return int(replicas_str)
def get_avg_cpu_usage(self):
"""Query Prometheus for average CPU usage"""
query = f'''
avg(rate(container_cpu_usage_seconds_total{{
container_label_com_docker_swarm_service_name="{self.service_name}"
}}[5m])) * 100
'''
result = self.prom.custom_query(query)
if result:
return float(result[0]['value'][1])
return 0.0
def calculate_desired_replicas(self, current_replicas, current_cpu):
"""Calculate desired replica count"""
if current_cpu == 0:
return current_replicas
desired = int(current_replicas * (current_cpu / self.target_cpu))
desired = max(self.min_replicas, min(self.max_replicas, desired))
# Limit scaling steps
max_step = max(1, current_replicas // 4)
if desired > current_replicas:
desired = min(desired, current_replicas + max_step)
elif desired < current_replicas:
desired = max(desired, current_replicas - max_step)
return desired
def scale_service(self, replicas):
"""Execute scaling command"""
subprocess.run(
['docker', 'service', 'scale', f'{self.service_name}={replicas}'],
check=True
)
self.last_scale_time = time.time()
print(f"Scaled {self.service_name} to {replicas} replicas")
def should_scale(self):
"""Check if cooldown period has passed"""
return (time.time() - self.last_scale_time) > self.cooldown_seconds
def run(self):
"""Main autoscaling loop"""
while True:
try:
current_replicas = self.get_current_replicas()
current_cpu = self.get_avg_cpu_usage()
print(f"Current replicas: {current_replicas}, CPU: {current_cpu:.2f}%")
if self.should_scale():
desired_replicas = self.calculate_desired_replicas(
current_replicas, current_cpu
)
if desired_replicas != current_replicas:
print(f"Scaling from {current_replicas} to {desired_replicas}")
self.scale_service(desired_replicas)
else:
print("No scaling needed")
else:
print(f"Cooldown active, skipping scaling check")
except Exception as e:
print(f"Error: {e}")
time.sleep(60) # Check every minute
if __name__ == '__main__':
autoscaler = SwarmAutoscaler(
service_name='myapp',
prometheus_url='http://prometheus:9090'
)
autoscaler.run()
Running the Autoscaler
Deploy the autoscaler as a service:
# Build autoscaler image docker build -t autoscaler:latest . # Deploy autoscaler service docker service create \ --name autoscaler \ --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \ --replicas 1 \ --constraint 'node.role==manager' \ autoscaler:latest
The autoscaler must run on a manager node with access to the Docker socket.
Multi-Metric Autoscaling
Combining Multiple Metrics
Scale based on multiple metrics simultaneously:
def get_scaling_signal(self):
"""Combine multiple metrics for scaling decision"""
cpu_usage = self.get_avg_cpu_usage()
memory_usage = self.get_avg_memory_usage()
request_rate = self.get_request_rate()
# Normalize metrics to 0-100 scale
cpu_score = (cpu_usage / self.target_cpu) * 100
memory_score = (memory_usage / self.target_memory) * 100
rate_score = (request_rate / self.target_rate) * 100
# Use highest metric as scaling signal
return max(cpu_score, memory_score, rate_score)
This approach scales based on whichever metric is most constrained.
Weighted Metrics
Apply weights to prioritize certain metrics:
def get_weighted_signal(self):
"""Calculate weighted scaling signal"""
cpu_usage = self.get_avg_cpu_usage()
memory_usage = self.get_avg_memory_usage()
response_time = self.get_avg_response_time()
# Weights sum to 1.0
cpu_weight = 0.5
memory_weight = 0.3
latency_weight = 0.2
cpu_score = (cpu_usage / self.target_cpu) * cpu_weight
memory_score = (memory_usage / self.target_memory) * memory_weight
latency_score = (response_time / self.target_latency) * latency_weight
return (cpu_score + memory_score + latency_score) * 100
Schedule-Based Scaling
Predictive Scaling
Scale proactively based on expected patterns:
import datetime
def get_scheduled_min_replicas(self):
"""Adjust minimum replicas based on time of day"""
now = datetime.datetime.now()
hour = now.hour
day_of_week = now.weekday()
# Business hours (9 AM - 6 PM, Monday-Friday)
if 0 <= day_of_week <= 4 and 9 <= hour < 18:
return 10 # Higher baseline during business hours
# Weekend
if day_of_week >= 5:
return 3 # Lower baseline on weekends
# Night hours
return 5 # Normal baseline
This prevents scale-down before predictable traffic increases.
Event-Based Scaling
Scale for known events:
def check_special_events(self):
"""Check for scheduled events requiring extra capacity"""
now = datetime.datetime.now()
# Black Friday
if now.month == 11 and now.day == 24:
self.min_replicas = 20
self.max_replicas = 50
# Regular period
else:
self.min_replicas = 2
self.max_replicas = 20
Using External Autoscaling Tools
Docker Flow Swarm Listener
Docker Flow Swarm Listener monitors Swarm events and can trigger webhooks:
docker service create \ --name swarm-listener \ --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \ --publish 8080:8080 \ --constraint 'node.role==manager' \ vfarcic/docker-flow-swarm-listener
Configure it to call your autoscaler on service events.
Orbiter
Orbiter is an autoscaling daemon for Swarm:
docker service create \ --name orbiter \ --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \ --env SWARM_MODE=true \ --env PROMETHEUS_URL=http://prometheus:9090 \ --constraint 'node.role==manager' \ gianarb/orbiter
Configure scaling rules via labels on services:
docker service update \ --label-add orbiter.enabled=true \ --label-add orbiter.metrics.cpu.target=70 \ --label-add orbiter.min_replicas=2 \ --label-add orbiter.max_replicas=10 \ myapp
Custom Webhooks
Implement webhook-based scaling:
from flask import Flask, request
import subprocess
app = Flask(__name__)
@app.route('/scale/<service_name>', methods=['POST'])
def scale_service(service_name):
data = request.json
replicas = data.get('replicas')
if not replicas:
return {'error': 'replicas required'}, 400
try:
subprocess.run(
['docker', 'service', 'scale', f'{service_name}={replicas}'],
check=True
)
return {'status': 'success', 'replicas': replicas}
except Exception as e:
return {'error': str(e)}, 500
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
Prometheus Alertmanager can call this webhook when metrics exceed thresholds.
Queue-Based Autoscaling
Scaling Based on Queue Depth
For worker services processing queued tasks:
def get_queue_depth(self):
"""Query message queue for pending tasks"""
# Example for Redis queue
import redis
r = redis.Redis(host='redis', port=6379)
return r.llen('task_queue')
def calculate_replicas_from_queue(self):
"""Scale based on queue depth"""
queue_depth = self.get_queue_depth()
# Target: each worker processes ~100 tasks
tasks_per_worker = 100
desired = (queue_depth // tasks_per_worker) + 1
desired = max(self.min_replicas, min(self.max_replicas, desired))
return desired
Hybrid Queue and Resource Scaling
Combine queue depth with resource utilization:
def hybrid_scaling_decision(self):
"""Scale based on both queue and resources"""
queue_replicas = self.calculate_replicas_from_queue()
resource_replicas = self.calculate_replicas_from_resources()
# Use the higher of the two
return max(queue_replicas, resource_replicas)
Handling Scaling Edge Cases
Scale-Down Safety
Implement safeguards for scale-down:
def safe_to_scale_down(self):
"""Check if it's safe to remove replicas"""
# Check recent error rates
error_rate = self.get_error_rate()
if error_rate > 0.05: # More than 5% errors
return False
# Check connection count
active_connections = self.get_active_connections()
if active_connections > (self.current_replicas * 80): # Near capacity
return False
# Check time since last scale event
if (time.time() - self.last_scale_time) < 600: # Less than 10 minutes
return False
return True
Graceful Scale-Down
Ensure tasks finish before removal:
# Configure service with graceful shutdown docker service update \ --stop-grace-period 60s \ --update-order stop-first \ myapp
Tasks get 60 seconds to finish processing before forceful termination.
Preventing Flapping
Implement hysteresis to prevent rapid scale up/down:
class ScalingHistory:
def __init__(self):
self.history = []
self.max_history = 10
def add(self, replica_count):
self.history.append(replica_count)
if len(self.history) > self.max_history:
self.history.pop(0)
def is_stable(self):
"""Check if replica count is stable"""
if len(self.history) < self.max_history:
return True
# Calculate variance
avg = sum(self.history) / len(self.history)
variance = sum((x - avg) ** 2 for x in self.history) / len(self.history)
# Low variance indicates stability
return variance < 2.0
Only scale if recent history shows instability.
Monitoring Autoscaling Effectiveness
Tracking Scaling Events
Log all scaling actions:
import logging
logging.basicConfig(
filename='autoscaler.log',
level=logging.INFO,
format='%(asctime)s - %(message)s'
)
def scale_service(self, replicas):
old_replicas = self.get_current_replicas()
subprocess.run(['docker', 'service', 'scale', f'{self.service_name}={replicas}'])
logging.info(f"Scaled {self.service_name}: {old_replicas} -> {replicas}")
logging.info(f"Metrics - CPU: {self.get_avg_cpu_usage():.2f}%, Memory: {self.get_avg_memory_usage():.2f}%")
Metrics for Autoscaler Performance
Track autoscaler metrics:
Scale Frequency: Number of scaling events per hour Time to Scale: How quickly scaling responds to demand changes Over-Scaling Rate: Percentage of time scaled above needed capacity Under-Scaling Rate: Percentage of time scaled below needed capacity Cost Efficiency: Resource costs vs demand met
Visualizing Scaling Behavior
Create Grafana dashboards showing:
- Service replica count over time
- CPU/memory utilization over time
- Request rate and response times
- Scaling events annotations
- Cost metrics
This visualizes how well autoscaling responds to demand.
Best Practices for Autoscaling
Design Services for Scaling
Stateless Design: Services should not store state locally Fast Startup: Containers should start quickly to respond to scale events Graceful Shutdown: Handle SIGTERM signal to finish in-flight requests Health Checks: Implement health endpoints for accurate scaling decisions
Set Appropriate Targets
Conservative Targets: Aim for 60-70% utilization, leaving headroom for spikes Metric Selection: Choose metrics that directly correlate with user experience Multiple Metrics: Use multiple metrics to avoid false signals
Test Autoscaling Behavior
Load Testing: Simulate traffic patterns to verify scaling behavior Chaos Engineering: Introduce failures to test scaling resilience Boundary Testing: Test minimum and maximum replica scenarios
Monitor and Tune
Regular Review: Periodically review scaling parameters Adjust Thresholds: Update targets based on observed patterns Optimize Cooldowns: Balance responsiveness with stability
Cost Management
Set Maximum Limits: Prevent runaway scaling costs Schedule-Based Minimums: Reduce baseline capacity during low-demand periods Track Costs: Monitor cloud or infrastructure costs related to scaling
Metric-based autoscaling transforms Swarm from a manually managed platform to a responsive, self-adapting system. By collecting metrics through tools like Prometheus, implementing intelligent scaling logic that considers multiple factors and constraints, and deploying autoscalers as services within your cluster, you create infrastructure that automatically matches capacity to demand. Whether using simple scripts, sophisticated multi-metric algorithms, or external tools, effective autoscaling reduces costs during low-demand periods while ensuring sufficient capacity during traffic spikes, all without manual intervention.