The docker stack deploy command provides a powerful way to define and manage multi-service applications through declarative configuration files. This approach allows you to describe your entire application architecture in a single file and deploy it with one command.
Understanding Stack Deployment
Stack deployment represents a declarative approach to managing complex applications. Instead of creating individual services one at a time with separate commands, you define all services, networks, volumes, and configurations in a compose file, then deploy everything together as a cohesive unit.
What is a Stack?
A stack is a collection of interrelated services that are deployed and managed together. The stack provides a namespace for these services, allowing multiple stacks to coexist without naming conflicts and making it easy to manage all components of an application as a single unit.
The docker stack deploy Command
The basic syntax for deploying a stack is straightforward:
docker stack deploy -c compose-file.yml stack-name
This single command reads your compose file, creates or updates all defined services, networks, volumes, and configurations, and ensures your application matches the declared state.
Command Options
The docker stack deploy command accepts several options that control deployment behavior:
The -c or --compose-file flag specifies the compose file to use. You can specify this flag multiple times to use multiple compose files, which are merged in order.
The --prune flag removes services that are no longer defined in the compose file. Without this flag, services that existed in previous deployments but are removed from the compose file will continue running.
The --resolve-image flag controls how image tags are resolved. The options are always, changed, or never, determining when the orchestrator should pull fresh image data from registries.
The --with-registry-auth flag sends registry authentication details along with the task specifications, allowing workers to pull images from private registries.
Compose File Structure
The compose file used with docker stack deploy follows a specific format, typically version 3 or higher. This file uses YAML syntax to define your application's architecture.
Service Definitions
Each service in your stack is defined under the services key. A service definition includes the image to use, networking configuration, volume mounts, environment variables, and deployment-specific parameters.
version: '3.8'
services:
web:
image: nginx:latest
deploy:
replicas: 3
restart_policy:
condition: on-failure
ports:
- "80:80"
database:
image: postgres:13
deploy:
replicas: 1
placement:
constraints:
- node.role == manager
environment:
POSTGRES_PASSWORD: example
volumes:
- db-data:/var/lib/postgresql/data
volumes:
db-data:
The deploy Key
The deploy key within service definitions contains orchestration-specific configuration. This key is only used during stack deployment and is ignored when using the compose file with standard container commands.
Replica Configuration
The replicas setting specifies how many task instances should run for the service. This is one of the most commonly used deployment parameters.
Restart Policies
The restart_policy controls what happens when tasks exit. You can specify conditions like on-failure, any, or none, along with delay, max_attempts, and window parameters to fine-tune restart behavior.
Resource Constraints
Resource limits and reservations ensure services get the resources they need without monopolizing host resources:
deploy:
resources:
limits:
cpus: '0.50'
memory: 512M
reservations:
cpus: '0.25'
memory: 256M
Placement Constraints
Placement constraints control which hosts can run tasks for a service. These constraints use a simple expression syntax:
deploy:
placement:
constraints:
- node.labels.environment == production
- node.labels.storage == ssd
Update Configuration
Update configuration parameters control how the orchestrator performs rolling updates:
deploy:
update_config:
parallelism: 2
delay: 10s
failure_action: rollback
monitor: 30s
max_failure_ratio: 0.3
The parallelism setting determines how many tasks to update simultaneously. The delay specifies how long to wait between update batches. The failure_action defines what to do if an update fails, with options like pause, continue, or rollback.
Network Configuration
Networks defined in your compose file are automatically created during stack deployment. Services can connect to these networks, enabling communication between components.
services:
web:
networks:
- frontend
- backend
api:
networks:
- backend
- database-net
db:
networks:
- database-net
networks:
frontend:
backend:
database-net:
Networks are scoped to the stack namespace, preventing conflicts between stacks. Services within the stack can reach each other using service names as hostnames.
Network Configuration Options
Networks can be configured with specific options:
networks:
frontend:
driver: overlay
driver_opts:
encrypted: "true"
attachable: true
The attachable option allows standalone containers to connect to the network, useful for debugging or running one-off tasks that need to communicate with stack services.
Volume Management
Volumes defined in the compose file are created during stack deployment if they don't already exist. This ensures your services have the persistent storage they need.
services:
app:
volumes:
- app-data:/data
- app-config:/etc/app/config
volumes:
app-data:
driver: local
app-config:
driver: local
driver_opts:
type: none
device: /opt/app-config
o: bind
Volumes persist beyond stack removal unless explicitly deleted, protecting your data from accidental loss during redeployment.
Configuration and Secrets
Stacks support two mechanisms for managing sensitive and non-sensitive configuration data: configs and secrets.
Configs
Configs store non-sensitive configuration data that services need:
services:
web:
configs:
- source: nginx-config
target: /etc/nginx/nginx.conf
configs:
nginx-config:
file: ./nginx.conf
Configs are immutable once created. To change a config, you create a new version with a different name and update your service to use the new config.
Secrets
Secrets store sensitive data like passwords, API keys, and certificates:
services:
db:
secrets:
- db-password
- db-root-password
secrets:
db-password:
external: true
db-root-password:
file: ./db_root_password.txt
Secrets are stored encrypted and only made available to services that explicitly declare them. They're mounted as files in the container, typically in /run/secrets/.
Deployment Process
When you run docker stack deploy, several things happen behind the scenes:
First, the compose file is parsed and validated. The orchestrator checks for syntax errors and ensures all required fields are present.
Next, the orchestrator creates any networks, volumes, configs, and secrets defined in the compose file that don't already exist.
Then, for each service, the orchestrator either creates a new service or updates an existing one. If the service already exists, the orchestrator compares the current configuration with the desired configuration and makes necessary changes.
Finally, the orchestrator schedules tasks to achieve the desired replica count for each service, placing them according to specified constraints and available resources.
Updating a Stack
The same docker stack deploy command is used to update an existing stack. When you modify your compose file and redeploy, the orchestrator intelligently updates only what changed.
Update Behavior
If a service's image changes, the orchestrator performs a rolling update, replacing tasks with new ones running the updated image according to the update configuration parameters.
If deployment parameters change (like replica count or resource constraints), the orchestrator adjusts the running services accordingly.
If networking or volume configuration changes, the orchestrator applies these changes where possible, though some changes may require service recreation.
Update Strategy
The orchestrator's update strategy minimizes disruption. Services continue running while updates are applied, and the orchestrator can automatically roll back if updates fail.
Multiple Compose Files
You can use multiple compose files with stack deployment, allowing you to separate base configuration from environment-specific overrides:
docker stack deploy -c base.yml -c production.yml myapp
Files are merged in order, with later files overriding earlier ones. This pattern is useful for maintaining one base configuration while customizing for different environments.
File Merging Behavior
When merging files, simple values (like strings and numbers) are replaced. Lists are replaced entirely, not merged. Maps (dictionaries) are merged, with later files overriding specific keys.
Stack Namespacing
Stack names serve as namespaces. All resources created by a stack are prefixed with the stack name, preventing conflicts between stacks.
For example, deploying a stack named "myapp" with a service named "web" creates a service actually named "myapp_web". Networks, volumes, and other resources follow the same pattern.
This namespacing allows you to deploy multiple instances of the same application (perhaps for different clients or environments) without conflicts.
Environment Variables
Services can use environment variables in two ways: defined directly in the compose file or loaded from the deployment environment.
services:
app:
environment:
DATABASE_URL: postgres://db:5432/myapp
DEBUG: "false"
env_file:
- ./common.env
- ./production.env
You can also use variable substitution in the compose file itself, allowing values to be provided at deployment time:
services:
app:
image: myapp:${VERSION:-latest}
environment:
API_KEY: ${API_KEY}
Port Publishing
Services can publish ports to make them accessible from outside the cluster:
services:
web:
ports:
- "8080:80"
- "443:443"
Published ports are made available on all hosts, and traffic to those ports is automatically routed to available service tasks through the ingress routing mesh.
Port Publishing Modes
The default mode is ingress, where published ports are available on every host. You can also use host mode, which publishes the port only on hosts running tasks for that service:
services:
web:
ports:
- target: 80
published: 8080
protocol: tcp
mode: host
Health Checks
Services can define health checks that the orchestrator uses to determine if tasks are healthy:
services:
web:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
The orchestrator periodically runs the health check and considers a task unhealthy if it fails the specified number of retries. Unhealthy tasks are replaced automatically.
Labels
Labels provide metadata for services and other resources. They're key-value pairs that can be used for organization, filtering, and integration with external tools:
services:
web:
deploy:
labels:
com.example.description: "Web frontend"
com.example.department: "engineering"
com.example.version: "1.0"
Labels don't affect runtime behavior but are useful for documenting and organizing services.
Dependencies
While explicit service dependencies aren't enforced during stack deployment, you can define dependency relationships for documentation:
services:
web:
depends_on:
- api
- cache
api:
depends_on:
- database
These dependencies don't control startup order but communicate architectural relationships within your compose file.
Deployment Workflow
A typical deployment workflow involves preparing your compose file, ensuring images are available in a registry accessible to all hosts, creating any external secrets or configs, and then running the deploy command.
Pre-deployment Preparation
Before deploying, ensure all referenced images are pushed to a registry. Swarm hosts need to pull images, so they must be available in a shared repository.
Create any external secrets or configs referenced in your compose file. External resources must exist before deployment.
Verify that any required labels or conditions referenced in placement constraints exist on your hosts.
Running the Deployment
Execute the deploy command with your compose file and chosen stack name. Monitor the output for any errors or warnings.
The command typically completes quickly, but actual deployment takes longer as tasks are scheduled and containers start. The initial command just submits the deployment; the orchestrator continues working in the background.
Post-deployment Verification
After deploying, verify that services are running correctly. Check that the correct number of tasks are running and that they're healthy. Test connectivity between services and external access to published ports.
Removing a Stack
Stacks are removed with the docker stack rm command:
docker stack rm myapp
This removes all services, networks, and configs created by the stack. Volumes are preserved by default to prevent data loss. Secrets are also preserved unless they were created as part of the stack (non-external secrets).
Stack Deployment Best Practices
Version Control
Keep your compose files in version control. This provides history, enables collaboration, and allows you to roll back to previous configurations.
Image Tagging
Use specific image tags rather than latest. This ensures deployments are reproducible and makes it clear which version of an application is running.
Resource Limits
Always define resource limits for services. This prevents any single service from consuming all available resources and affecting other services.
Health Checks
Implement health checks for services. This enables automatic recovery from failures and ensures traffic is only sent to healthy tasks.
External Secrets
Use external secrets for sensitive data rather than embedding secrets in compose files. This keeps sensitive data out of version control and provides better security.
Advanced Deployment Scenarios
Blue-Green Deployments
While stack deployment doesn't directly support blue-green deployments, you can implement them using multiple stacks with different names and switching traffic between them.
Canary Deployments
Canary deployments can be implemented by creating services with different configurations and gradually shifting traffic from old to new versions.
Multi-Environment Management
Use multiple compose files or variable substitution to manage deployments across different environments (development, staging, production) from a single set of base configuration files.
Troubleshooting Deployment Issues
Service Creation Failures
If a service fails to create, check the compose file syntax, verify image availability, and ensure placement constraints can be satisfied by available hosts.
Update Failures
If an update fails, the orchestrator's behavior depends on the configured failure action. Check task logs to diagnose why new tasks are failing and adjust the configuration or image accordingly.
Network Connectivity Issues
If services can't communicate, verify that they're connected to the same networks and that service names are being used as hostnames rather than IP addresses.
Stack Deployment Patterns
Monolithic Stack
Deploy all components of an application in a single stack. This is simple to manage and works well for applications with tightly coupled components.
Microservices Stack
Deploy each microservice as a separate stack. This provides isolation and independent deployment but requires more coordination for inter-service communication.
Shared Infrastructure Stack
Deploy shared infrastructure components (databases, message queues, caching layers) in one stack and application services in separate stacks. This allows applications to be deployed and updated independently while sharing infrastructure.
Compose File Validation
Before deploying, validate your compose file syntax and structure. The docker-compose config command can be used to check syntax and see how multiple files merge together, though it may include options not supported by stack deployment.
Create a deployment checklist that includes validation steps, image availability checks, and backup procedures to ensure smooth deployments.
Understanding Deployment Output
The deployment command provides feedback about what's being created or updated. Pay attention to warnings about deprecated options or incompatible settings.
The command completes quickly, but this doesn't mean deployment is finished—it means the desired state has been submitted to the orchestrator. Actual deployment happens asynchronously as tasks are scheduled and started.
Stack deployment provides a powerful, declarative way to manage complex applications. By defining your entire application architecture in compose files, you gain reproducibility, version control, and the ability to deploy consistent environments across development, staging, and production.