Debugging containerized applications requires understanding container lifecycle events, analyzing logs, inspecting container state, and tracing execution flow. Docker Compose provides comprehensive tooling for monitoring container behavior, diagnosing failures, and understanding system state. Mastering these debugging techniques enables rapid problem identification and resolution in both development and production environments.
Understanding Container Lifecycle States
Containers transition through several states during their lifecycle:
- Created: Container exists but hasn't started
- Running: Container is executing
- Restarting: Container is automatically restarting
- Paused: Container execution is suspended
- Exited: Container has stopped
- Dead: Container in error state, cannot be started
View current container states:
docker-compose ps
This shows all services and their current states, exit codes, and port mappings.
Viewing Container Logs
Logs are the primary debugging tool for containerized applications:
# View logs for all services docker-compose logs # View logs for specific service docker-compose logs api # Follow logs in real-time docker-compose logs -f # Show timestamps docker-compose logs -t # Limit to last N lines docker-compose logs --tail=100 # Since specific time docker-compose logs --since 2024-01-15T10:00:00
Combine options for targeted debugging:
docker-compose logs -f --tail=50 --timestamps api
Multi-Service Log Viewing
Monitor multiple services simultaneously:
# View logs from multiple services docker-compose logs api worker database # Follow multiple services docker-compose logs -f api worker
Compose color-codes output by service, making it easy to distinguish sources.
Inspecting Container Configuration
View complete container configuration:
# Inspect specific container
docker inspect $(docker-compose ps -q api)
# Get specific field
docker inspect --format='{{.State.Status}}' $(docker-compose ps -q api)
# View network settings
docker inspect --format='{{json .NetworkSettings}}' $(docker-compose ps -q api) | jq
The docker inspect command reveals everything about a container: environment variables, mounts, networks, resource limits, and runtime state.
Checking Container Resource Usage
Monitor real-time resource consumption:
# View all containers docker stats # View specific containers docker stats $(docker-compose ps -q) # Single snapshot (no streaming) docker stats --no-stream
This displays CPU, memory, network I/O, and block I/O usage for containers.
Executing Commands in Running Containers
Access containers for interactive debugging:
# Start interactive shell docker-compose exec api sh # Execute specific command docker-compose exec api ps aux # Run as different user docker-compose exec -u root api sh # Execute without allocating TTY (for scripts) docker-compose exec -T api cat /app/config.json
This enables direct inspection of filesystem, processes, and runtime state.
Viewing Container Events
Monitor Docker events in real-time:
# View all events docker events # Filter by container docker events --filter container=$(docker-compose ps -q api) # Filter by event type docker events --filter event=start --filter event=die # Since specific time docker events --since '2024-01-15T10:00:00'
Events show create, start, die, stop, kill, and other lifecycle transitions.
Compose Event Monitoring
Track Compose-specific operations:
# Watch as services start docker-compose up -d & docker events --filter type=container # Monitor during operation docker events --filter label=com.docker.compose.project=myproject
This reveals how Compose orchestrates container lifecycle.
Checking Container Health Status
View health check results:
# Show health status
docker-compose ps
# Inspect health check details
docker inspect --format='{{json .State.Health}}' $(docker-compose ps -q api) | jq
Health check output includes the last several check results with timestamps and exit codes.
Analyzing Container Exit Codes
Exit codes indicate why containers stopped:
# View exit codes
docker-compose ps
# Get specific exit code
docker inspect --format='{{.State.ExitCode}}' $(docker-compose ps -q api)
Common exit codes:
- 0: Success
- 1: Generic error
- 2: Misuse of shell command
- 126: Command cannot execute
- 127: Command not found
- 130: Terminated by Ctrl+C
- 137: Killed (SIGKILL)
- 143: Terminated (SIGTERM)
Viewing Container Process List
See processes running inside containers:
# View processes docker-compose top # View specific service docker-compose top api # View all processes in detail docker-compose exec api ps auxf
This reveals what's actually executing inside containers.
Debugging Network Connectivity
Test network connections between services:
# Ping another service docker-compose exec api ping database # Test port connectivity docker-compose exec api nc -zv database 5432 # Trace network route docker-compose exec api traceroute database # DNS lookup docker-compose exec api nslookup database
These commands verify service discovery and network connectivity.
Inspecting Environment Variables
View container environment:
# View all environment variables
docker-compose exec api env
# Get specific variable
docker-compose exec api printenv DATABASE_URL
# Inspect from outside
docker inspect --format='{{json .Config.Env}}' $(docker-compose ps -q api) | jq
Verify that expected environment variables are set correctly.
Debugging Container Startup Issues
When containers fail to start:
# View all service status docker-compose ps -a # Check logs for failed service docker-compose logs failed-service # Try starting manually docker-compose up failed-service # Start with explicit output docker-compose up --no-deps failed-service
The --no-deps flag starts only the specified service without its dependencies, isolating the problem.
Analyzing Configuration
Validate and view final Compose configuration:
# Show resolved configuration docker-compose config # Validate without running docker-compose config --quiet # Show services only docker-compose config --services # Resolve variable substitution docker-compose config --resolve-image-digests
This reveals how Compose interprets your configuration, showing resolved environment variables and merged settings.
Debugging Volume Mounts
Verify volume mounts are working:
# List volumes docker volume ls # Inspect specific volume docker volume inspect myapp_data # Check mounts in container docker-compose exec api mount | grep /app # Verify file accessibility docker-compose exec api ls -la /app/data
Confirms volumes are mounted at expected paths with correct permissions.
Tracing Container Filesystem Changes
See what changed in container filesystem:
# Show filesystem changes docker diff $(docker-compose ps -q api)
Output shows:
- A: Added files
- C: Changed files
- D: Deleted files
This helps understand what containers modify during execution.
Debugging Build Issues
When builds fail:
# Build with verbose output docker-compose build --progress=plain # Build without cache docker-compose build --no-cache api # Build and show all output DOCKER_BUILDKIT=0 docker-compose build api
Verbose output reveals exactly where builds fail.
Inspecting Image Layers
Understand image composition:
# Show image history docker history $(docker-compose images -q api) # Detailed layer information docker inspect $(docker-compose images -q api)
Reveals each layer's size and creation command, helping identify bloated images.
Debugging Port Mapping Issues
Verify port mappings:
# Show port mappings docker-compose ps # Check specific port docker-compose port api 3000 # Test port connectivity from host curl http://localhost:3000 # Check from inside container docker-compose exec api curl http://localhost:3000
Distinguishes between internal and external connectivity issues.
Analyzing Container Logs with Filters
Filter logs for specific patterns:
# Grep for errors docker-compose logs api | grep ERROR # Filter by time window docker-compose logs --since 2024-01-15T10:00:00 --until 2024-01-15T11:00:00 api # Get logs by severity (if structured) docker-compose logs api | grep '"level":"error"'
Targeted log filtering speeds up problem identification.
Debugging Dependency Issues
When services won't start due to dependencies:
# Check dependency graph docker-compose config --services # Start services individually docker-compose up database docker-compose up api # Start with dependency checking docker-compose up --abort-on-container-exit
The --abort-on-container-exit flag stops everything if any service exits, useful for debugging startup sequences.
Monitoring Container Restart Behavior
Track why containers restart:
# Watch events for restarts
docker events --filter event=start --filter event=die
# Check restart count
docker inspect --format='{{.RestartCount}}' $(docker-compose ps -q api)
# View last start time
docker inspect --format='{{.State.StartedAt}}' $(docker-compose ps -q api)
Frequent restarts indicate startup or runtime failures.
Debugging Memory Issues
Identify memory problems:
# Check memory usage
docker stats --no-stream $(docker-compose ps -q api)
# View memory limits
docker inspect --format='{{.HostConfig.Memory}}' $(docker-compose ps -q api)
# Check for OOM kills
docker inspect --format='{{.State.OOMKilled}}' $(docker-compose ps -q api)
# View last exit code (137 indicates OOM kill)
docker inspect --format='{{.State.ExitCode}}' $(docker-compose ps -q api)
Accessing Container Metadata
Retrieve detailed container metadata:
# Get container ID
docker-compose ps -q api
# Get container name
docker-compose ps | grep api
# List all labels
docker inspect --format='{{json .Config.Labels}}' $(docker-compose ps -q api) | jq
# Get creation time
docker inspect --format='{{.Created}}' $(docker-compose ps -q api)
Debugging Compose File Syntax
Validate Compose file syntax:
# Validate configuration docker-compose config # Check for specific service docker-compose config --services | grep api # Resolve and validate docker-compose -f docker-compose.yml -f docker-compose.prod.yml config
Syntax errors are caught before attempting to start services.
Examining Service Dependencies
Understand service startup order:
# View depends_on relationships docker-compose config | grep -A5 depends_on # Start with explicit dependency order docker-compose up -d database && sleep 5 && docker-compose up -d api
Manual startup helps isolate dependency-related issues.
Debugging Communication Between Services
Test inter-service communication:
# From one service to another docker-compose exec api curl http://database:5432 # Check service name resolution docker-compose exec api nslookup api # Verify network connectivity docker-compose exec api ping -c 3 worker
Capturing Network Traffic
Analyze network traffic for debugging:
# Install tcpdump in container docker-compose exec api apk add tcpdump # Capture traffic docker-compose exec api tcpdump -i any -w /tmp/capture.pcap # Or from host docker run --rm --net container:$(docker-compose ps -q api) \ nicolaka/netshoot tcpdump -i any
Debugging Slow Container Startup
Identify startup bottlenecks:
# Time startup time docker-compose up -d # Check logs during startup docker-compose logs -f api & docker-compose up -d api # Monitor events during startup docker events --filter container=$(docker-compose ps -q api) & docker-compose up -d api
Viewing Detailed Container Info
Get comprehensive container details:
# Full JSON output
docker inspect $(docker-compose ps -q api) | jq
# Specific sections
docker inspect --format='{{json .State}}' $(docker-compose ps -q api) | jq
docker inspect --format='{{json .Config}}' $(docker-compose ps -q api) | jq
docker inspect --format='{{json .NetworkSettings}}' $(docker-compose ps -q api) | jq
Debugging File Permission Issues
Check file permissions in containers:
# Check file ownership docker-compose exec api ls -ln /app # Check running user docker-compose exec api id # Check file permissions docker-compose exec api stat /app/config.json # Test file access docker-compose exec api cat /app/config.json
Analyzing Service Logs in Detail
Deep dive into service logs:
# Get full log with metadata docker logs --details $(docker-compose ps -q api) # Export logs to file docker-compose logs api > api-logs.txt # Stream logs continuously docker-compose logs -f --tail=100 api 2>&1 | tee debug.log
Debugging Signal Handling
Test how containers handle signals:
# Send SIGTERM docker-compose kill -s SIGTERM api # Send SIGKILL docker-compose kill -s SIGKILL api # Stop gracefully docker-compose stop -t 30 api # Watch how container responds docker events --filter container=$(docker-compose ps -q api)
Checking Container Resource Limits
Verify resource constraints:
# Check CPU limits
docker inspect --format='{{.HostConfig.NanoCpus}}' $(docker-compose ps -q api)
# Check memory limits
docker inspect --format='{{.HostConfig.Memory}}' $(docker-compose ps -q api)
# Check all resource limits
docker inspect --format='{{json .HostConfig}}' $(docker-compose ps -q api) | jq
Debugging DNS Issues
Troubleshoot DNS resolution:
# Check DNS servers docker-compose exec api cat /etc/resolv.conf # Test DNS resolution docker-compose exec api nslookup google.com # Test service name resolution docker-compose exec api nslookup api # Detailed DNS debug docker-compose exec api dig database
Monitoring Container Creation
Watch containers being created:
# Monitor creation events
docker events --filter event=create &
docker-compose up -d
# Check creation timestamps
docker inspect --format='{{.Created}}' $(docker-compose ps -q)
Debugging Container Crashes
Investigate crashed containers:
# Keep container after crash
docker-compose up --abort-on-container-exit
# View last crash logs
docker-compose logs --tail=100 api
# Check exit code
docker-compose ps -a
# Inspect crash state
docker inspect --format='{{.State}}' $(docker-compose ps -q api)
Using Compose in Debug Mode
Enable verbose Compose output:
# Verbose mode docker-compose --verbose up # Very verbose mode docker-compose --verbose --log-level DEBUG up # Trace mode (most verbose) docker-compose --log-level DEBUG up
This shows Compose's internal decision-making process.
Comparing Running vs Desired State
Check configuration drift:
# Compare current state to compose file docker-compose ps # Show differences docker-compose config # Verify configuration matches running state docker-compose up -d --dry-run
Debugging Build Context Issues
Verify build context contents:
# Check what's sent to Docker daemon docker-compose build --progress=plain api # List .dockerignore patterns cat .dockerignore # Verify context size du -sh build-context/
Inspecting Container Capabilities
Check container security capabilities:
# View capabilities
docker inspect --format='{{json .HostConfig.CapAdd}}' $(docker-compose ps -q api)
# Check if privileged
docker inspect --format='{{.HostConfig.Privileged}}' $(docker-compose ps -q api)
# View security options
docker inspect --format='{{json .HostConfig.SecurityOpt}}' $(docker-compose ps -q api)
Best Practices for Debugging
Start simple: Test one service at a time to isolate issues
Use specific tags: Avoid latest tags during debugging for consistency
Check logs first: Most issues reveal themselves in logs
Verify configuration: Use docker-compose config before starting services
Test interactively: Use docker-compose run for one-off debugging sessions:
docker-compose run --rm api sh
Monitor in real-time: Use docker-compose logs -f while reproducing issues
Compare working vs broken: Test same config in different environments
Document findings: Keep notes on what debugging commands revealed
Use verbose output: Enable debug logging when issues are unclear
Check the obvious: Verify files exist, permissions are correct, services are running
Isolate problems: Test components individually before testing the whole system
Reproduce consistently: Ensure you can reliably reproduce issues before debugging
Docker Compose provides comprehensive debugging capabilities through log analysis, container inspection, event monitoring, and interactive execution. By mastering these debugging techniques—from basic log viewing to detailed container state inspection—you can rapidly identify and resolve issues in containerized applications. Understanding container lifecycle events, analyzing resource usage, and leveraging Compose's debugging tools enables effective troubleshooting in both development and production environments.