Docker Compositor Role

Overview

This role deploys and manages Docker Compose stacks with intelligent change detection, database initialization, and monitoring integration. It creates the necessary directory structure, generates docker-compose.yml from templates, implements idempotency checks to prevent unnecessary restarts, schedules Centreon monitoring downtimes during updates, and handles database initialization for PostgreSQL and MariaDB/MySQL containers.

Purpose

• Declarative Docker Stacks: Define entire Docker Compose stacks as Ansible variables
• Intelligent Updates: Only restart containers when compose file actually changes
• Database Initialization: Automatically create databases and users on first run
• Monitoring Integration: Schedule Centreon downtimes during container updates
• Volume Management: Create and configure all required volume directories
• Environment Variables: Manage sensitive configuration via .env files
• Service Orchestration: Start databases first, then dependent services
• Clean Updates: Stop containers before applying compose file changes
• Idempotent: Safe to run repeatedly without unnecessary disruption

Requirements

• Ansible 2.9 or higher
• Collection: community.docker (for docker_compose_v2, docker_container_exec modules)
• Docker installed on target host (see docker_install role)
• Docker Compose V2 installed (docker compose, not legacy docker-compose)
• Centreon monitoring server (for downtime scheduling, optional)
• Variables: docker_compose_directory, docker_data_path, docker_user, docker_compose_file, docker_compose_stack_name

Role Variables

Required Variables (typically from host_vars)

Variable	Required	Description
docker_compose_directory	Yes	Directory containing docker-compose.yml
docker_data_path	Yes	Base path for container data volumes
docker_user	Yes	User that owns Docker files and volumes
docker_compose_file	Yes	Name of the compose file (e.g., docker-compose.yml)
docker_compose_stack_name	Yes	Docker Compose project name
docker_compositor_definition	Yes	Complete service definition structure

Optional Variables

Variable	Default	Description
docker_compositor_postgres_databases	[]	List of PostgreSQL databases to create
docker_compositor_mysql_databases	[]	List of MariaDB/MySQL databases to create
docker_compositor_downtime_duration_minutes	60	Centreon downtime duration (minutes)
centreon_host_name	Auto-detected	Host name in Centreon for downtime scheduling
vault_centreon_admin_password	From vault	Centreon admin password (for CLAPI)
vault_docker_mysql_root_password	From vault	MySQL root password (for database creation)

Variable Details

docker_compositor_definition

The core variable that defines your entire Docker Compose stack.

Structure:

docker_compositor_definition:
  state: present  # or 'stopped', 'absent'

  services:
    - name: service_name
      image: docker/image:tag
      container_name: unique_container_name
      restart_policy: unless-stopped

      # Optional: Volumes
      volumes:
        - folder_path_on_host: /path/on/host
          folder_path_in_container: /path/in/container
          owner: docker_user
          group: docker_user
          mode: '0755'
          create_folder: true  # Default: true

      # Optional: Environment variables
      environment:
        PUBLIC_VAR: "value"

      # Optional: Sensitive environment variables (.env file)
      sensitive_env_vars:
        SECRET_KEY: "{{ vault_secret }}"
        PASSWORD: "{{ vault_password }}"

      # Optional: Ports
      ports:
        - "8080:80"
        - "443:443"

      # Optional: Networks
      networks:
        - network_name

      # Optional: Depends on
      depends_on:
        - database_service

      # Optional: Health check
      healthcheck:
        test: ["CMD", "curl", "-f", "http://localhost"]
        interval: 30s
        timeout: 10s
        retries: 3

      # Optional: Labels
      labels:
        description: "Service description"

Complete example:

docker_compositor_definition:
  state: present

  services:
    - name: postgresql
      image: postgres:15-alpine
      container_name: postgresql
      restart_policy: unless-stopped

      volumes:
        - folder_path_on_host: "{{ docker_data_path }}/postgresql/data"
          folder_path_in_container: /var/lib/postgresql/data
          owner: "{{ docker_user }}"
          group: "{{ docker_user }}"
          mode: '0755'

      sensitive_env_vars:
        POSTGRES_USER: postgres
        POSTGRES_PASSWORD: "{{ vault_postgres_password }}"

      networks:
        - app_network

    - name: webapp
      image: myapp/webapp:latest
      container_name: webapp
      restart_policy: unless-stopped

      volumes:
        - folder_path_on_host: "{{ docker_data_path }}/webapp/config"
          folder_path_in_container: /app/config
          owner: "{{ docker_user }}"
          group: "{{ docker_user }}"
          mode: '0755'

      environment:
        APP_ENV: production
        LOG_LEVEL: info

      sensitive_env_vars:
        DATABASE_URL: "postgresql://user:{{ vault_db_password }}@postgresql:5432/webapp"
        SECRET_KEY: "{{ vault_app_secret }}"

      ports:
        - "8080:8080"

      depends_on:
        - postgresql

      networks:
        - app_network

docker_compositor_postgres_databases

List of PostgreSQL databases and users to create automatically.

Structure:

docker_compositor_postgres_databases:
  - container: postgresql_container_name
    database: database_name
    user: database_user
    password: "{{ vault_db_password }}"

Example:

docker_compositor_postgres_databases:
  - container: postgresql
    database: immich
    user: immich
    password: "{{ vault_immich_db_password }}"

  - container: postgresql
    database: paperless
    user: paperless
    password: "{{ vault_paperless_db_password }}"

docker_compositor_mysql_databases

List of MariaDB/MySQL databases and users to create automatically.

Structure:

docker_compositor_mysql_databases:
  - container: mariadb_container_name
    database: database_name
    user: database_user
    password: "{{ vault_db_password }}"

Example:

docker_compositor_mysql_databases:
  - container: mariadb
    database: nextcloud
    user: nextcloud
    password: "{{ vault_nextcloud_db_password }}"

Dependencies

This role requires:

• docker_install role (to install Docker and Compose V2)
• Centreon server configured (for optional downtime scheduling)
• Ansible Vault for sensitive variables (database passwords, API keys)

Example Playbook

Basic Usage

---
- name: Deploy Docker Compose Stack
  hosts: docker
  become: true

  roles:
    - docker_compositor

With Database Initialization

---
- name: Deploy Stack with PostgreSQL
  hosts: docker
  become: true

  vars:
    docker_compositor_postgres_databases:
      - container: postgresql
        database: myapp
        user: myapp
        password: "{{ vault_myapp_db_password }}"

  roles:
    - docker_compositor

Multiple Services

---
- name: Deploy Complete Application Stack
  hosts: docker
  become: true

  vars:
    docker_compositor_definition:
      state: present
      services:
        - name: postgresql
          image: postgres:15-alpine
          container_name: postgresql
          # ... (see full example above)

        - name: redis
          image: redis:7-alpine
          container_name: redis
          # ... (configuration)

        - name: webapp
          image: myapp/webapp:latest
          container_name: webapp
          # ... (configuration)

    docker_compositor_postgres_databases:
      - container: postgresql
        database: webapp
        user: webapp
        password: "{{ vault_webapp_db_password }}"

  roles:
    - docker_compositor

What This Role Does

Step 1: Prerequisites

1. Creates compose directory (e.g., /home/user/docker/compose/)
2. Creates volume directories for all services with proper ownership
3. Creates Glance config directory (if Glance service present)
4. Copies Glance configuration files (glance.yml, home.yml)
5. Creates .env files for services with sensitive variables (mode 0600)

Step 2: Idempotency Check

1. Checks if current compose file exists
2. Generates temporary compose file from template
3. Calculates SHA256 checksums for both files
4. Compares checksums to detect changes
5. Deletes temporary file
6. If changes detected:
   • Calculate downtime window (start time + duration)
   • Schedule Centreon downtime for “Docker Containers Uptime” service
   • Stop all containers cleanly

Step 3: Installation/Update

1. Deploys docker-compose.yml from template
2. Starts database containers first (postgresql, mariadb)
3. Waits 15 seconds for databases to initialize
4. Initializes PostgreSQL databases:
   • Check if user exists (idempotent)
   • Create user if needed
   • Check if database exists (idempotent)
   • Create database if needed with correct owner
5. Initializes MariaDB/MySQL databases:
   • Create database if not exists
   • Create user if not exists
   • Grant privileges
   • Flush privileges
6. Starts all services with Docker Compose V2:
   • Project name: docker_compose_stack_name
   • Remove orphans: true (cleanup old services)
   • Recreate: auto (only if changed)
   • Pull: missing (pull images if not present)
   • Dependencies: true (respect depends_on)
   • Wait: true (wait for health checks)
   • Wait timeout: 600 seconds (10 minutes)

Idempotency Check Details

Why Checksum Comparison?

Docker Compose restarts containers even if the compose file hasn’t actually changed (timestamps, comments, whitespace). The role:

1. Generates what the new compose file would be
2. Compares SHA256 checksum with current file
3. Only stops/restarts if content actually changed

Benefits

• Prevents unnecessary downtime: Containers keep running if no changes
• Reduces monitoring noise: No false alerts from unnecessary restarts
• Faster playbook runs: Skip compose operations if no changes
• Better uptime: Services only restart when needed

Centreon Downtime Scheduling

When changes detected:

downtime_start: "2026/01/07 14:30"  # Current time
downtime_end: "2026/01/07 15:30"    # Current time + duration
duration: 3600 seconds (60 minutes)
comment: "Docker compose file update"

Prevents monitoring alerts during planned container restarts.

Database Initialization

PostgreSQL Initialization

The role uses idempotent SQL queries:

-- Check if user exists, create only if not
SELECT 1 FROM pg_user WHERE usename = 'myapp' | grep -q 1 ||
CREATE USER myapp WITH PASSWORD 'password';

-- Check if database exists, create only if not
SELECT 1 FROM pg_database WHERE datname = 'myapp' | grep -q 1 ||
CREATE DATABASE myapp OWNER 'myapp';

Features:

• Won’t fail if user/database already exists
• Safe to run multiple times
• Sets correct ownership
• Executes inside PostgreSQL container via docker_container_exec

MySQL/MariaDB Initialization

CREATE DATABASE IF NOT EXISTS myapp;
CREATE USER IF NOT EXISTS 'myapp'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON myapp.* TO 'myapp'@'%';
FLUSH PRIVILEGES;

Features:

• Uses IF NOT EXISTS (idempotent)
• Creates user with wildcard host (% - any container)
• Grants all privileges on specific database only
• Flushes privileges to apply immediately

Database Startup Wait

15-second pause after starting database containers ensures:

• Database server fully initialized
• Ready to accept connections
• Prevents initialization failures due to “connection refused”

Docker Compose V2 Features

The role uses modern docker compose (V2):

Feature	Value	Purpose
project_name	Set	Consistent stack naming
project_src	Set	Directory with compose file
state	present/stopped	Desired service state
remove_orphans	true	Cleanup removed services
recreate	auto	Recreate only if changed
pull	missing	Pull only if image missing
dependencies	true	Respect depends_on
wait	true	Wait for health checks
wait_timeout	600	Health check timeout (10 min)

Environment Variable Management

Public Variables (in compose file)

Defined in environment section:

environment:
  APP_ENV: production
  LOG_LEVEL: info
  PUBLIC_API_URL: https://api.example.com

Visible in: docker-compose.yml, docker inspect, logs

Sensitive Variables (.env files)

Defined in sensitive_env_vars section:

sensitive_env_vars:
  DATABASE_PASSWORD: "{{ vault_db_password }}"
  API_KEY: "{{ vault_api_key }}"
  SECRET_KEY: "{{ vault_secret }}"

Stored as: {service_name}.env with mode 0600 (owner read/write only)

Referenced in compose:

env_file:
  - service_name.env

Security: Not visible in compose file, protected file permissions

Volume Directory Creation

The role automatically creates volume directories:

volumes:
  - folder_path_on_host: /home/user/docker/data/webapp/config
    folder_path_in_container: /app/config
    owner: user
    group: user
    mode: '0755'
    create_folder: true  # Default, can set to false to skip

Features:

• Creates all parent directories
• Sets correct ownership (prevents permission errors)
• Configurable permissions per volume
• Optional creation (set create_folder: false to skip)

Glance Dashboard Integration

Role includes special handling for Glance (dashboard service):

1. Creates /glance/config/ directory
2. Copies glance.yml (main configuration)
3. Copies home.yml (dashboard layout)

Files location: roles/docker_compositor/files/glance.yml

Security Considerations

• Sensitive Variables: Stored in .env files with mode 0600
• No Logging: Database passwords use no_log: true
• Vault Storage: All secrets should be in Ansible Vault
• User Ownership: All files owned by docker_user, not root
• Centreon Password: Protected with no_log: true in downtime command
• Database Passwords: Never in compose file or logs
• File Permissions: .env files readable only by owner

Tags

This role does not define any tags. Use playbook-level tags if needed:

- hosts: docker
  roles:
    - docker_compositor
  tags:
    - docker
    - containers
    - compose
    - services

Notes

• Requires Docker Compose V2 (docker compose, not docker-compose)
• Database initialization is idempotent (safe to run multiple times)
• Idempotency check prevents unnecessary container restarts
• Centreon downtime scheduling is optional (fails gracefully if unavailable)
• Service start order: databases first, then dependent services
• Health checks respected with 10-minute timeout
• Orphan containers automatically removed
• Images pulled only if missing (not on every run)

Troubleshooting

”docker compose command not found”

Cause: Docker Compose V2 not installed

Solution:

# Install Docker Compose V2
apt-get install docker-compose-plugin  # Debian
dnf install docker-compose-plugin      # RedHat

# Or run docker_install role first
ansible-playbook site.yml --tags docker

Database initialization fails with “connection refused”

Cause: Database not ready after 15-second wait

Solution: Increase wait time in install.yml:

- name: Wait for database containers to be fully ready
  ansible.builtin.pause:
    seconds: 30  # Increase from 15 to 30

Containers don’t start after update

Check logs:

# View compose logs
cd /home/user/docker/compose/
docker compose logs

# Check specific service
docker compose logs webapp

# Check container status
docker compose ps

Common issues:

• Missing .env file (check file created with correct name)
• Wrong volume paths (check directories created)
• Port conflicts (check docker ps for port usage)
• Image pull failure (check docker compose pull)

Permission denied errors in containers

Cause: Volume directory ownership incorrect

Solution:

# Check directory ownership
ls -la /home/user/docker/data/webapp/

# Fix ownership manually
chown -R user:user /home/user/docker/data/webapp/

# Re-run role to fix
ansible-playbook site.yml --tags docker

PostgreSQL: “database already exists” error

Should not happen - role uses idempotent queries.

If it occurs:

# Check database exists
docker exec postgresql psql -U postgres -l

# Role should detect existing database and skip creation

Centreon downtime scheduling fails

Not critical - role continues even if downtime fails.

Check:

# Verify Centreon accessible from Docker host
ping centreon-ip

# Test CLAPI manually
ssh centreon
centreon -u admin -p 'password' -o HOST -a show

Compose file not updating despite changes

Check:

# View current checksum
sha256sum /home/user/docker/compose/docker-compose.yml

# Generate new and compare
ansible-playbook site.yml --check  # Dry run

Force update:

# Delete compose file to force regeneration
rm /home/user/docker/compose/docker-compose.yml
ansible-playbook site.yml

Testing the Role

Verify Prerequisites

# Check Docker installed
docker --version

# Check Compose V2
docker compose version

# Check directories created
ls -la /home/user/docker/compose/
ls -la /home/user/docker/data/

Verify Services Running

cd /home/user/docker/compose/

# List running containers
docker compose ps

# Should show all services "Up"
# Example:
# NAME        IMAGE                  STATUS
# postgresql  postgres:15-alpine     Up 5 minutes
# webapp      myapp/webapp:latest    Up 5 minutes

Test Database Initialization

# PostgreSQL
docker exec postgresql psql -U postgres -c "\l"  # List databases
docker exec postgresql psql -U postgres -c "\du"  # List users

# MySQL/MariaDB
docker exec mariadb mysql -u root -p'password' -e "SHOW DATABASES;"
docker exec mariadb mysql -u root -p'password' -e "SELECT user FROM mysql.user;"

Test Idempotency

# Run role twice
ansible-playbook site.yml
# (containers start)

ansible-playbook site.yml
# (should show "changed=0" for compose task if no changes)

Test Updates

# Modify service in host_vars
docker_compositor_definition:
  services:
    - name: webapp
      image: myapp/webapp:latest
      # Add new environment variable
      environment:
        NEW_VAR: "new_value"

# Run role
ansible-playbook site.yml

# Role should:
# 1. Detect compose file changed
# 2. Schedule Centreon downtime
# 3. Stop containers
# 4. Update compose file
# 5. Restart containers with new configuration

Best Practices

1. Use Ansible Vault for all sensitive variables
2. Pin image versions in production (not :latest)
3. Test in development before deploying to production
4. Monitor container logs after deployment
5. Document services in comments or external docs
6. Use health checks for critical services
7. Set restart policies appropriately (unless-stopped for production)
8. Back up volumes regularly (see docker_data_backup role)
9. Review .env files for correct variable names
10. Keep databases on persistent volumes (not named volumes in compose)

Related Roles

This role is often used with:

• docker_install: Install Docker and Compose V2 prerequisites
• docker_data_backup: Back up Docker volumes to NAS
• docker_data_restore: Restore Docker volumes from backup
• deploy_system_monitoring: Monitor container health with NRPE
• telegraf_agent: Collect Docker metrics for InfluxDB/Grafana

License

MIT

Author

Created for homelab infrastructure management.