Prince PARK

A comprehensive guide for configuring Traefik to use both static wildcard certificates and dynamic Let’s Encrypt certificates simultaneously.

Table of Contents

• Overview
• How Certificate Selection Works
• Quick Reference Table
• Directory Structure
• Configuration Files
  • 1. Traefik Static Configuration
  • 2. Dynamic TLS Configuration
  • 3. Docker Compose for Traefik
  • 4. Example Application Configurations
• Step-by-Step Setup
• Docker Label Patterns
• Common Scenarios
• Certificate Selection Decision Tree
• Testing and Verification
• Troubleshooting
• Certificate Management
• Security Best Practices
• Advanced Configuration

Overview

This configuration enables Traefik to:

1. Use a static wildcard certificate for all *.apps.abc.com domains
2. Automatically obtain and manage Let’s Encrypt certificates for other domains (e.g., app1.com, app2.com)
3. Automatically select the correct certificate based on the domain being accessed

Use Cases

• Internal applications: Use wildcard certificate for *.apps.abc.com (webapp.apps.abc.com, api.apps.abc.com, etc.)
• Customer-facing domains: Use Let’s Encrypt for app1.com, app2.com, etc.
• Mixed environments: Same application accessible via both internal and external domains

How Certificate Selection Works

Traefik automatically selects certificates using this priority logic:

┌─────────────────────────────────────────────────────────────┐
│ Incoming HTTPS Request for domain: example.com              │
└──────────────────┬──────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────────┐
│ Step 1: Does domain match *.apps.abc.com pattern?           │
└──────────────────┬──────────────────────────────────────────┘
                   │
          ┌────────┴─────────┐
          │                  │
         YES                NO
          │                  │
          ▼                  ▼
    ┌──────────┐      ┌──────────────────────────────────────┐
    │ USE      │      │ Step 2: Is certresolver specified?   │
    │ STATIC   │      └────────┬─────────────────────────────┘
    │ WILDCARD │               │
    │ CERT     │      ┌────────┴──────┐
    └──────────┘      │               │
                     YES             NO
                      │               │
                      ▼               ▼
               ┌──────────────┐  ┌──────────┐
               │ USE/REQUEST  │  │ ERROR    │
               │ LET'S        │  │ or       │
               │ ENCRYPT      │  │ DEFAULT  │
               │ CERT         │  │ CERT     │
               └──────────────┘  └──────────┘

Certificate Matching Examples

Domain	Matches Pattern?	Label Config	Certificate Used
webapp.apps.abc.com	✅ Yes (*.apps.abc.com)	tls=true	Static Wildcard
api.apps.abc.com	✅ Yes (*.apps.abc.com)	tls=true	Static Wildcard
admin.apps.abc.com	✅ Yes (*.apps.abc.com)	tls=true	Static Wildcard
app1.com	❌ No	tls.certresolver=letsencrypt	Let’s Encrypt
app2.com	❌ No	tls.certresolver=letsencrypt	Let’s Encrypt
www.app1.com	❌ No	tls.certresolver=letsencrypt	Let’s Encrypt

Quick Reference Table

The Two Label Patterns

Certificate Type	Domain Examples	Docker Label	Auto-Renewal
Static Wildcard	*.apps.abc.com	tls=true	❌ Manual
Dynamic Let’s Encrypt	app1.com, app2.com	tls.certresolver=letsencrypt	✅ Automatic

Critical Label Difference

✅ CORRECT for *.apps.abc.com:

- "traefik.http.routers.myapp.tls=true" # No certresolver

❌ WRONG for *.apps.abc.com:

- "traefik.http.routers.myapp.tls.certresolver=letsencrypt" # Will bypass wildcard!

✅ CORRECT for app1.com:

- "traefik.http.routers.app1.tls.certresolver=letsencrypt" # With certresolver

❌ WRONG for app1.com:

- "traefik.http.routers.app1.tls=true" # Won't get Let's Encrypt cert

Directory Structure

Create the following directory structure on your host:

traefik/
├── docker-compose.yml                  # Traefik service definition
├── traefik.yml                         # Static configuration
├── dynamic/                            # Dynamic configuration files
│   └── tls-certificates.yml           # Static wildcard certificate config
├── certs/                              # Your static certificates
│   └── wildcard.apps.abc.com/
│       ├── fullchain.pem              # Certificate + intermediate chain
│       └── privkey.pem                # Private key
├── letsencrypt/                        # Let's Encrypt storage (auto-generated)
│   └── acme.json                      # Auto-created, stores LE certificates
└── logs/                               # Traefik logs
    ├── traefik.log
    └── access.log

Configuration Files

1. Traefik Static Configuration

File: traefik.yml

# Traefik Static Configuration
# Main configuration file

global:
  checkNewVersion: true
  sendAnonymousUsage: false

api:
  dashboard: true
  insecure: false

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true

  websecure:
    address: ":443"

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    network: web
    watch: true

  # File provider for static wildcard certificate
  file:
    directory: "/etc/traefik/dynamic"
    watch: true

# Let's Encrypt Configuration for Dynamic Certificates
certificatesResolvers:
  letsencrypt:
    acme:
      email: "[email protected]" # ← CHANGE THIS
      storage: "/letsencrypt/acme.json"

      # HTTP Challenge (recommended for simple setups)
      httpChallenge:
        entryPoint: web

      # OR TLS Challenge (uncomment if preferred)
      # tlsChallenge: {}

      # OR DNS Challenge (uncomment for wildcard LE certs or firewall scenarios)
      # dnsChallenge:
      #   provider: cloudflare  # or your DNS provider
      #   delayBeforeCheck: 0
      #   resolvers:
      #     - "1.1.1.1:53"
      #     - "8.8.8.8:53"

log:
  level: INFO
  filePath: "/var/log/traefik/traefik.log"

accessLog:
  filePath: "/var/log/traefik/access.log"

Important: Change [email protected] to your actual email address for Let’s Encrypt notifications.

2. Dynamic TLS Configuration

File: dynamic/tls-certificates.yml

# Dynamic TLS Configuration
# Defines your static wildcard certificate for *.apps.abc.com

tls:
  certificates:
    # Static WILDCARD certificate for *.apps.abc.com
    - certFile: /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem
      keyFile: /etc/traefik/certs/wildcard.apps.abc.com/privkey.pem
      stores:
        - default

  stores:
    default:
      defaultCertificate:
        # Optional: Set a default fallback certificate
        certFile: /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem
        keyFile: /etc/traefik/certs/wildcard.apps.abc.com/privkey.pem

  options:
    default:
      minVersion: VersionTLS12
      sniStrict: true
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256

3. Docker Compose for Traefik

File: docker-compose.yml

version: "3.8"

services:
  traefik:
    image: traefik:v2.10
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - web
    ports:
      - "80:80"
      - "443:443"
    volumes:
      # Traefik static configuration
      - ./traefik.yml:/traefik.yml:ro

      # Dynamic configuration directory
      - ./dynamic:/etc/traefik/dynamic:ro

      # Static wildcard certificate
      - ./certs:/etc/traefik/certs:ro

      # Let's Encrypt certificate storage
      - ./letsencrypt:/letsencrypt

      # Logs
      - ./logs:/var/log/traefik

      # Docker socket
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      - "traefik.enable=true"

      # Dashboard (uses static wildcard certificate)
      - "traefik.http.routers.traefik.rule=Host(`traefik.apps.abc.com`)"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls=true"

      # Dashboard authentication
      # Generate password: echo $(htpasswd -nb admin yourpassword) | sed -e s/\\$/\\$\\$/g
      - "traefik.http.routers.traefik.middlewares=auth"
      - "traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$8evjzfst$$YgiYLjSK1e5RJTyNvR4QH0"

networks:
  web:
    external: true
# Create the network first: docker network create web

4. Example Application Configurations

File: docker-compose-apps.yml (separate file for your applications)

version: "3.8"

services:
  # ================================================================
  # APPS USING STATIC WILDCARD CERTIFICATE (*.apps.abc.com)
  # ================================================================

  # Example 1: Web application on *.apps.abc.com
  webapp:
    image: nginx:alpine
    container_name: webapp
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.webapp.rule=Host(`webapp.apps.abc.com`)"
      - "traefik.http.routers.webapp.entrypoints=websecure"
      # Key: tls=true WITHOUT certresolver
      - "traefik.http.routers.webapp.tls=true"
      - "traefik.http.services.webapp.loadbalancer.server.port=80"

  # Example 2: API on *.apps.abc.com
  api:
    image: nginx:alpine
    container_name: api
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`api.apps.abc.com`)"
      - "traefik.http.routers.api.entrypoints=websecure"
      - "traefik.http.routers.api.tls=true"
      - "traefik.http.services.api.loadbalancer.server.port=80"

  # Example 3: Admin panel on *.apps.abc.com
  admin:
    image: nginx:alpine
    container_name: admin
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.admin.rule=Host(`admin.apps.abc.com`)"
      - "traefik.http.routers.admin.entrypoints=websecure"
      - "traefik.http.routers.admin.tls=true"
      - "traefik.http.services.admin.loadbalancer.server.port=80"

  # ================================================================
  # APPS USING DYNAMIC LET'S ENCRYPT CERTIFICATES
  # ================================================================

  # Example 4: App1 with Let's Encrypt certificate
  app1:
    image: nginx:alpine
    container_name: app1
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app1.rule=Host(`app1.com`) || Host(`www.app1.com`)"
      - "traefik.http.routers.app1.entrypoints=websecure"
      # Key: tls.certresolver=letsencrypt
      - "traefik.http.routers.app1.tls.certresolver=letsencrypt"
      - "traefik.http.services.app1.loadbalancer.server.port=80"

  # Example 5: App2 with Let's Encrypt certificate
  app2:
    image: nginx:alpine
    container_name: app2
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app2.rule=Host(`app2.com`) || Host(`www.app2.com`)"
      - "traefik.http.routers.app2.entrypoints=websecure"
      - "traefik.http.routers.app2.tls.certresolver=letsencrypt"
      - "traefik.http.services.app2.loadbalancer.server.port=80"

  # Example 6: App3 with Let's Encrypt certificate
  app3:
    image: nginx:alpine
    container_name: app3
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app3.rule=Host(`app3.com`)"
      - "traefik.http.routers.app3.entrypoints=websecure"
      - "traefik.http.routers.app3.tls.certresolver=letsencrypt"
      - "traefik.http.services.app3.loadbalancer.server.port=80"

  # ================================================================
  # SPECIAL CASE: App accessible via both certificate types
  # ================================================================

  # Example 7: App with both internal and external domains
  multiapp:
    image: nginx:alpine
    container_name: multiapp
    restart: unless-stopped
    networks:
      - web
    labels:
      - "traefik.enable=true"

      # Router 1: Internal access (static wildcard)
      - "traefik.http.routers.multiapp-internal.rule=Host(`multi.apps.abc.com`)"
      - "traefik.http.routers.multiapp-internal.entrypoints=websecure"
      - "traefik.http.routers.multiapp-internal.tls=true"
      - "traefik.http.routers.multiapp-internal.service=multiapp"

      # Router 2: External access (Let's Encrypt)
      - "traefik.http.routers.multiapp-external.rule=Host(`multiapp.com`)"
      - "traefik.http.routers.multiapp-external.entrypoints=websecure"
      - "traefik.http.routers.multiapp-external.tls.certresolver=letsencrypt"
      - "traefik.http.routers.multiapp-external.service=multiapp"

      # Shared service definition
      - "traefik.http.services.multiapp.loadbalancer.server.port=80"

networks:
  web:
    external: true

Step-by-Step Setup

Step 1: Create Directory Structure

# Create directories
mkdir -p traefik/{dynamic,certs/wildcard.apps.abc.com,letsencrypt,logs}
cd traefik

Step 2: Place Your Wildcard Certificate

# Copy your wildcard certificate files
cp /path/to/your/wildcard-fullchain.pem certs/wildcard.apps.abc.com/fullchain.pem
cp /path/to/your/wildcard-privkey.pem certs/wildcard.apps.abc.com/privkey.pem

# Verify the files exist
ls -la certs/wildcard.apps.abc.com/

Note: Your certificate must be valid for *.apps.abc.com (wildcard).

Step 3: Set Permissions for Let’s Encrypt Storage

# Create acme.json with correct permissions
touch letsencrypt/acme.json
chmod 600 letsencrypt/acme.json

Critical: The acme.json file must have 600 permissions or Traefik will refuse to start.

Step 4: Create Configuration Files

Create the following files with the content from the Configuration Files section:

1. traefik.yml (main configuration)
2. dynamic/tls-certificates.yml (wildcard certificate definition)
3. docker-compose.yml (Traefik deployment)

Important: Update the email address in traefik.yml:

email: "[email protected]" # ← Change this

Step 5: Create Docker Network

docker network create web

This network will be shared by Traefik and all your applications.

Step 6: Deploy Traefik

# Start Traefik
docker-compose up -d

# Check logs
docker logs traefik

# Follow logs
docker logs -f traefik

Step 7: Verify Traefik is Running

# Check container status
docker ps | grep traefik

# Check if wildcard certificate is loaded
docker logs traefik | grep -i certificate

You should see messages like:

Configuration loaded from file: /traefik.yml
Loading certificate from file: /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem

Step 8: Deploy Your First Application

Create a simple test application:

# test-app.yml
version: "3.8"

services:
  test:
    image: nginx:alpine
    container_name: test-app
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.test.rule=Host(`test.apps.abc.com`)"
      - "traefik.http.routers.test.entrypoints=websecure"
      - "traefik.http.routers.test.tls=true"
      - "traefik.http.services.test.loadbalancer.server.port=80"

networks:
  web:
    external: true

# Deploy test app
docker-compose -f test-app.yml up -d

# Test it (replace with your server IP)
curl -k https://test.apps.abc.com

Step 9: Verify Certificate

# Check which certificate is being served
echo | openssl s_client -servername test.apps.abc.com -connect YOUR_SERVER_IP:443 2>/dev/null | openssl x509 -noout -subject -issuer -dates

Docker Label Patterns

Pattern 1: Static Wildcard Certificate (*.apps.abc.com)

Use this for any subdomain of apps.abc.com:

labels:
  - "traefik.enable=true"
  - "traefik.http.routers.ROUTER_NAME.rule=Host(`subdomain.apps.abc.com`)"
  - "traefik.http.routers.ROUTER_NAME.entrypoints=websecure"
  - "traefik.http.routers.ROUTER_NAME.tls=true" # ← No certresolver
  - "traefik.http.services.SERVICE_NAME.loadbalancer.server.port=PORT"

Key Points:

• Use tls=true (without certresolver)
• Traefik automatically uses the static wildcard certificate
• Works for: webapp.apps.abc.com, api.apps.abc.com, anything.apps.abc.com

Pattern 2: Dynamic Let’s Encrypt Certificate

Use this for custom domains like app1.com, app2.com:

labels:
  - "traefik.enable=true"
  - "traefik.http.routers.ROUTER_NAME.rule=Host(`app1.com`)"
  - "traefik.http.routers.ROUTER_NAME.entrypoints=websecure"
  - "traefik.http.routers.ROUTER_NAME.tls.certresolver=letsencrypt" # ← With certresolver
  - "traefik.http.services.SERVICE_NAME.loadbalancer.server.port=PORT"

Key Points:

• Use tls.certresolver=letsencrypt
• Traefik automatically requests certificate from Let’s Encrypt
• Certificate is automatically renewed
• Can include multiple domains: Host(\app1.com`) || Host(`www.app1.com`)`

Pattern 3: Multiple Domains with Different Certificates

labels:
  - "traefik.enable=true"

  # Router 1: Internal domain (static wildcard)
  - "traefik.http.routers.app-internal.rule=Host(`app.apps.abc.com`)"
  - "traefik.http.routers.app-internal.entrypoints=websecure"
  - "traefik.http.routers.app-internal.tls=true"
  - "traefik.http.routers.app-internal.service=app"

  # Router 2: External domain (Let's Encrypt)
  - "traefik.http.routers.app-external.rule=Host(`app.com`)"
  - "traefik.http.routers.app-external.entrypoints=websecure"
  - "traefik.http.routers.app-external.tls.certresolver=letsencrypt"
  - "traefik.http.routers.app-external.service=app"

  # Service (shared)
  - "traefik.http.services.app.loadbalancer.server.port=3000"

Common Scenarios

Scenario 1: Simple Internal Web Application

services:
  myapp:
    image: myapp:latest
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.rule=Host(`myapp.apps.abc.com`)"
      - "traefik.http.routers.myapp.entrypoints=websecure"
      - "traefik.http.routers.myapp.tls=true"
      - "traefik.http.services.myapp.loadbalancer.server.port=3000"

Result: Uses static wildcard certificate *.apps.abc.com

Scenario 2: Customer-Facing Application

services:
  customer-app:
    image: customer-app:latest
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.customer.rule=Host(`myapp.com`) || Host(`www.myapp.com`)"
      - "traefik.http.routers.customer.entrypoints=websecure"
      - "traefik.http.routers.customer.tls.certresolver=letsencrypt"
      - "traefik.http.services.customer.loadbalancer.server.port=80"

Result: Traefik automatically requests and manages Let’s Encrypt certificate for myapp.com and www.myapp.com

Scenario 3: API with Authentication Middleware

services:
  api:
    image: api:latest
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`api.apps.abc.com`)"
      - "traefik.http.routers.api.entrypoints=websecure"
      - "traefik.http.routers.api.tls=true"
      - "traefik.http.routers.api.middlewares=api-auth"
      - "traefik.http.middlewares.api-auth.basicauth.users=user:$$apr1$$..."
      - "traefik.http.services.api.loadbalancer.server.port=8080"

Result: Uses static wildcard certificate with basic authentication

Scenario 4: Application with Custom Headers

services:
  webapp:
    image: webapp:latest
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.webapp.rule=Host(`webapp.apps.abc.com`)"
      - "traefik.http.routers.webapp.entrypoints=websecure"
      - "traefik.http.routers.webapp.tls=true"
      - "traefik.http.routers.webapp.middlewares=webapp-headers"
      - "traefik.http.middlewares.webapp-headers.headers.customrequestheaders.X-Custom-Header=value"
      - "traefik.http.services.webapp.loadbalancer.server.port=80"

Certificate Selection Decision Tree

User accesses: https://example.apps.abc.com
                        │
                        ▼
┌──────────────────────────────────────────────┐
│ Does "example.apps.abc.com" match            │
│ wildcard pattern "*.apps.abc.com"?           │
└──────────────────┬───────────────────────────┘
                   │
          ┌────────┴─────────┐
          │                  │
        YES                 NO
          │                  │
          ▼                  ▼
┌─────────────────┐  ┌──────────────────────────┐
│ Check router    │  │ Check if router has      │
│ configuration   │  │ certresolver defined     │
└────────┬────────┘  └──────────┬───────────────┘
         │                      │
         ▼                      ▼
┌─────────────────┐  ┌──────────────────────────┐
│ Has tls=true    │  │ certresolver=letsencrypt │
│ (no resolver)?  │  │                          │
└────────┬────────┘  └──────────┬───────────────┘
         │                      │
         ▼                      ▼
┌─────────────────┐  ┌──────────────────────────┐
│ USE STATIC      │  │ Check acme.json for      │
│ WILDCARD CERT   │  │ existing certificate     │
│ *.apps.abc.com  │  └──────────┬───────────────┘
└─────────────────┘             │
                       ┌────────┴─────────┐
                       │                  │
                   EXISTS            DOESN'T EXIST
                       │                  │
                       ▼                  ▼
              ┌─────────────────┐  ┌──────────────┐
              │ USE EXISTING    │  │ REQUEST NEW  │
              │ LET'S ENCRYPT   │  │ CERTIFICATE  │
              │ CERTIFICATE     │  │ FROM LE      │
              └─────────────────┘  └──────────────┘

Testing and Verification

Test 1: Verify Wildcard Certificate is Loaded

# Check Traefik logs for certificate loading
docker logs traefik | grep -i "certificate"

# Expected output:
# Loading certificate from file: /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem

Test 2: Check Certificate Details

# Check what certificate is served for a *.apps.abc.com domain
echo | openssl s_client -servername test.apps.abc.com -connect YOUR_SERVER_IP:443 2>/dev/null | openssl x509 -noout -text | grep -A2 "Subject Alternative Name"

# Expected output for wildcard:
# DNS:*.apps.abc.com

Test 3: Verify Let’s Encrypt Certificate

# Check certificate issuer for a custom domain
echo | openssl s_client -servername app1.com -connect YOUR_SERVER_IP:443 2>/dev/null | openssl x509 -noout -issuer

# Expected output:
# issuer=C=US, O=Let's Encrypt, CN=...

Test 4: Check All Certificates in Traefik

# View Let's Encrypt certificates
docker exec traefik cat /letsencrypt/acme.json | jq '.letsencrypt.Certificates[] | {domains: .domain.main}'

# View static wildcard certificate
docker exec traefik openssl x509 -in /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem -noout -text

Test 5: Test HTTPS Connection

# Test with curl
curl -I https://test.apps.abc.com

# Expected: HTTP/2 200 (or appropriate status)

# Test certificate validation
curl -v https://test.apps.abc.com 2>&1 | grep -E "subject|issuer|SSL"

Test 6: Access Traefik Dashboard

# Access dashboard (replace with your server IP or domain)
https://traefik.apps.abc.com/dashboard/

# Default credentials (if you didn't change them):
# Username: admin
# Password: password (from the bcrypt hash in docker-compose.yml)

Troubleshooting

Issue 1: Wrong Certificate Being Used

Symptom: *.apps.abc.com domain shows Let’s Encrypt certificate instead of wildcard.

Cause: Using tls.certresolver=letsencrypt instead of tls=true

Solution:

# Change this:
- "traefik.http.routers.X.tls.certresolver=letsencrypt"

# To this:
- "traefik.http.routers.X.tls=true"

Issue 2: Certificate Not Loading

Symptom: Traefik logs show errors about certificate files

Checks:

# 1. Verify files exist
docker exec traefik ls -la /etc/traefik/certs/wildcard.apps.abc.com/

# 2. Check dynamic config is loaded
docker exec traefik cat /etc/traefik/dynamic/tls-certificates.yml

# 3. Verify certificate is valid
docker exec traefik openssl x509 -in /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem -noout -text

# 4. Check file permissions
ls -la certs/wildcard.apps.abc.com/

Solution:

• Ensure certificate files are readable (644 permissions)
• Verify paths in tls-certificates.yml are correct
• Check certificate is valid and not expired

Issue 3: Let’s Encrypt Certificate Not Being Requested

Symptom: App accessible via HTTP but not HTTPS, or shows default certificate

Checks:

# 1. Check if certresolver is specified
docker inspect CONTAINER_NAME | grep certresolver

# 2. Check Traefik logs
docker logs traefik | grep -i "acme\|certificate\|error"

# 3. Verify acme.json permissions
ls -la letsencrypt/acme.json
# Must be: -rw------- (600)

Solution:

# Ensure you have:
- "traefik.http.routers.X.tls.certresolver=letsencrypt"

# Not:
- "traefik.http.routers.X.tls=true"

Issue 4: Let’s Encrypt Rate Limit Error

Symptom: Error in logs: too many certificates already issued

Cause: Let’s Encrypt rate limits (50 certificates per domain per week)

Solution:

# Use staging environment for testing
# In traefik.yml:
certificatesResolvers:
  letsencrypt:
    acme:
      caServer: https://acme-staging-v02.api.letsencrypt.org/directory
      # ... rest of config

Note: Staging certificates will show as untrusted in browsers. Remove caServer line for production.

Issue 5: Container Not Getting Certificate

Symptom: Container accessible via HTTP but not HTTPS

Checklist:

• ✅ Container is on the web network
• ✅ traefik.enable=true label exists
• ✅ Entry point is set to websecure
• ✅ TLS is properly configured
• ✅ DNS points to your server
• ✅ Ports 80 and 443 are open

Verification:

# Check container is on correct network
docker inspect CONTAINER_NAME | grep NetworkMode

# Check container labels
docker inspect CONTAINER_NAME | jq '.[0].Config.Labels'

# Check Traefik sees the container
docker logs traefik | grep CONTAINER_NAME

Issue 6: “Unable to obtain ACME certificate” Error

Symptom: Traefik logs show ACME errors

Common Causes:

1. Port 80 not accessible (for HTTP challenge)
2. DNS not pointing to server
3. Firewall blocking traffic
4. Invalid email address

Solution:

# Test port 80 is accessible
curl -I http://YOUR_SERVER_IP

# Test DNS resolution
dig app1.com
nslookup app1.com

# Check firewall
sudo ufw status
sudo iptables -L

Issue 7: Self-Signed Certificate Warning

Symptom: Browser shows “Your connection is not private”

Causes:

1. Using Let’s Encrypt staging environment
2. Certificate not yet issued
3. Wrong certificate being served

Solution:

# 1. Check which certificate is being served
echo | openssl s_client -servername YOUR_DOMAIN -connect YOUR_SERVER_IP:443 2>/dev/null | openssl x509 -noout -issuer

# 2. Wait a few minutes for Let's Encrypt issuance
# Check logs:
docker logs traefik | grep -i acme

# 3. If using staging, remove caServer line from traefik.yml

Issue 8: Wildcard Certificate Not Covering Subdomain

Symptom: Certificate error for domain that should be covered

Important: Wildcard certificates *.apps.abc.com are valid for:

• ✅ test.apps.abc.com
• ✅ api.apps.abc.com
• ✅ anything.apps.abc.com
• ❌ apps.abc.com (base domain)
• ❌ sub.test.apps.abc.com (two levels deep)

Solution:

• For base domain: Get separate certificate or include both apps.abc.com and *.apps.abc.com in certificate
• For multi-level: Get certificate that includes multiple levels or use separate certificate

Certificate Management

Updating the Wildcard Certificate

When your wildcard certificate expires or needs renewal:

# 1. Replace certificate files
cp /path/to/new-fullchain.pem certs/wildcard.apps.abc.com/fullchain.pem
cp /path/to/new-privkey.pem certs/wildcard.apps.abc.com/privkey.pem

# 2. Traefik automatically reloads (due to watch: true)
# Verify reload in logs:
docker logs traefik | tail -20

# 3. No restart needed!

Let’s Encrypt Certificate Renewal

Let’s Encrypt certificates are automatically renewed by Traefik:

• Renewal starts 30 days before expiration
• Happens automatically in the background
• No manual intervention required
• Stored in letsencrypt/acme.json

Monitor renewals:

# Check certificate expiration dates
docker exec traefik cat /letsencrypt/acme.json | jq '.letsencrypt.Certificates[] | {domain: .domain.main, cert: .certificate}' | head -50

# Check Traefik logs for renewal activity
docker logs traefik | grep -i "renew"

Certificate Expiration Monitoring

# Check wildcard certificate expiration
docker exec traefik openssl x509 -in /etc/traefik/certs/wildcard.apps.abc.com/fullchain.pem -noout -enddate

# Check specific Let's Encrypt certificate (from outside)
echo | openssl s_client -servername app1.com -connect YOUR_SERVER_IP:443 2>/dev/null | openssl x509 -noout -dates

Backup Important Files

What to backup:

# 1. Let's Encrypt certificates
cp letsencrypt/acme.json letsencrypt/acme.json.backup

# 2. Static wildcard certificate
tar -czf wildcard-cert-backup.tar.gz certs/

# 3. Configuration files
tar -czf config-backup.tar.gz traefik.yml dynamic/ docker-compose.yml

Restore from backup:

# Restore acme.json
cp letsencrypt/acme.json.backup letsencrypt/acme.json
chmod 600 letsencrypt/acme.json

# Restart Traefik
docker-compose restart traefik

Security Best Practices

1. File Permissions

# acme.json must be 600
chmod 600 letsencrypt/acme.json

# Certificate files should be 644
chmod 644 certs/wildcard.apps.abc.com/*.pem

# Private keys should be 600
chmod 600 certs/wildcard.apps.abc.com/privkey.pem

2. Use Read-Only Mounts

In docker-compose.yml, use :ro for static files:

volumes:
  - ./traefik.yml:/traefik.yml:ro
  - ./dynamic:/etc/traefik/dynamic:ro
  - ./certs:/etc/traefik/certs:ro

3. Enable Dashboard Authentication

Always protect the Traefik dashboard:

# Generate password hash
echo $(htpasswd -nb admin yourpassword) | sed -e s/\\$/\\$\\$/g

# Add to docker-compose.yml
- "traefik.http.middlewares.auth.basicauth.users=admin:HASH"

4. Use Strong TLS Configuration

Already configured in tls-certificates.yml:

• Minimum TLS 1.2
• Strong cipher suites
• SNI strict mode enabled

5. Regular Updates

# Update Traefik image
docker-compose pull
docker-compose up -d

# Check for updates
docker images | grep traefik

6. Monitor Logs

# Watch for security issues
docker logs -f traefik | grep -i "error\|warning\|fail"

# Set up log rotation
# Add to /etc/logrotate.d/traefik:
/path/to/traefik/logs/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
}

7. Firewall Configuration

# Allow only necessary ports
sudo ufw allow 80/tcp   # HTTP
sudo ufw allow 443/tcp  # HTTPS
sudo ufw enable

# Block Traefik dashboard from internet (if needed)
# Access via VPN or SSH tunnel

8. Use Secrets for Sensitive Data

For production, use Docker secrets or environment variables:

environment:
  - CLOUDFLARE_EMAIL=${CLOUDFLARE_EMAIL}
  - CLOUDFLARE_API_KEY=${CLOUDFLARE_API_KEY}

Advanced Configuration

Using DNS Challenge for Let’s Encrypt

If you need wildcard Let’s Encrypt certificates or are behind a firewall:

# In traefik.yml
certificatesResolvers:
  letsencrypt:
    acme:
      email: "[email protected]"
      storage: "/letsencrypt/acme.json"
      dnsChallenge:
        provider: cloudflare # or your DNS provider
        delayBeforeCheck: 0
        resolvers:
          - "1.1.1.1:53"
          - "8.8.8.8:53"

Required environment variables:

# In docker-compose.yml
environment:
  - [email protected]
  - CF_API_KEY=your-api-key
  # OR
  - CF_DNS_API_TOKEN=your-api-token

Supported DNS providers: Cloudflare, Route53, DigitalOcean, OVH, GoDaddy, and many more

Multiple Certificate Resolvers

certificatesResolvers:
  letsencrypt-http:
    acme:
      email: "[email protected]"
      storage: "/letsencrypt/acme-http.json"
      httpChallenge:
        entryPoint: web

  letsencrypt-dns:
    acme:
      email: "[email protected]"
      storage: "/letsencrypt/acme-dns.json"
      dnsChallenge:
        provider: cloudflare

Use in labels:

- "traefik.http.routers.X.tls.certresolver=letsencrypt-http"
# or
- "traefik.http.routers.Y.tls.certresolver=letsencrypt-dns"

Custom Middleware

Rate Limiting:

labels:
  - "traefik.http.middlewares.ratelimit.ratelimit.average=100"
  - "traefik.http.middlewares.ratelimit.ratelimit.burst=50"
  - "traefik.http.routers.X.middlewares=ratelimit"

IP Whitelist:

labels:
  - "traefik.http.middlewares.ipwhitelist.ipwhitelist.sourcerange=192.168.1.0/24,172.16.0.0/16"
  - "traefik.http.routers.X.middlewares=ipwhitelist"

CORS Headers:

labels:
  - "traefik.http.middlewares.cors.headers.accesscontrolallowmethods=GET,OPTIONS,PUT,POST,DELETE"
  - "traefik.http.middlewares.cors.headers.accesscontrolalloworigin=*"
  - "traefik.http.routers.X.middlewares=cors"

Automatic HTTPS Redirect

Already configured in traefik.yml, but you can customize:

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true # 301 redirect

Custom Default Certificate

If you want a different default certificate:

# In tls-certificates.yml
tls:
  stores:
    default:
      defaultCertificate:
        certFile: /etc/traefik/certs/default/fullchain.pem
        keyFile: /etc/traefik/certs/default/privkey.pem

Additional Resources

Official Documentation

• Traefik Documentation
• Let’s Encrypt Documentation
• Docker Documentation

Useful Tools

• htpasswd: Generate password hashes for basic auth
• openssl: Test SSL/TLS certificates
• curl: Test HTTP/HTTPS endpoints
• jq: Parse JSON (for acme.json)

Certificate Authorities

• Let’s Encrypt – Free automated certificates
• Let’s Encrypt Rate Limits
• SSL Labs Server Test – Test your SSL configuration

Quick Command Reference

# Create structure
mkdir -p traefik/{dynamic,certs/wildcard.apps.abc.com,letsencrypt,logs}

# Set permissions
chmod 600 letsencrypt/acme.json

# Create network
docker network create web

# Start Traefik
docker-compose up -d

# View logs
docker logs -f traefik

# Check certificate
echo | openssl s_client -servername DOMAIN -connect IP:443 2>/dev/null | openssl x509 -noout -text

# Restart Traefik
docker-compose restart traefik

# Update Traefik
docker-compose pull && docker-compose up -d

# Backup acme.json
cp letsencrypt/acme.json letsencrypt/acme.json.backup

# Generate auth hash
echo $(htpasswd -nb admin password) | sed -e s/\\$/\\$\\$/g

# View Let's Encrypt certificates
docker exec traefik cat /letsencrypt/acme.json | jq

Summary

This configuration provides a robust solution for managing both static and dynamic certificates in Traefik:

1. Static wildcard certificate (*.apps.abc.com): Manually managed, used for internal applications
2. Dynamic Let’s Encrypt certificates: Automatically obtained and renewed for customer-facing domains
3. Automatic certificate selection: Traefik intelligently chooses the right certificate based on domain and labels

Key Takeaways

• Use tls=true for *.apps.abc.com domains (static wildcard)
• Use tls.certresolver=letsencrypt for other domains (dynamic Let’s Encrypt)
• Traefik handles all certificate selection automatically
• Let’s Encrypt certificates are automatically renewed
• Static wildcard certificate must be manually renewed

The Two-Label Rule

Remember this simple rule:

Domain Type	Label	Certificate
*.apps.abc.com	tls=true	Static Wildcard
Everything else	tls.certresolver=letsencrypt	Let’s Encrypt

Last Updated: January 2025