Docker log management: from docker logs to a real workflow

Docker log management is three problems wearing one hat: where logs go on disk, who can read them, and how you find a line from Tuesday's deploy.

docker logs solves none of those for a team. It solves "what is this container printing right now" for one person with SSH.

Where Docker stores logs

Default driver is json-file. Logs live on the host under /var/lib/docker/containers/.... They grow until rotation kicks in.

Production compose should set logging limits:

services:
  api:
    image: your-api:latest
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "5"

Without max-size, a noisy container can fill the disk and take down every other service on the box. This is the most common "we had no log management" failure mode.

Other drivers:

Changing drivers does not replace a viewer. It changes where bytes land.

What docker logs actually gives you

For a single container on a machine you already SSH into, docker logs is fine:

docker logs -f --tail 200 --timestamps api
docker logs api 2>&1 | grep "request_id=abc"
docker logs --since 30m api

The flags matter during incidents. --tail avoids replaying ten thousand lines. --timestamps helps when you correlate with nginx access logs. --since is faster than piping through tail when you know the deploy window.

The ceiling shows up fast:

• No search across containers without scripting
• No access control (anyone with Docker socket sees everything)
• No audit trail of who read prod
• Remote access means VPN plus SSH plus remembering container names

That is why teams graduate to a viewer before they graduate to Loki.

Three levels of maturity

Level 1: SSH and docker logs

Works for one admin. Breaks for contractors, on-call without laptop VPN, and anyone who needs prod vs staging separation.

If you are here, at least script the repetitive parts. A shell alias for docker logs -f --tail 100 on your main services saves ten minutes a week. Still not a team workflow.

Level 2: Self-hosted log viewer

Browser or native app tailing over WebSockets. One container, Docker socket or remote API, optional auth.

Tools in this bucket:

• DockLog: team RBAC, K8s + Docker, alerts, audit log (why we built it)
• Dozzle: fast Docker tail, multi-agent, strong solo-dev UX (comparison)
• Portainer: logs inside a wider admin UI (comparison)
• Lazydocker: terminal UI for individuals (comparison)

Pick based on people, not features lists. Five developers on one VPS need permissions. One person on a NUC needs speed.

A practical week-one setup for a small team:

1. Deploy the viewer on the same host (or a trusted jump box with socket access)
2. Turn auth on before exposing port 443
3. Create one user per person, not one shared admin password
4. Scope staging users to staging-* container patterns (RBAC guide)
5. Wire one alert for a pattern you already grep manually (alert setup)

Level 3: Centralized log platform

Loki, Elasticsearch, OpenSearch, cloud logging. Indexed search, retention policies, correlation with metrics.

You need this when:

• Incidents start with "search all services for this UUID across two weeks"
• Compliance asks for retention proof
• More than one host generates logs you must query together

You do not need this on day one of a side project. K8s without a logging stack and DockLog vs Loki cover the middle ground.

Many teams run Level 2 for live tail and Level 3 for historical search. That is a valid split. Live debugging happens in a viewer; postmortems happen in Loki.

Log management practices that actually help

• Structured logs. JSON lines parse better than free-text paragraphs. Include request_id, level, service.
• One line per event. Multi-line stack traces are painful in every viewer. Some teams log the summary on one line and attach trace IDs.
• Separate stdout and files. Twelve-factor says stdout. Apps that only write /var/log/app.log inside the container still work, but you must mount or sidecar to get them in Docker's logging pipeline.
• Name containers predictably. acme-api-staging beats f3a8c2. RBAC patterns like acme-* depend on it. RBAC guide.
• Alert on patterns, not on every ERROR string. "Connection refused" during deploy is noise. Same string for ten minutes is an incident. Alert setup.

Structured logging example

Instead of:

ERROR: payment failed for user 42 because card declined

Prefer:

{"level":"error","service":"payments","request_id":"req_8f2a","user_id":42,"reason":"card_declined","msg":"payment failed"}

Viewers can highlight level=error. Alert rules can match service=payments without waking you for errors in a cron job. When you eventually add Loki, labels map cleanly.

Multi-host reality

One VPS is simple. Three VPS instances for staging, prod, and a worker box is where docker logs falls apart entirely.

Options, from light to heavy:

For two or three hosts, many teams run one DockLog instance per environment and keep RBAC patterns identical. That is boring and it works.

When logs live inside the container

Some legacy apps write only to /var/log/app.log. Docker's default logging driver captures stdout/stderr, not arbitrary files.

Fixes:

• Reconfigure the app to log to stdout (best)
• Use a sidecar that tails the file and prints to stdout
• Mount the log directory and tail from the host (awkward with RBAC)

If you skip this, your viewer shows an empty stream while the disk inside the container fills up. Check with docker exec before blaming the viewer.

Security checklist

Any log tool with Docker socket access is root-equivalent on that host.

• TLS in front (Caddy, nginx, Traefik). Reverse proxy guide.
• Auth on, DISABLE_AUTH only on localhost smoke tests
• Per-user container filters on shared machines
• Audit log when multiple people touch production

Logs often contain secrets by accident: connection strings in stack traces, JWT fragments in debug lines, user emails in signup flows. Treat log access like database read access. Contractors get staging patterns, not *.

Docker log management vs monitoring

Logs tell you what the app said. Metrics tell you how full the pipe is. You often need both, but not the same product.

Our Docker monitoring tools roundup maps the full landscape. Health checks cover the layer between "container running" and "app printing errors."

Troubleshooting common pain

Disk full on the host: check docker system df and logging driver rotation first. A container logging stack traces in a loop can eat gigabytes before anyone opens a viewer.

Missing logs in the viewer: confirm the app writes to stdout, not only to a file inside the container. Confirm the container name matches your RBAC pattern.

Logs stop streaming in the browser: reverse proxy probably dropped WebSocket upgrade headers. Fix nginx or Caddy, not the app. Details in the reverse proxy post.

Timestamps look wrong: Docker stores UTC. Your app may log local time. Correlate using request_id, not clock comparison across systems.

Suggested path for a growing team

1. Add max-size / max-file today (five minutes)
2. Deploy a self-hosted viewer with auth this week
3. Wire one alert to Slack for a pattern you already grep manually
4. Revisit Loki or SaaS when search across hosts becomes weekly pain

Log management is not a single purchase. It is rotation, access control, tailing, and maybe search. Nail the first three before you invoice for the fourth.

For how logs fit next to uptime checks and metrics on a budget, see self-hosted monitoring.