Backups and restore tests

A backup you have never restored is a guess. The stack already dumps the database nightly. Your job is the off-box copy, plus the habit that turns a guess into a guarantee: restoring it somewhere safe on a schedule.

1. Check the nightly dump in data/backups/daily/.
2. Copy data/backups/ off the box on a schedule.
3. Restore into a throwaway Postgres monthly.

A second, finer-grained backup lives inside the app (a restorable copy of one workspace, optionally pushed to Google Drive), covered in Backup and export. This page is the whole-box backup an operator keeps, the box underneath it.

What to back up

All of the instance's state lives in bind-mounted folders under ./data/: Postgres, uploaded files, the stack's own nightly dumps, installed modules, and Caddy's certificates. Copying that whole tree is a complete backup:

tar czf cobblr-data-$(date +%F).tar.gz ./data

• Do this with the stack stopped, or accept that a live copy of the database files is a crash-consistent snapshot rather than a clean one.
• For a clean, self-contained database copy that does not need the stack stopped, take a logical dump instead (next section).

The nightly dump the stack already takes

The backup container in the stack runs pg_dumpall once a day into data/backups/daily/. To take one by hand at any moment:

docker compose exec db pg_dumpall -U cobblr | gzip > cobblr-backup-$(date +%F).sql.gz

• Retention: a weekly copy on Sundays, pruned to 30 daily and 12 weekly files.
• A sanity floor: a dump below it is discarded as failed rather than kept, so an empty file can never masquerade as a good night. Retention and the floor are configurable.
• pg_dumpall captures every database on the instance, including each workspace's own database, in one file. It is clean while the stack runs, so there is nothing to schedule and nothing to stop.

Keep more than one, and keep one off the box

1. The rotation is already handled on the box. A single overwritten backup fails you the moment a bad backup overwrites a good one, so the stack keeps a run of dated daily dumps, thinned to weekly further back, and you can reach past a problem you did not notice immediately.
2. The off-box copy is yours. A backup that lives only on the box it protects is gone with the box, so keep at least one copy on a different machine or drive. A cron entry on another machine that pulls data/backups/ (rsync over ssh, rclone to a cloud drive) is enough.
3. The in-app path can also push a workspace backup to Google Drive on a schedule once you configure the operator credentials for it. See Backup and export.

Prove it restores

Restore into a disposable database and check the data is really there. Nothing touches your live instance:

# Start a throwaway Postgres from the same image your stack runs.
docker run -d --name cobblr-restore-test \
  -e POSTGRES_PASSWORD=throwaway -p 54329:5432 \
  ghcr.io/cobblrhq/cobblr-db:${COBBLR_VERSION:-latest}

# Load the most recent dump into it.
gunzip -c cobblr-backup-$(date +%F).sql.gz | \
  docker exec -i cobblr-restore-test psql -U postgres

# Sanity-check: list the restored databases.
docker exec cobblr-restore-test psql -U postgres -c '\l'

# Tear it down.
docker rm -f cobblr-restore-test

• Use the same database image your stack runs (the command above reads your pinned COBBLR_VERSION). A workspace dump can rely on a Postgres extension a stock image does not carry, and a restore that silently skips it is not a real test.
• Run this on a schedule. Monthly is a sensible floor, so silent backup corruption is caught while it is an annoyance rather than a disaster. This is the step people skip and regret, and the difference between "I have backups" and "I can recover".
• For a real recovery, Rollback covers pointing the live instance at a restored copy.