Backups and restore

PostgreSQL and RustFS are separate authoritative stores and form one recovery set. Back up both under the same timestamp before upgrades or other risky changes. Redis is a reconstructible cache and is not part of the authoritative pair.

Replace <ts> with one timestamped directory name before running any command. Do not execute the angle-bracket placeholder literally.

PostgreSQL backup

Run from ~/dev/lobe:

sudo -n docker exec lobe-postgres pg_dump -U postgres --format=custom -f /tmp/lobechat.dump lobechat && sudo -n docker cp lobe-postgres:/tmp/lobechat.dump backups/<ts>/lobechat.dump && sudo -n docker exec lobe-postgres rm /tmp/lobechat.dump

This produces a custom-format logical dump outside the Docker volume. Verify that PostgreSQL can read its catalog:

pg_restore --list backups/<ts>/lobechat.dump

A file that merely exists is not sufficient; pg_restore --list must complete successfully.

RustFS backup

The backup client uses the exact pinned minio/mc digest from Compose, joins the private runtime network, reads credentials from .env without printing them, and mirrors bucket lobe to the paired backup directory:

sudo -n docker run --rm --network lobe_lobe-network --env-file .env -v "$PWD/backups/<ts>/rustfs:/backup" --entrypoint /bin/sh minio/mc:latest@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727 -c 'mc alias set rustfs http://rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" && mc mirror rustfs/lobe /backup/lobe'

Verify the mirrored tree:

ls -R backups/<ts>/rustfs/lobe

The listing must show the expected object tree. Do not enable shell tracing around the --env-file command.

Recovery set register

For each <ts>, retain together:

• backups/<ts>/lobechat.dump
• backups/<ts>/rustfs/lobe/
• application image identity expected by the data:
  • lobehub/lobehub:2.2.15@sha256:9186fbb2a1cd593eada518560695e1f14f0fa570ed987ba1bf1b3008456422ca for the currently documented release

A Docker volume, volume snapshot name, or running container is not a backup receipt.

Restore prerequisites

Before restore:

• select the application digest matched to the backup
• bring the stack to a state where PostgreSQL and RustFS are healthy
• make the selected PostgreSQL dump available inside lobe-postgres
• make the selected RustFS mirror available to the pinned minio/mc container
• ensure application writes cannot race the restore
• understand that --clean --if-exists replaces database objects in the selected database

Do not delete volumes as a rollback mechanism.

PostgreSQL restore

Copy the selected dump into the PostgreSQL container, restore it into lobechat, then remove the temporary copy:

sudo -n docker cp backups/<ts>/lobechat.dump lobe-postgres:/tmp/lobechat.dump
sudo -n docker exec lobe-postgres pg_restore -U postgres -d lobechat --clean --if-exists /tmp/lobechat.dump
sudo -n docker exec lobe-postgres rm /tmp/lobechat.dump

The required restore operation is pg_restore -U postgres -d lobechat --clean --if-exists /tmp/lobechat.dump inside lobe-postgres.

RustFS restore

Reverse the mirror direction with the same pinned client and private network:

sudo -n docker run --rm --network lobe_lobe-network --env-file .env -v "$PWD/backups/<ts>/rustfs:/backup" --entrypoint /bin/sh minio/mc:latest@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727 -c 'mc alias set rustfs http://rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" && mc mirror /backup/lobe rustfs/lobe'

After both restores, reconcile the matched application digest and repeat verification. The bucket policy still makes restored object URLs anonymously readable; writes remain signed.

Upgrade rollback boundary

• application rollback:
  • restore the previous pinned image reference and run sudo -n docker compose up -d
• database rollback after an incompatible migration:
  • restore the matched pre-upgrade PostgreSQL dump
  • restore the matching RustFS mirror when object state changed
• forbidden shortcut:
  • deleting lobe_postgres_data or lobe_rustfs_data