Operations runbook

Backup & disaster recovery

Cerynix encrypts database and evidence backups with age before writing them. A usable recovery plan also requires off-box copies, an off-box private identity and regular restore tests.

Create and protect backups

backup

cd /opt/cerynix/app
scripts/backup.sh

Expect db.sql.gz.age and evidence.tar.age. Confirm both exist, monitor the scheduled job, and copy them off the VM. Store BACKUP_AGE_IDENTITY separately; losing it makes the encrypted backup unrecoverable.

./install.sh --update also writes its own encrypted pre-upgrade database dump (cerynix-db-backup-<ts>.sql.gz.age, encrypted to the same public recipient). It is a database-only recovery point — decrypt it with the off-box private identity the same way — and does not replace the full off-box backup and restore drill above.

Prove recovery

isolated drill

BACKUP_AGE_IDENTITY=/path/to/cerynix-backup.key \
  scripts/restore-drill.sh

Run drills on a schedule and record RPO, RTO and the application checks performed. Never use production as the first restore test.

Recover production

Declare the incident, select the recovery point and preserve failed-system logs and disks.
Stop user writes and isolate the target. Verify the chosen backup and matching application release in a recovery environment.

Restore only during an approved maintenance window:

BACKUP_AGE_IDENTITY=/path/to/cerynix-backup.key \
  scripts/restore.sh backups/<timestamp>

Start the matching application version; verify health, login, critical records, evidence objects, reports and integrations before reopening access.

Destructive operationrestore.sh overlays database content. Do not run it against an active production service. Take a safety copy first and keep the incident rollback point.

After recovery

Record data-loss bounds, timings and validation evidence; rotate exposed credentials; resume off-box backup replication; and update the runbook from the incident review.

← Support Safe upgrades →