Skip to main content

Backups architecture

Web services​

Currently, the backups for the ISC Learn Moodle platform (hannibal server) and the other webservices (Marcellus server) are done on a NAS hosted in Energypolis, 23N312 (Synology DS923, on the school intranet — not reachable from the rack or the VPN, verified 2026-08-01).

For Moodle, data retention is 3 years, with 12 monthly versions, for the current month 4 weekly versions + a daily backup for he last 7 days.

Single copy, and an empty stub (August 2026)

Two findings from the ISC Learn restore exercise:

  • The DS923 was the only backup location for ISC Learn. Since 2026-08-01 a second, on-site copy exists: the full pull used for the DR mirror stays at rumba:/hdd/hannibal-mirror/ (~320 GB on the RAIDZ2 pool) and is refreshed by a delta rsync, by hand (procedure; last run 2026-09-05). Since 2026-09-05 each pull ends with a ZFS snapshot of hdd/hannibal-mirror, so the copy carries versions (8 weeks, then monthly for a year) — the retention the DS923's --delete-before rsync cannot prove. The first one, hannibal-2026-09-05, holds the pre-purge state of the Moodle course backups. The copy covers the whole host, not only Moodle (Shibboleth SP key, TLS, fail2ban, crontabs, other vhosts, docker volumes — added 2026-09-05); its age is that of the last pull, which is still started by hand (todo).
  • The rumba_backup share on the rack FS2500 was an empty stub: Scripts/backup_hannibal.sh contained only an echo, and Backups/ held two empty test folders from September 2025. It was deleted on 2026-08-02 with the Volume 2 rebuild; the second reachable copy it was supposed to provide still has to be built.

The restore path itself is now tested (2026-08-01, full working mirror on rumba) — see the runbook.

The wiki archive​

The DokuWiki behind wiki.isc-vs.ch was switched off in August 2026, its content having become this documentation. wiki-archive-2026-08-06.tar.gz (61 MB, sha256 7f5da349…e20ef8) holds the pages, the full attic/ edit history, the media and the DokuWiki configuration; it lives at rumba:/hdd/backup/wiki-archive/ plus a copy off-site with pmudry. It contains the wiki's secretzone namespace, so it stays off any published location.

Student homes​

Not really a backup system, but the students' persistent data live on the NFS share of the Synology NAS FS2500, mounted in every carnaval lab VM with a ~/nas_home symlink to it.

Rumba​

Two nightly vzdump jobs cover all guests (details in rumba § storage layout):

  • 02:00 → hdd-backup (local ZFS RAIDZ2), snapshot mode, zstd, keep-daily=3 — the fast on-host copy for everyday restores.
  • 03:00 → PBS on the FS2500 (since 2026-08-03), where the full 14 daily / 8 weekly / 6 monthly retention lives (widened 2026-08-04) — dedup makes that schedule cost ≈ one full copy (~300 GB) instead of 28.
  • ISC Learn staging copy at /hdd/hannibal-mirror/ (see the caution above).

The PBS-on-NAS decision (2026-08-02): a backup server must sit outside the failure domain it protects, which ruled out a PBS or NFS datastore on rumba itself; the MikroTik RDS2216 (ARM64, no PBS) and Synology Active Backup (no LXC support) were also rejected. What is still not covered: off-site. The target is a second PBS pulling a sync job from this one (target architecture); until then the off-site story for guests is manual.

Disco and Chacha​

Part of CALC@HEI — tracked on the CALC@HEI todo page.