Skip to main content

External VPS

Two Virtual Private Servers hosted at Infomaniak predate the rack:

  1. Hannibal hosts the ISC Learn Moodle platform and a few co-tenant sites. It has its own section: ISC Learn, the server, history & operations.
  2. Marcellus is the legacy VPS, compromised in September 2026 and to be discarded — this page.

Dated operations on marcellus: history. Open items: ops todo → hosting.

Marcellus​

Ubuntu 22.04.5, 2 vCPU / 5.8 GB RAM / no swap, public 195.15.229.64 (+ IPv6). Apache 2.4 fronts everything on 443; Docker, ChirpStack and the databases sit behind it. Inventoried read-only August 2026 — the numbers below are from that survey.

Compromised host — network cut by Infomaniak on 2026-09-07

The ikarus WordPress was taken over through the Duplicator installer left in its web root, and www-data ran a PyInstaller payload from /tmp on 2026-09-05/06 that reached a malware sinkhole (incident). The ikarus vhost is disabled and the payload and installer are in /root/quarantine-2026-09-07/. Every PHP site on this host shares www-data, so none of them is trusted until rebuilt; the host itself is to be discarded, not cleaned (ops todo). Until Infomaniak lifts the block the only access is the manager's noVNC console.

  • SSH: ssh ubuntu@marcellus.begincoding.net (port 22), key-based with passwordless sudo. PasswordAuthentication no, so the shared password in secretzone is a console/sudo password, not a remote login vector.
  • No host firewall: ufw inactive, INPUT policy ACCEPT, no fail2ban — every listening port is exposed to the Internet.
  • Apache forward proxying: ProxyRequests is Off in every vhost since 2026-09-07 (apt, lora and the two tisc vhosts had on; mod_proxy_connect is not loaded, so CONNECT never worked, and the default 443 vhost is advpro, which never had it).
  • Disk: / 82 % full (20 GB) after the tMatch deletion (2026-08-10); /srv 9 % (8.3 GB of 98 GB) since the frozen Moodle was removed on 2026-09-05.

Vhosts and the certificate​

Read from /etc/apache2/sites-{available,enabled}/ on 2026-08-18: 11 enabled vhost blocks serving 13 names, plus a catch-all HTTP→HTTPS redirect. The 19 counted in the August inventory included the four tMatch vhosts, deleted 2026-08-10, and the frozen isc.hevs.ch. Count the .conf files, not the directory entries: a stray fullchain.pem also sits in sites-enabled/, unread because the include glob is *.conf.

All names live on the single lora.begincoding.net certbot lineage, so one failed authorization fails all of them — the same trap as hannibal's certificate. Both bit us in August 2026 and both are now fixed.

The lineage carries 15 names, valid to 2026-11-17 (verified on the certificate, 2026-08-19). apt.chezmoicamarche.ch is deliberately not among them: the name is NXDOMAIN, so it could never answer a challenge and would fail the renewal for everything else. Its vhost still exists and still serves, but only by IP. The re-issue that got there is in the history.

This lineage uses authenticator = apache and installer = apache, so an ordinary certbot renew reloads Apache itself and needs no deploy hook.

caution
certonly does not reload Apache, whatever the installer line says

certonly obtains without installing, so after a re-issue mod_ssl keeps serving the old certificate — and mod_ssl never re-reads its files on its own. Verified 2026-08-19: right after a successful certonly --apache, a new SAN still failed openssl x509 -checkhost, and systemctl reload apache2 fixed it. Always re-issue with a --dry-run first, then reload, then check a name with openssl s_client -servername. The installer line does survive certonly (checked). Renewal is triggered twice over — certbot.timer and a root cron at 03:15 — which is redundant but harmless.

VhostServesState
lora.begincoding.netChirpStack LoRaWAN server (:8180)live
ikarus-sozialearbeit-vs.ch (+ www., ikarus.snowmon.ch)WordPress, 1.3 GBvhost disabled 2026-09-07 — compromised through its Duplicator installer (incident); the snowmon alias 301s to the canonical name
inf1.begincoding.netWordPress, 2.2 GBlive
advpro.begincoding.netWordPresslive
scala.begincoding.net, sin.begincoding.netstatic course siteslive
apt.chezmoicamarche.chapt repositorylive, but the name does not resolve
marcellus.begincoding.netlanding page + monit at /monit/live
grafana.marcellus.begincoding.netGrafana (:3000)live
tisc.isc-vs.ch, tisc.isc-vs.devTISC editor (Docker, :8081/:8082)DNS removed 2026-09-18 — migrated to tisc.isc-vs.ch on its own guest (CT 123 on rumba); the containers on marcellus itself are untouched, still counted in the decommission list
gptisc.begincoding.net→ :8888vhost disabled 2026-08-18 (backend was gone)
influxdb2.marcellus.begincoding.net→ :8086vhost disabled 2026-08-18; the InfluxDB service and its data are untouched
portainer.marcellus.begincoding.net→ :9000vhost disabled 2026-08-18 (backend was gone)
isc.hevs.ch—vhost file and content removed 2026-09-05; production is on hannibal

Per-vhost traffic cannot be measured: every site except gptisc logs to the one access.log with the combined format, which carries no vhost field. Deciding what is still used needs owners, not logs.

Containers​

Three, all up: tisc-app-prod + tisc-db (postgres 15) and http_tisc_ch (nginx). Until 2026-08-10 there were six: tmatch_app + tmatch_db and freeipa-server went with tMatch.

tMatch deleted (2026-08-10)​

tMatch, its four vhosts, its compose project (including the freeipa-server that came with it), its volumes, images and data directories were removed on request, deliberately without a backup, and the six DNS records deleted at Infomaniak. The source survives on GitHub; the database does not. Full account: history.

Verifying a deletion needs a second resolver

Right after the change, Cloudflare's resolver still answered for two of the four names, and even ns41.infomaniak.com returned an authoritative-looking A with a ~20 min TTL. Google's resolver said NXDOMAIN for all four — the residue was cache and replica lag. Cross-check a deletion against an independent resolver before concluding it did not work, and note that nsany1/2.infomaniak.com answer without the aa flag for isc-vs.ch, so they cannot settle the question either.

Native services and databases​

ChirpStack (application/network/gateway-bridge/packet-multiplexer, UDP 1700–1701), Mosquitto, Redis, monit, telegraf, Grafana, shibd, InfluxDB (disabled), and PHP-FPM 7.3, 7.4, 8.1 and 8.2 all running — the first three are EOL.

  • MySQL (localhost only): wordpress_inf1 24 MB, wordpress_ikarus, wp_advpro, grafana, phpmyadmin, plus five orphans whose files went with the frozen Moodle (September 2026): moodle_isc 159 MB, moodle_test 23 MB and three wordpress_isc_hevs_ch* (42 MB), each with its own MySQL user — kept on purpose, droppable in one statement (todo).
  • PostgreSQL 12 (:5432): chirpstack_as, chirpstack_ns, wp_advpro. PostgreSQL 14 (:5433): empty — nothing but the templates.

Odds and ends​

  • Eight shell accounts (local, tonio, remi, pim, apt, tisc_ch, tisc_dev, ubuntu); last non-ubuntu login was May 2026. tmatch_ch and tmatch_dev were deleted with tMatch on 2026-08-11.
  • Membership of the docker group is equivalent to root on this host, and it currently holds ubuntu, pim, tisc_dev and tisc_ch — so an unclaimed account still has full control. Who the accounts and their SSH keys belong to is recorded in secretzone/marcellus.md; settle it with the vhost ownership question (ops todo).
  • The ubuntu crontab fetches https://online.oouu.ch?uid=f6313da3-… every 5 minutes. Purpose unrecorded — nobody has claimed it.
  • 4.6 GB of Docker images and 2.5 GB of build cache are reclaimable.

Credentials: secretzone/marcellus.md. Migration and decommissioning are tracked in the ops todo.