UPS monitoring (NUT)

The rack's BlueWalker VFI 2000 RMG PF1 has no network interface, so since 2026-08-03 it is read over USB from rumba using NUT (Network UPS Tools). Deployed by provisioning/ups/deploy-ups.sh. Since the same day the UPS also powers rumba (both PSUs) and nothing else, which is what makes the shutdown policy a live question.

At a glance

Host	rumba — the only machine cabled to the UPS
Version	NUT 2.8.1-5 (Debian 13 main), driver nutdrv_qx / subdriver=cypress
UPS name	rackups — upsc rackups on rumba
upsd	listening on 127.0.0.1:3493 and 192.168.88.51:3493
Consumers	upsmon on rumba (log/notify only) and the rack status page, polling every 30 s from CT 108
Shutdown	none — see Shutdown policy

What it reports

Read on 2026-08-03, after rumba's two PSUs were moved onto the UPS (upsc rackups):

Variable	Value
ups.status	OL	on line (mains), not on battery
ups.load	14 %	≈ 265 W, i.e. all of rumba and nothing else
output.current	1.0–1.1 A
ups.power.nominal	2000	confirms the 2 kVA nameplate
input.voltage / output.voltage	244 V / 229.5 V
battery.voltage	82.0 V	on 72 V nominal, i.e. float
ups.temperature	19 °C	bottom of the rack, cooler than the PDU probe higher up
input.transfer.low / .high	218 V / 242 V	outside this window it switches to battery

Nothing per-outlet is available (outlet.0.switchable: no), nor input current or real/apparent power.

danger

NUT's battery.runtime and battery.charge are junk — the real ones are elsewhere

The Q* protocol carries neither runtime nor true charge, so nutdrv_qx estimates both from battery voltage against an uncalibrated runtimecal. The charge is not merely imprecise: the driver computes (voltage − low) / (high − low), and this unit reports battery.voltage.high: 78.00 while the pack floats at 82.0 V — permanently out of range, so battery.charge is pinned at 100 %.

Worse, the UPS firmware intermittently answers the Q* status query with a zero-filled battery voltage — upsc rackups battery.voltage returned 00000 in 21 of 90 one-second samples on 2026-08-03 (bursty; ~2 % of netdata's 10 s samples over 8 h). The protocol has no checksum and nutdrv_qx applies no plausibility filter, so upsd serves the zeros and the derived charge collapses with the UPS steadily on line. (Netdata's gauges interpolate, so its charts show ramps to 35–60 V rather than the raw zeros.) Only battery.voltage is affected — input/output voltage never zeroed in 8 h — and ups.status has not flapped over 24 h, so status-keyed alerting is unaffected.

Via NUT, trust only ups.status, ups.load and output.current; treat any single voltage sample as suspect. The status page and the alerting deliberately do not publish charge or runtime at all, so no threshold can be keyed on them by accident. Netdata's stock UPS alarms needed the same treatment. The genuine values are on the HID interface.

The HID Power Device interface — where the real numbers are

The UPS exposes two USB interfaces. nutdrv_qx uses interface 0, which is just the Cypress serial tunnel. Interface 1 is a proper HID Power Device interface, owned by the kernel's hid-generic at /dev/hidraw1, and it carries genuine battery telemetry — decoded on 2026-08-03:

	Value
RunTimeToEmpty	3300 s — 55 min	a real measurement, and it tracks load: 26 460 s when the UPS was empty
RemainingCapacity	100 %	genuine, unlike NUT's saturated 100 %
RemainingCapacityLimit	10 %	the UPS's own low-battery threshold
PresentStatus	bitfield	real Overload, NeedReplacement, InternalFailure, ShutdownImminent and BelowRemainingCapacityLimit bits

Everything nutdrv_qx reports is there too, and agrees with it — which is what pinned the field mapping.

Read it with ups-read-hid on rumba (--json for machine output), installed by deploy-ups.sh from provisioning/ups/read-hid.py. Read-only, and safe to run while NUT is up since the two drivers are on different interfaces.

Three traps encoded in that script — do not "clean them up"

• The descriptor's unit exponents are unusable (exponent 6 on the voltage reports would render an 82.0 V battery as 8.2 × 10⁸). Every scaled field is really in tenths; the script hard-codes that.
• The UPS returns EPROTO on back-to-back GET_FEATURE requests. A bare loop silently yields nothing for about half the fields, which looks like missing data rather than an error. The script paces reads 50 ms apart and retries.
• usbhid-ups cannot be pointed at interface 1 in NUT 2.8.1: it hard-claims interface 0 and bails. usb_set_altinterface does not help — that sets the alternate setting of interface 0, not the interface number. Using this data from inside NUT needs a driver patch, not a config option.

On the status page

The rack status page polls upsd from CT 108 every 30 s: the UPS tile, a real dot on the UPS machine card, a load chart, load/battery/input columns in both history CSVs, and a Telegram alert keyed on ups.status (mains loss, overload, replace-battery, upsd silent). Details and the CSV column-order rule are on the status-page side.

LIST VAR needs no login — upsd authenticates writes only — so CT 108 holds no UPS credential; the read-only monitor account in upsd.users stays free for a future upsmon secondary.

Netdata alarms

Netdata on the rumba host auto-detects upsd and ships three stock alarms. Two of the three are wrong on this hardware, and both fired for days — as mail, because the node is claimed to Netdata Cloud, which notifies on every transition regardless of the to: role in the agent's health file.

deploy-ups.sh therefore installs provisioning/ups/netdata-upsd-health.conf as /etc/netdata/health.d/upsd.conf — a same-named user file fully replaces the stock one, and netdata's daily auto-update does not touch /etc/netdata/:

• upsd_ups_battery_charge (stock warn < 75 %) keys on the junk charge above and fired repeatedly on 2026-08-03 with the UPS on line. It keeps its chart but loses its thresholds.
• upsd_ups_last_collected_secs keeps its purpose — upsd gone silent — but on absolute thresholds (warn > 60 s, crit > 300 s, delay: up 1m) since 2026-08-05. Stock's 5 × $update_every resolved to 5 s, and the driver intermittently lets its data go stale (Data for UPS [rackups] is stale in upsd's log), leaving netdata with collection gaps of 6–14 s: 40 false WARNING/CLEAR pairs in two days. Anything keyed on this UPS must tolerate ten-second gaps; only a gap of minutes means upsd is really gone.
• upsd_10min_ups_load stays stock — ups.load is trustworthy.

caution

Do not add /etc/netdata/go.d/upsd.conf to slow the polling down

Netdata's upsd job is created by service discovery (net_listeners, on port 3493) and is named local, hence the upsd_local_rackups.* charts. A file-based job does not override it: it adds a second job, so upsd gets polled twice under a parallel upsd_rackups.* chart tree, with every alarm duplicated (tried and reverted 2026-08-05). Overriding the discovery rule would mean shipping a copy of the whole stock sd/net_listeners.conf, freezing every other collector's discovery — not worth it, and unnecessary: netdata's 1 s polling never touches the serial link, since upsd answers LIST VAR from the driver's cached copy.

Shutdown policy

upsmon runs with powervalue 0 and MINSUPPLIES 0: it logs and notifies, but nothing is ever shut down. That was correct while the UPS carried nothing, and it is still what is configured.

The shutdown policy is now behind the cabling

rumba no longer has a mains feed independent of this UPS, so in a real outage it runs on battery and is then hard-killed when the battery empties, taking the public sites, the ISC Learn DR mirror and the status page down uncleanly.

Autonomy is no longer a guess — the HID interface reports 55 minutes at rumba's load, which is ample for a graceful shutdown and removes any reason to calibrate runtimecal. What remains is a plumbing choice (upsmon decides on nutdrv_qx's LB flag and cannot see the HID interface): trust LB and set powervalue 1, run a small watchdog on ups-read-hid, or bridge the HID data into a dummy-ups device. The trade-offs are weighed in the todo; whether LB is firmware-reported (rather than derived from the bogus charge estimate) is only settled by a controlled discharge.

Either way, rumba shutting itself down also stops the alerting that would report it: the Telegram message marks the start of an outage, never the end.