TWO CHANGES, together because they touch the same packager hunks.
1. THE DRIVER.
The BrightSign package used to be MANUFACTURED. scripts/build-server-zip.sh dropped
better-sqlite3 from package.json and then installed db/sqlite-compat.js into
node_modules under that name, so every require resolved to the façade. It worked —
and it shipped a database layer that no test had ever executed. That is the same
shape as the TELEMETRY_COLLECTOR TDZ crash that took production down while 1676
tests and four CI jobs were green: a build-time rewrite cannot be tested by the
build that performs it.
db/sqlite-driver.js now decides at runtime: the native driver when it loads, the
node:sqlite façade otherwise. One artifact, one code path, and — the point — both
branches reachable from a test. ST_SQLITE_DRIVER=node runs the entire suite the way
a player runs it, and a new CI job does exactly that on Node 24 with --omit=optional
so the fallback is reached the same way it is on hardware, not by an env var alone.
better-sqlite3 becomes an optionalDependency, so a host with no compiler installs
cleanly and falls back rather than failing. preflight-deps stops trying to rebuild a
native module on a host that has no toolchain and a working built-in driver — on a
player that was a five-minute node-gyp failure ending in a server that never started.
Asking for the native driver BY NAME (ST_SQLITE_DRIVER=better-sqlite3) still fails
loudly, because a production box that has lost its native module is broken and should
say so rather than quietly running something else.
⚠️ NODE 24 IN PRACTICE. node:sqlite is unflagged only from 23.4; on the 22.x line it
needs --experimental-sqlite and on 20.x it does not exist. So the code probes rather
than comparing versions, the player package pins engines >=24, and the built-in cases
skip on the Node 20 CI job rather than failing there.
Verified on Node 24, both drivers, full suite:
better-sqlite3 1762 pass / 0 fail
node:sqlite 1762 pass / 0 fail
and the built payload resolves node:sqlite with no better-sqlite3 present at all.
2. THE LICENCE.
The ffprobe/ffmpeg binaries added in the previous commit are LGPL 2.1 and statically
linked, so the licence text has to travel WITH them — a link on a website is not the
copy the licence asks to accompany the work. The packager now copies
COPYING.LGPLv2.1 and a build README into bin/, and refuses to build if the licence is
missing. legal/third-party.html gains an LGPL section with the written offer required
by section 6 for static linking, and the exact configure line.
It also drops Sharp, which that page still listed although #263 removed it, and names
what actually does the image work now (jimp, @jsquash/webp, @jsquash/avif).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014kfhrUPit5MCqxeTQyqr56
Bold reported loop lag that grew with uptime and reset on restart. The
signature they saw — mean = p50 = p99 = max, identical to two decimals —
is not a fixed cost paid on every cycle. It is what an IntervalHistogram
window reports when it recorded exactly ONE delay: the mean is the raw
value, and every percentile returns the bucket ceiling above it.
Reproduced against their exact numbers (1329.07 / 1329.59). So the loop
took one long turn that swallowed the sampling second, episodically —
which is what they later confirmed independently.
The turn is ours, and it is now measured rather than theorised. Probing
the real worker against a real WAL with one reader mid-transaction: a
single main-thread write blocked for 4,936ms behind the worker's
wal_checkpoint(TRUNCATE), which then reported WAL 8.8MB -> 8.8MB. TRUNCATE
is the blocking form and its locks are held ACROSS connections, so moving
it to a worker kept the fsync off the loop but not the lock; and it does
not throw when it cannot get those locks, it returns busy=1 having sat on
SQLite's 5s busy timeout and reclaimed nothing. Five seconds of stalled
loop for zero benefit, and silent.
It was reached far too easily. The rule was "escalate if the WAL grew
across three consecutive 15s runs" — which any sustained 45-second write
burst satisfies. A customer's fleet powering on in the morning does it
daily.
Two gates, because either alone leaves the hole open. A size FLOOR, so a
WAL in the lower half of its budget can't buy a blocking checkpoint it has
nothing to reclaim from. And a COOLDOWN, because the floor alone fixes
nothing for Bold — their WAL already sits at 6.2MB against a 16MB
high-water, above any sane floor, so every burst would still escalate.
However long the pressure lasts, our own maintenance may now stall the
loop at most once per window. The high-water rule bypasses both and is
untouched: a runaway WAL is the one case worth blocking for, so the
"WAL cannot grow forever" invariant is exactly as strong as before.
A busy TRUNCATE now says so in the log instead of reading like a success.
Also softened the adjacent path: when the worker is declared
unrecoverable, engageFallback() re-arms inline autocheckpoint on the main
connection — a state that is STICKY for the life of the process, i.e.
exactly the shape of "degrades with uptime, a restart fixes it". It used
to also run an unconditional main-thread TRUNCATE on the way in; that now
happens only when the WAL is genuinely over high-water, and the fallback
state is served on /api/status rather than being inferable only from a log
line that may have rolled.
Telemetry, so the next report is self-explanatory: loop_lag carries
`samples` (~50 when healthy, 1 when a single turn swallowed the second),
`tick_gap_ms` measured on the WALL CLOCK independently of the histogram,
and `worst_tick_gap_ms`/`worst_tick_at` — monotone, so five-minute polling
can no longer miss an episode.
Band semantics are deliberately unchanged. A one-sample window during a
real stall is the correct trigger for the shed valve; suppressing it would
blind the protection at exactly the moment it is needed.
Separately, device_telemetry gets the age sweep it never had. The
per-heartbeat row cap only ever trims the device whose heartbeat is being
handled, so a device that STOPS reporting leaves its rows behind forever.
The new sweep is per-device (rides idx_telemetry_device rather than
scanning), chunked and yielding like the device_status_log one, and
defaults to 30 days to match the uptime report's own default window — so
it cannot remove rows that report would have shown.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014skWYXJUWhF73EvNPgB2AS
Close the disk-fill trap: with wal_autocheckpoint=0 a dead worker means nothing checkpoints.
Controller now respawns an unexpectedly-dead worker (bounded: RespawnMax/RespawnWindowMs +
backoff); on exhaustion it re-arms a conservative inline autocheckpoint (FallbackPages) on the
main connection + reclaims the backlog, logging loudly. Clean stopWalCheckpointer() teardown is
distinguished via a 'stopping' flag so SIGTERM never triggers respawn. Env-gated worker
fault-injection (WAL_CKPT_FAIL_START) for tests. Local only — no bump/tag.
Disable wal_autocheckpoint on the main connection; run PASSIVE checkpoints from a
worker_threads worker with its OWN better-sqlite3 handle, escalating to TRUNCATE on a
size high-water or PASSIVE-starvation. Removes the synchronous fsync-heavy checkpoint
from the event loop. Config: walCheckpointIntervalMs/HighWaterMB/StarvationRuns.
Local only — no bump/tag.
