Upgrading the runtime does not go through scripts/upgrade.sh, so nothing
reinstalls dependencies — which is precisely when the one remaining native
module goes stale. The runbook now covers the version floor imposed by
--env-file-if-exists, why the better-sqlite3 pin is exact, and why a version
without a matching prebuild can turn Restart=always into a boot loop.
Also records that this build changes dependencies in both directions: rolling
back past it needs the reinstall too, because earlier builds import sharp at
runtime and this one drops it from production dependencies.
Kept deployment-neutral — no hostnames, addresses, or environment specifics.
The README covers installing, upgrading, backing up and admin recovery -- the
happy paths. It says nothing about the parts that actually go wrong: which
deployment shape you are on and why the other shape's commands silently do
nothing, how to tell a deploy really took when a version string cannot prove it,
and the handful of traps that have each cost real time at least once.
docs/operations.md is that runbook. The load-bearing entries:
- the served APK is a bind-mounted FILE, so it must be replaced in place. mv
or cp gives the host a new inode while the container keeps serving the old
bytes, with nothing in any log to say so;
- the advertised apk_size must equal the served bytes or displays download,
reject and retry forever -- and the OTA query parameter is `version`, where
the wrong name produces a result that looks identical to a broken OTA;
- a version string does not prove new code is running, and neither does the
build hash: it covers the frontend, so a server-only change deploys with an
unchanged hash and looks exactly like a stale image;
- ownership before checkout, because a partial checkout leaves VERSION updated
while the code is the old release and no migrations ran;
- a service user with no home directory makes npm install nothing while
appearing to succeed;
- a prerelease sorts below its own release, and the Android update check
offers one to any older client on the stable channel;
- native modules are built for one Node ABI, and the mismatch presents as
hundreds of unrelated test failures rather than one clear error.
Deliberately generic: no addresses, hostnames, customer names or credentials, so
it is useful to anyone self-hosting rather than a description of one estate.
Every claim was checked against the code or the workflows rather than recalled.
