Prepares for the Node 22 move by decoupling it from the database driver, so the
two upgrades land as independently reversible steps rather than one flag day.
9.6.0 cannot work on Node 22. It uses the raw V8 API (220 v8:: references, zero
napi_), so it is ABI-locked per Node major, and its GitHub release assets carry
prebuilds for ABI 108/115/120 only — nothing for Node 22's 127. Its install script
is `prebuild-install || node-gyp rebuild --release`, so on Node 22 it silently
falls through to compiling raw-V8 code against Node 22 headers. That is not just
slow: lib/preflight-deps.js rebuilds synchronously before the server listens, and
prod's systemd unit is TimeoutStartSec=90 with Restart=always, so a slow or failing
compile is an unbootable loop rather than the intended self-heal.
12.9.0 ships prebuilds for BOTH Node 20 (ABI 115) and Node 22 (127), so neither the
current runtime nor the target has to compile anything.
THE PIN IS EXACT ON PURPOSE — ^12.9.0 would defeat it. 12.10.0 dropped the Node 20
prebuild while still advertising "20.x" in engines, so a caret resolves to 12.11.x
and reintroduces the from-source compile on today's runtime. Verified against the
release assets per version:
12.0.0 / 12.2.0 / 12.4.5 / 12.6.2 / 12.9.0 ABIs 115,127,...
12.10.0 / 12.10.1 / 12.11.1 ABIs 127,137,141,147 — no 115
The reasoning is recorded in preflight-deps.js, which is where anyone hitting the
matching failure will already be reading.
13.x was considered and rejected FOR NOW: it is the first N-API release, which ends
the per-major ABI problem for good (8 prebuilds keyed by platform, not ABI) and is
where we should eventually land — but engines is ">=22", so it cannot be adopted
while prod, alpha and CI all run Node 20. It is also three weeks old with three
patch releases, which is young for the one component that owns all the data.
No API changes to absorb: every major from 10 to 13 bumped only for dropping EOL
Node/Electron versions, so the ~1486 .prepare(), 46 .transaction() and 59 .pragma()
call sites are untouched.
Verified on Node 20: installed from a PREBUILT binary (no obj.target, so no
compilation), opens a real database, and the WAL path the #149 checkpointer depends
on still works — journal_mode=WAL engages on a file DB, pragma(...,{simple:false})
returns the expected shape, and a second connection from another handle reads and
runs wal_checkpoint(TRUNCATE). 1649/1649 tests pass.
* spike: replace sharp with pure-JS image ops (jimp + jsquash WASM)
Removes the last native dependency from the ingest path, so the server no longer
needs a per-platform/per-ABI prebuilt to thumbnail an image. Motivated by getting
the server onto hardware with no toolchain, but the ABI tax is paid on every
install — it is the same failure class lib/preflight-deps.js exists to explain.
lib/image-ops.js is the whole surface: metadata() and writeThumbnail(), which are
the only two things ingest ever asked sharp for.
Format parity holds. jpeg/png/gif/tiff/bmp are native to Jimp; webp and avif go
through @jsquash WASM, whose bundled .wasm must be compiled by hand because the
packages locate it with fetch(file://) and Node has no file:// fetch — the only
symptom otherwise is a bare "fetch failed". heic is unsupported, as it already
was: sharp advertises heif but its prebuilt libvips refuses HEVC.
#170 is preserved by a different mechanism. Jimp applies EXIF orientation at
decode and rewrites the tag to 1, so metadata() reports display dimensions and
imageDisplayDims() runs as a no-op instead of swapping W/H a second time. The
helper stays in the path so the rule keeps living in one place.
Verified: 1643/1643 tests pass, and ingest was exercised in a child process with
node_modules/sharp moved aside — jpeg, EXIF-rotated jpeg, png, webp, avif, gif
all measured and thumbnailed correctly, corrupt input still yields nulls with no
phantom thumbnail_path.
KNOWN BLOCKER, do not ship as-is: Jimp is pure JS on the main thread, where sharp
handed work to a libvips threadpool. A 12MP photo goes 65ms -> 1079ms, and the
event loop stalls for 1003ms of it (sharp: zero stalls). thumbnail-backfill.js
walks a whole library at boot, so this reproduces #240 exactly — blocked loop,
missed heartbeats, panels marked offline, reconnect churn. Needs a worker_thread
offload before this is viable; image-ops.js is the seam for it.
* Run image decoding on a worker thread
Fixes the blocker the previous commit shipped with. Pure-JS decoding costs ~1s of
solid CPU for a 12MP photo, and in-process that is not a slow upload but a stalled
event loop — no heartbeats, no socket traffic. thumbnail-backfill.js walks a whole
library at boot, so it reproduced #240 (blocked loop -> missed heartbeats -> panels
offline -> reconnect churn) from our own maintenance. sharp never did this because
libvips works on a threadpool.
image-ops.js is now a dispatcher over image-ops-worker.js; the work moved unchanged
to image-ops-core.js, so callers and their failure contract are untouched.
Measured on a 12MP photo: 1079ms wall with the loop stalled 1003ms, to 1881ms wall
for two ops with ZERO stalls and 185 timer ticks serviced. Wall time is worse and
that is fine — it is off the main thread now.
Design notes, all load-bearing:
- ONE JOB AT A TIME. A decoded 12MP bitmap is ~48MB of RGBA; overlapping jobs
multiply peak memory by queue depth, which is the wrong failure on the small
targets this change exists to reach. Costs no throughput — the work is CPU-bound
and one busy worker already saturates its core.
- unref'd while idle, ref'd only in flight. Otherwise scripts/backfill-rotation-
dims.js never exits and `node --test` hangs forever. Verified: a CLI-style run
exits in 104ms, code 0.
- decode failures reply as messages, so one bad upload cannot tear down the worker
and take unrelated queued jobs with it.
- in-process fallback if a thread cannot be had, warned rather than silent.
test/image-ops.test.js pins the loop-liveness property, which no functional test
would catch. Its thresholds were mutation-tested against the inline path: the first
version passed there too (4MP stalls only ~355ms, under a non-flaky threshold), so
the fixture is 12MP and the thresholds sit in the gap between the two behaviours —
worker ~90 ticks/~0ms, inline ~3 ticks/~897ms. It now fails inline, as a guard must.
1647/1647 pass. Ingest re-verified with node_modules/sharp moved aside.
* Measure and thumbnail an image from a single decode
Ingest asked for metadata() then writeThumbnail(), which decoded the file twice.
That pairing was free under sharp, whose .metadata() only parses the header, but
every decode here is a full one — ~1s for a 12MP photo — so the naive translation
doubled the most expensive thing on the ingest path.
image-ops.measureAndThumbnail() returns both from one decode. Full ingest of a
12MP photo: 2 decodes/~1.9s -> 1150ms, still with zero event-loop stalls.
The subtlety is the failure contract. In the two-call version width and height
were assigned BEFORE the thumbnail was attempted, so a failed thumbnail still left
usable dimensions on the row — the player needs them to letterbox. Merging naively
would have turned any thumbnail failure into total metadata loss. So a WRITE
failure is reported ({thumbnailWritten:false, thumbnailError}) with the dimensions
intact, and the caller sets thumbnail_path only when the write succeeded, keeping
the phantom-path discipline. A DECODE failure still throws — there is nothing to
report about an unreadable image.
backfill-rotation-dims.js deliberately keeps the separate calls: it probes every
image row but regenerates a thumbnail only for the few whose dimensions changed,
so pairing them there would decode files it has no reason to thumbnail.
Tests count decodes rather than timing them — an exact property, and a wall-clock
comparison would be flaky under load. The count filters for reads of the file under
test: Node's ESM loader also goes through fs.promises.readFile, so a raw call count
picks up jimp's and the WASM codecs' lazy loading and reads 30 instead of 1.
1649/1649 pass. Ingest re-verified across all 7 formats with sharp moved aside.
* Dockerfile: sharp is no longer a production dependency
--omit=dev now leaves it out entirely; better-sqlite3 is the only native module
the builder stage still needs a toolchain for.
A second review pass, run against the previous commit, found four blockers — two of
them introduced by the fixes in that commit. It also confirmed the original account
takeover is closed: a hostile IdP with real TLS, discovery, JWKS and RS256 driving the
real routers now stops at domain_not_allowed, and all 16 bypass variants are refused.
DOMAIN OWNERSHIP (the root cause, not the symptom)
A claimed domain used to mean "nobody else claimed it". It now means the organization
published a record in that domain's own DNS — TXT or CNAME, at a dedicated
_screentinker-verify name rather than the apex, where an edit would sit beside SPF.
- an unverified domain routes NOBODY and cannot be asserted; it reserves the name
- an unverified claim LAPSES after 8 hours, so a domain cannot be held against its
real owner, and lapsing rotates the token so a record left over from an abandoned
attempt cannot satisfy a later claim
- a verified domain never expires — re-proving on a timer would log a customer out
over a DNS edit made months later
- routing and confinement read the VERIFIED set only, never the typed column
- configuring SSO now requires a verified email address
- platform admins are emailed when a domain is claimed; nothing is ever sent to the
claimed domain, which would let any tenant make this product email third parties
Instance-wide providers are exempt from all of it: they are the operator's own
configuration and keep the trust they have always had.
BLOCKERS FROM THE REVIEW
- two unauthenticated remote crashes, both one request, both "async handler throws
before its try": `Cookie: st_oidc_tx=%` (unguarded decodeURIComponent) and the
fail-closed secret added last commit, which turned a JWT_SECRET rotation into a
permanent crash loop. Fixed the CLASS with asyncRoute() rather than the instances.
- the SSRF guard was bypassable via IPv4-mapped IPv6 ([::ffff:127.0.0.1]) and also
refused every host beginning "fc"/"fd" (fcm.googleapis.com). Addresses are now
parsed and compared by RANGE. 42 cases verified.
- the takeover fix had NO test — the test named after it asserted two struct fields
and passed with the guard deleted. The decision is now a pure function and four
mutations were confirmed to turn the suite red.
- the PUT path never received the TOCTOU fix, so two orgs could end up holding one
domain and forEmail handed routing to the attacker's older row.
ALSO
- linking compared slugs, so an org could never rotate its own IdP, and fell open on
an empty auth_provider. It now asks which ORGANIZATION owns the slug.
- an account stranded by a deleted provider can be reclaimed by password reset —
proof of the mailbox, which is stronger than the IdP assertion that created it.
- /sso/claim accepted a pre-TOTP mfa_pending token and returned the full user row;
it now takes a purpose-built 120s claim token with a pinned algorithm and typ.
- the rate limiter keyed on a caller-controlled path, so a trailing slash bought a
fresh bucket — a real login brute-force bypass.
- domain_not_allowed and account_exists_other_provider rendered as "please try
again", advice that can never work.
- malformed asserted addresses are refused rather than trimmed into shape.
- dead config (microsoftTenantId defaulted to 'common', which the provider code now
refuses) and the orphaned google-auth-library dependency removed.
1591 tests pass. Domain lifecycle verified end to end against a running server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bvjey4FNam49MN7ybjcq6A
Two mistakes in the previous commit, both of which broke CI.
The lockfile was not regenerated after adding puppeteer-core to
devDependencies, and `npm ci` requires the two to agree — so every job that
installs dependencies failed before running anything.
The smoke test was also placed in test/, which I described as keeping it out of
`npm test`. It does not: `node --test` globs that directory, so the runner
picked it up regardless of intent, tried to drive a browser as a unit test, and
failed. It now lives beside the server as smoke-ui.js, with a note saying why,
so the next person does not put it back.
Verified the way it should have been the first time: npm ci succeeds, native
modules still load, npm test is 807/807 with no browser involved, and
`npm run smoke` is 32/32 on its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uaeo9MvzKoyXuN6ZsbhtkL
A whole class of defect found today was invisible to the unit suite, to a
syntax check and to review, and appeared only in front of a browser: a context
menu whose only item read "schedule.ctx_new", pointer handlers stacking on every
calendar render so one drop fired five PUTs, and a week grid that scrolled
sideways on a phone. Nothing in the repo could have caught any of them.
This keeps the checks that earned their place and throws away the scratch
scripts around them. It boots a server, drives every view, and asserts each view
renders, none raises an uncaught error, no untranslated key reaches the screen,
the calendar binds its handlers once however many times it re-renders, and
nothing overflows horizontally at phone width.
Deliberately NOT part of `npm test`. It needs a real browser, which CI does not
have, so it is `npm run smoke` and exits 0 with an explanation when puppeteer or
Chrome is missing — a test that fails for want of tooling teaches people to
ignore failures. puppeteer-core rather than puppeteer, so installing it does not
pull down a private copy of Chrome; it drives whichever one is already there.
Verified both ways: 32/32 against current main, and it fails on the listener
stacking when that fix is reverted. The missing-key case is covered by the unit
guard instead, since a context menu only exists once it has been opened.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uaeo9MvzKoyXuN6ZsbhtkL
sharp decodes uploaded files directly (lib/content-ingest.js, routes/content.js
both call sharp(file.path) on whatever a user uploaded), so its bundled libvips is
part of the request path rather than a build-time detail. Moves 0.33.5 -> 0.35.3,
libvips 8.15 -> 8.18.
Validated against the calls this codebase actually makes, because it is a major
bump: metadata() still reports EXIF orientation (1/3/6/8 all round-trip, which is
what lib/media-orientation.js exifSwapsWH and the rotation-aware dimensions depend
on), a bare .rotate() still auto-orients, and resize().jpeg().toFile() is unchanged.
png/webp/jpeg/gif/avif all still encode and decode, and malformed input still throws
rather than crashing.
The new libpng is stricter, which surfaced a latent problem in the AUTH-01 test: its
1x1 PNG literal had a corrupt IDAT chunk whose stored CRC did not match its data. The
old decoder accepted it; the new one refuses with "vipspng: libpng read error", so no
thumbnail was written and the content-gate assertions failed with a 404 that reads
like an auth regression. Replaced with a PNG whose every chunk CRC verifies. The
stricter decode is the correct behaviour and is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
