screentinker/server/test
screentinker 9c70fcc790
feat(diagnostics): device incident log — offline cause, network-vs-reboot, display-sleep (#175)
* feat(diagnostics): device incident log — why a screen went offline/black, with device-attested cause

Field request (Bold/s_t_r_o_b_e): "screens go offline randomly — let us see the cause." Answers it
across the fleet with a unified incident log, and — the key insight — lets the DEVICE disambiguate
the cause the server can't: if the app process survived the gap it was a NETWORK problem (not a
reboot), and it can even tell a dropped Wi-Fi/Ethernet link from a link-up-but-server-unreachable
(router/upstream) failure.

Schema:
- device_status_log gains reason + detail (WHY each offline transition happened).
- NEW device_events table (unified incident feed): type (offline/online/display_off/display_on/
  crash/reboot/network/app_error) + reason + detail, indexed, age-pruned + per-device capped.

Server:
- Capture the socket.io disconnect REASON (transport_close/ping_timeout/transport_error) instead of
  discarding it — recorded in the offline-cause log. devices.offline_reason stays on the EXIT-SIGNAL
  contract (crashed/clean_exit/silent) — a separate axis, preserved (violent death = 'silent').
- device:event handler (typed incidents) + device:connectivity-report handler (device-attested).
  lib/incident-classify.js (pure, unit-tested) composes reason+detail: cold_start->reboot;
  link_lost->network "Wi-Fi/Ethernet link lost"; else network "LAN up, server unreachable
  (router/upstream)"; appends SSID / weak-signal (rssi<-75) / IP-changed. On a report it upgrades the
  most-recent offline row from the server's guess to the device's ground truth.
- heartbeat timeout -> 'heartbeat_timeout'; retention/cap for device_events.
- Device-detail API returns statusLog.reason/detail + the last 50 device_events.

Device (Android WebSocketService): ConnectivityManager default-network callback (link-lost during a
gap) + Wi-Fi SSID/RSSI + IP snapshot -> device:connectivity-report on reconnect (app survived =>
network); ACTION_SCREEN_ON/OFF receiver -> device:event display_on/off ("screen went black"). All
guarded/feature-detected; no manifest change; compiles clean.

Web + Tizen players: reconnect connectivity-report (link_lost from navigator.onLine during the gap)
+ visibilitychange -> display_off/on. Best-effort (no wifi detail in a browser). Tizen exit-signal
marker slice untouched.

CMS (device-detail): the offline cause on the uptime-timeline hover + a new "Recent incidents" panel
(merged offline periods + typed events, friendly labels, detail, relative time + down-duration).

Built as a 4-way parallel agent fan-out over disjoint domains against a locked contract, then
integrated. Verified: full server suite 443/443 (incl. the seam fix keeping the exit-signal contract
intact), Android compileDebugKotlin clean, all players + CMS node -c clean.

Refs #170.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(diagnostics): internet-reachability probe — split "our server down" from "no internet" (#170)

Follow-up to the incident log: when a device's link is UP but it's offline, "router/upstream" was a
catch-all. The device now probes a public host (1.1.1.1 / 8.8.8.8 :443) DURING the gap, so the cause
pinpoints blame:
  - link_lost=true                     -> Wi‑Fi/Ethernet link lost (device's own link)
  - link up, internet_ok=true          -> server_down: internet reachable, OUR server was unreachable
  - link up, internet_ok=false         -> no_internet: router/ISP down
  - link up, no probe result           -> generic router/upstream (unchanged fallback)

- Android WebSocketService: fire a short daemon-thread TCP probe (443, either host) at disconnect;
  the result rides the connectivity-report as internet_ok (omitted if the gap ends before it finishes).
- lib/incident-classify.js: 3-way split on internet_ok; new reasons server_down / no_internet.
- Frontend i18n: device.event.server_down / .no_internet labels.
- Tests: +3 classify cases (server_down, no_internet, link_lost wins over internet_ok). 12/12.

Verified: classify 12/12, Android compileDebugKotlin clean, node -c clean. Refs #170.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(diagnostics): log an 'upgrade' incident (old → new app_version) — server-side (#170)

When a device reports an app_version different from the stored one, applyDeviceInfo logs an
'upgrade' device_events row (detail 'old → new'). Server-side, so it covers Android/Tizen/web with
no client change; a fresh pair (no prior version) isn't counted. Adds device.event.upgrade label.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:26:04 -05:00
..
helpers test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
148-eviction-storm.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
148-half-open.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
156-schedule-read-path.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
admin-users.test.js feat(admin): Delete Organization + Workspace with cascade (#36) 2026-06-09 09:22:21 -05:00
agency-digest.test.js feat(api): batched email digest for agency uploads (#73) 2026-06-14 13:59:37 -05:00
agency-folder.test.js feat(agency): per-token upload folder — auto-created, subtree-confined (#158) (#171) 2026-07-12 21:23:25 -05:00
agency-gate.test.js feat(api): agency portal endpoints + router.param target seam (#73) 2026-06-13 22:48:42 -05:00
agency-layouts.test.js feat(api): GET /api/agency/layouts - device-free layout geometry (#73) 2026-06-14 13:53:30 -05:00
agency-list.test.js feat(api): GET /api/agency/playlists - a token's designated targets (#73) 2026-06-14 13:08:07 -05:00
agency-scope.test.js feat(api): agency-token security primitive - off-ladder scope + agencyGate (#73) 2026-06-13 21:30:38 -05:00
agency.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
ai-design.test.js fix(ai): de-overlap generated text + layer shapes behind text (#41) 2026-06-09 12:57:41 -05:00
api.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
apitoken-unit.test.js test(api): close #92 follow-up coverage gaps 2026-06-12 20:10:36 -05:00
apk-signature.test.js feat(device-owner): tier foundation + QR provisioning + content-expiry & device enhancements (#168) 2026-07-12 19:41:07 -05:00
billing-authz.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
billing-endpoint.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
billing-token-mint.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
billing-unit.test.js feat(#146): usage metering + admin-gated Billable Screens report (contract system-of-record) 2026-07-01 20:45:27 -05:00
block-authz.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
boot-health.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
branding.test.js feat(branding): instance-level default white-label branding (#15) 2026-06-08 16:55:22 -05:00
config-paths.test.js chore(version): single-source VERSION, env-configurable data paths, bump tooling 2026-06-10 12:56:03 -05:00
content-ack-dedup.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
content-ack-flood.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
content-ack-limiter.test.js fix(#143): content-ack flood control — per-device rate budget + loop-lag valve 2026-06-27 22:21:57 -05:00
content-ack-valve.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
content-expiry.test.js feat(device-owner): tier foundation + QR provisioning + content-expiry & device enhancements (#168) 2026-07-12 19:41:07 -05:00
debug-throughput.test.js feat(#146): /api/status.debug throughput counters (gauges -> gauges + work done) 2026-06-30 23:24:32 -05:00
device-block-and-auth.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
device-events.test.js feat(diagnostics): device incident log — offline cause, network-vs-reboot, display-sleep (#175) 2026-07-13 11:26:04 -05:00
device-owner-qr.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
device-pairing-notify.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
device-zone-contract.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
evicted-socket-rearm.test.js fix(#148) patch2: per-device session-settle debounce — absorb duplicate-socket storms 2026-07-02 19:12:46 -05:00
fingerprint-reclaim.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
flap-limiter.test.js fix(#146) B: sustained flap-rate limiter (the trigger fix), SNAT-safe identity chain 2026-06-30 20:59:53 -05:00
flap-paired-exemption.test.js fix(#148) Item 1: exempt paired+authenticated devices from the flap-limiter quarantine 2026-07-02 14:59:25 -05:00
flap-quarantine.test.js fix(#146) P0: auto-quarantine is in-memory + time-limited, never a DB block 2026-06-30 21:56:20 -05:00
ghcr-check.test.js feat(dashboard): version indicator + GHCR update check (#165) 2026-07-10 22:40:06 -05:00
group-sync-clock.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
i18n-tokens.test.js test(api): close #92 follow-up coverage gaps 2026-06-12 20:10:36 -05:00
identity-resolver.test.js fix(#146) P1.1: resolveIdentity short-circuits on device_id (zero-lookup hot path) 2026-06-30 21:57:55 -05:00
kill-switches.test.js fix(#146) P1.3: per-feature env kill switches + fallout doc section 2026-06-30 22:03:00 -05:00
log-selfprotect.test.js fix(#146) P3.7: coalescer carries the PEAK numeric over the window 2026-06-30 22:10:45 -05:00
loop-lag-integration.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
loop-lag.test.js feat(#142): event-loop lag telemetry (perf_hooks) + bounded storage 2026-06-27 19:01:08 -05:00
media-orientation.test.js fix(content+android): rotation-aware media — portrait upright on dashboard AND player (#170) (#172) 2026-07-12 22:05:11 -05:00
mute.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
observability-units.test.js feat(#146): always-on devices_connected + admin-toggleable /api/status debug block 2026-07-01 18:45:40 -05:00
openapi-contract.test.js docs(api): document /api/pip and the assignments muted field (#109/#129) 2026-06-18 17:36:12 -05:00
operator-block.test.js fix(#146) D: operator block — close the device_id-less gap + dashboard toggle 2026-06-30 21:28:04 -05:00
operator-permissions.test.js fix(roles): make platform_operator assignable + add deny/assign regression tests 2026-06-05 12:44:39 -05:00
ota-breaker-patchn.test.js fix(ota): treat legacy -patchN as a released version so the old fleet is offered updates 2026-07-08 23:21:26 -05:00
ota-breaker.test.js fix(#144): OTA update-check circuit-breaker + phantom guard + per-device keying 2026-06-28 23:36:52 -05:00
ota-check.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
ota-hardening.test.js fix(#146) P1.2: band-aware download caps — serve freely when healthy 2026-06-30 22:00:28 -05:00
pair-lockout.test.js fix(api): harden device pairing against brute-force (#87) 2026-06-12 20:16:12 -05:00
pip-overlay.test.js PiP overlay MVP: push image/web overlays to a device or group (#109) (#127) 2026-06-18 14:54:44 -05:00
player-media-health.test.js fix(#146): web player — guard PlayerMediaHealth calls by METHOD, not object (stale-module TypeError) 2026-07-01 22:19:22 -05:00
playlist-build-cost.test.js test(#146) P2.5: buildPlaylistPayload cost under mass reconnect — mitigated 2026-06-30 22:08:55 -05:00
provisioning-cleanup.test.js fix(#146) A: non-blocking maintenance — chunked+yield+band-gate all sweeps 2026-06-30 20:55:05 -05:00
provisioning.test.js fix(api): consolidate device pairing to /pair, remove vestigial bare endpoint (#90) 2026-06-12 20:13:16 -05:00
reconnect-storm-load.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
reconnect-throttle-integration.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
reconnect-throttle.test.js fix(#142): load-aware per-device reconnect throttle (the outage fix) 2026-06-27 19:18:00 -05:00
register-insert-crash.test.js fix(#146): crash-hardening — one device's handler throw can't take down the fleet 2026-06-29 23:38:11 -05:00
schedule-eval.test.js feat(scheduling): per-item schedule blocks (#74 dayparting, #75 auto-expire) 2026-06-11 15:46:41 -05:00
scheduled-reboot.test.js feat(device-owner): tier foundation + QR provisioning + content-expiry & device enhancements (#168) 2026-07-12 19:41:07 -05:00
schema-check.test.js fix(db): observable migrations + fail-fast schema verification (#37) 2026-06-09 09:31:52 -05:00
security-fixes.test.js fix(security): patch quick-win findings from the codebase review 2026-06-08 19:02:19 -05:00
session-settle.test.js fix(#148) patch2: per-device session-settle debounce — absorb duplicate-socket storms 2026-07-02 19:12:46 -05:00
socket-handler-isolation.test.js fix(#146): crash-hardening — one device's handler throw can't take down the fleet 2026-06-29 23:38:11 -05:00
status-debug-toggle.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
status-log-prune-hardening.test.js test: widen CI-fragile event-loop-gap timing bars (flaky prune/storm asserts) 2026-07-08 17:31:27 -05:00
status-log-prune.test.js fix(#146) A: non-blocking maintenance — chunked+yield+band-gate all sweeps 2026-06-30 20:55:05 -05:00
storm-harness.test.js test: widen CI-fragile event-loop-gap timing bars (flaky prune/storm asserts) 2026-07-08 17:31:27 -05:00
tcp-keepalive.test.js fix(#148) Items 2-4: mark-offline closes the socket + tighten ping + TCP keepalive 2026-07-02 14:59:25 -05:00
tenant-cascade-migration.test.js fix(db): cascade tenant resources on workspace/org delete (#18 follow-up) 2026-06-08 16:01:52 -05:00
thumbnail-proxy.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
tizen-eval-drift.test.js feat(scheduling): per-item schedule blocks (#74 dayparting, #75 auto-expire) 2026-06-11 15:46:41 -05:00
totp-keyrotation.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
totp-unit.test.js test(server): TOTP - bite, lockout, replay, recovery, st_ bypass, key-rotation (#100) 2026-06-13 20:48:55 -05:00
totp.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
user-deletion.test.js feat(admin): Delete Organization + Workspace with cascade (#36) 2026-06-09 09:22:21 -05:00
v4-core-liveness.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
v4-core-refresh-gate.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
v4-exit-signal-phase3.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
v4-exit-signal.test.js test(ci): OS-assigned ephemeral ports for subprocess suites — kill the port-collision flake (#176) 2026-07-13 09:51:40 -05:00
widget-render-xss.test.js fix(security): sanitize public widget render to close stored XSS 2026-06-08 19:11:14 -05:00