mirror of
https://github.com/screentinker/screentinker.git
synced 2026-08-13 22:03:13 -06:00
81e7d58099
Second head of the OTA-loop root cause (#144), on the connection/heartbeat layer: unbounded device-driven work with no circuit-breaker. Symptoms in Bold prod — devices shown OFFLINE in CMS while online+playing, loop-lag simmer (p99 300-1145ms), device_status_log grown to 1.1M rows. False-offline (two causes, both fixed): - evicted-socket re-arm race: evictPriorSocket runs before registerConnection, so the evicted old socket's disconnect armed a fresh offline timer for a just-reconnected device. Tag evicted socket ids and bail in the disconnect handler (ws/deviceSocket.js). - heartbeat checker false-positive: a device with a live socket in /device is UP even if its in-memory lastHeartbeat is stale under lag; skip it instead of marking offline (services/heartbeat.js). Storm containment: - batched/coalescing device_status_log writer (lib/status-log-writer.js): net state per device per flush, breaking the storm->bloat->slow-write->lag loop. - newest-N-per-device row-count cap in the global sweep (db/database.js): hard bound regardless of churn; trims the existing 1.1M backlog on the first sweep. Per-device prune unified to statusLogRetentionDays (was hardcoded 7d). - reconnect-throttle idle-bucket sweep (lib/reconnect-throttle.js): the #142 throttle already existed; added the memory-bound sweep it lacked (wired in server.js). No second breaker. - cosmetic: cap the OTA breaker level counter (lib/ota-breaker.js). - best-effort status-log flush on the crash path (server.js). Tests: load harness (test/reconnect-storm-load.test.js) proves breaker engage, clean offline-clear, no-throttle-on-normal-reconnect, batched writes, bounded loop-lag; cause-1 re-arm race proven with teeth (test/evicted-socket-rearm.test.js). Both mutation-checked (fail without their fix). Full suite 240/240. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| activationNudge.js | ||
| activity.js | ||
| agency-digest.js | ||
| alerts.js | ||
| email.js | ||
| heartbeat.js | ||
| loop-lag.js | ||
| scheduler.js | ||
| signupEmails.js | ||