A prompt now produces a full sign: the LLM writes the design AND image prompts,
the server generates the images and composites them with the crisp text layer.
- lib/image-gen.js: text-to-image with 3 BYO/self-hostable backends, all behind
the SSRF guard: 'sdcpp' (local stable-diffusion.cpp OpenAI-compatible server,
exact small sizes that fit VRAM), 'openai' (cloud / OpenAI-compatible, snapped
sizes), 'comfyui' (prompt/history/view API).
- ai.js: prompt asks for a background_prompt (preferred — full-bleed atmosphere)
and an optional foreground image element; after the design is normalized, the
bg + fg images are generated best-effort (a failed image never fails the sign)
and returned as data URLs. New image_* settings (provider/base_url/model),
image_provider whitelist, schema column + migration.
- designer.js: AI-images section in settings; generate applies the background
image; publish bakes the background image into the HTML so it survives.
- server.js: raise JSON body limit to 12mb for embedded image data URLs.
Verified end-to-end on local Vulkan SDXL (RTX 5090): prompt -> bg+fg images on
the canvas -> publish creates a widget with the images embedded. 63/63.
Note: prod (not self-hosted) requires a PUBLIC image endpoint (e.g. OpenAI); the
SSRF guard blocks localhost there. Follow-up: upload generated images to the
content store and reference by URL to avoid multi-MB widget configs.
Competitor pressure (Mandoe 'AI Magic Create'): prompt -> signage. We answer it
in a way that's actually BETTER for signage and costs the operator nothing.
Key idea: don't generate raw images (AI garbles text - fatal for menus/promos).
The LLM returns a STRUCTURED design spec (headline, supporting text, accent
shapes, palette) that the existing Designer renders with real fonts - crisp and
fully editable. Reuses the whole Designer.
BYOK, fully under the customer's control: each workspace configures its own
OpenAI-COMPATIBLE endpoint + key - OpenAI cloud OR self-hosted (Ollama / LM Studio
/ llama.cpp). Operator bears zero AI cost/liability.
- server/lib/secretbox.js: AES-256-GCM for the key at rest (never returned).
- routes/ai.js: GET/PUT /api/ai/settings (admin; key write-only) + POST
/generate-design (editor+). Output is strictly validated/normalized (cap count,
clamp ranges, px->%, strip HTML, validate colors) - never trust the model.
SSRF guard: hosted instances block private/internal targets; self-hosted (the
whole point of local AI) may point at localhost/LAN.
- Designer: an 'AI generate' panel (prompt + Generate) + a settings modal.
Verified end-to-end against local Ollama (llama3.1:8b): prompt -> editable design
on the canvas. Unit tests cover normalization + the SSRF guard. Suite 61/61.
Phase 2 (next): AI background images (OpenAI images / AUTOMATIC1111).
Multi-zone videos/images were cropped: every template zone inherited fit_mode
'cover' (fill+crop) and the layout editor had no control to change it, so a
landscape video in a tall split zone showed only a center strip. The player
already honors fit_mode (web object-fit, Android scaleType) - the gap was the UI
and the default. Add a per-zone Fit selector (Contain/Cover/Stretch) to the layout
editor, and make 'contain' (show the whole frame) the default for new zones, the
schema column, and the save fallbacks. Existing built-in templates are migrated
separately.
Smart TVs (Tizen, WebOS, Fire TV, Bravia) have no accessible browser
devtools, so when the player misbehaves on those platforms we previously
had zero visibility. This adds two paths to fix that:
- Visible debug overlay rendered on the TV screen for phone-photo capture
- Automatic server-side telemetry sink for hands-off error reporting
Client side (server/player/):
- Inline ES5 error trap as first script in index.html captures errors
even from parse-time failures in later scripts. Captures into
window.__debugLog with 200-entry cap.
- debug-overlay.js renders a fixed-position overlay covering the top 40%
of the screen. Activates via ?debug=1, d-e-b-u-g key sequence, Samsung
red button (keyCode 403), or smart-TV UA + ?autodebug=1. Freeze toggle
(F key or Samsung green) with visible FROZEN badge for phone capture.
pointer-events: none so touches pass through to the player underneath.
- Reporter machinery posts captured errors to /api/player-debug with
5-second debounce batching, sendBeacon on unload (with payload size
capping to stay under 64KB), 5-minute backoff after 429 responses.
UA-gated: smart-TV allow-list first (handles Tizen-with-Chrome/108),
modern-desktop deny-list second, default-report for unknown UAs.
- Two-pass djb2 fingerprint (16 hex chars) per error for future grouping.
- Absolute script src (/player/debug-overlay.js) so the script loads
regardless of trailing-slash on the player URL.
Server side:
- New player_debug_logs table (10000-row FIFO cap, indexed on
fingerprint + created_at). Schema in schema.sql, idempotent via
CREATE TABLE IF NOT EXISTS.
- POST /api/player-debug unauthenticated (so unpaired players can also
report), rate-limited 10/min/IP, per-field length caps to prevent abuse.
- Dynamic /player HTML route injects window.__playerConfig.debugReporting
based on PLAYER_DEBUG_REPORTING env var (defaults on; =off suppresses
all client telemetry traffic). Other player assets still served static.
- Admin routes (requireAuth + requireSuperAdmin):
GET /api/player-debug/list with pagination and filters
GET /api/player-debug/summary for UA family counts
DELETE /api/player-debug/older-than for manual purge
Admin view (#/admin/player-debug):
- UA family summary at top (Tizen/WebOS/Fire TV/Bravia/Edge/Chrome/etc)
- Filter row: UA contains, date range, has-error checkbox
- Paginated table with expand-row JSON viewer for error_data and context
- device_id labeled (self-reported) since field is unauthenticated input
- Manual delete-older-than button with confirmation dialog
Verified end-to-end with Playwright + Chromium (17/17 checks pass) plus
manual real-browser verification including UA-spoofed Tizen flow landing
rows in the admin view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 (assignments -> playlist_items) dropped zone_id during the
conversion: migrateAssignmentsToPlaylists INSERTed only (playlist_id,
content_id, widget_id, sort_order, duration_sec), and the new
playlist_items DDL omitted the zone_id column entirely. Every write
path on top of playlist_items inherited that omission - the
multi-zone layout assignment feature stopped working.
Frontend always sent zone_id correctly (device-detail.js:1015,1072
POST and PUT both include it; api.addAssignment and api.updateAssignment
forward the body verbatim). Server silently dropped it. The
assignments.js PUT route was the most direct evidence: it destructured
zone_id from req.body but never added it to the updates array.
Schema:
- schema.sql: add zone_id TEXT REFERENCES layout_zones(id) ON DELETE
SET NULL to fresh-install DDL.
- database.js migrations[]: add idempotent ALTER TABLE for existing
installs (the surrounding try/catch loop handles duplicate-column).
Backfill (new gated migration phase2_zone_id_backfill):
- Pre-migration snapshot copied to db/remote_display.pre-zone-id-
backfill-<ts>.db (one-off for this migration; the general
every-migration-snapshot framework is a separate concern, not built
here).
- Best-effort UPDATE playlist_items.zone_id from surviving
assignments rows via device.playlist_id + content_id/widget_id
match, LIMIT 1 for the multi-match edge case.
- Regenerates published_snapshot for every published playlist so the
JSON the player consumes carries zone_id going forward. Even with
zero rows backfilled (the common case post-Phase-2 cleanup) this
closes the snapshot-staleness gap.
- Stamps schema_migrations regardless so it won't re-run on next boot.
- On the live local DB: 0 playlist_items backfilled, 18
published_snapshots regenerated. On the April 13 prod fixture
(sandboxed copy): 0 backfilled, 7 regenerated. Expected and matches
our pre-flight finding that assignments was effectively scrubbed of
zone_id everywhere.
Route wiring (7 sites + 1 shared constant):
- assignments.js ITEM_SELECT: project pi.zone_id (read path so the
frontend display at device-detail.js:500 surfaces the value).
- assignments.js POST INSERT: include zone_id column + value.
- assignments.js PUT: actually use the already-destructured zone_id
in the updates allow-list. Treats undefined as "no change" so a PUT
that omits zone_id leaves the existing value intact; any explicit
value (including null) is written.
- assignments.js copy-to INSERT: preserve a.zone_id during
device-to-device playlist copy.
- playlists.js buildSnapshotItems: project pi.zone_id so the snapshot
JSON carries it. This is what the player's renderZones loop reads
(player/index.html:1338 matches a.zone_id === zone.id).
- playlists.js discard-revert INSERT: restore zone_id from snapshot
item on revert.
Out of scope (verified safe by SQL semantics + UI inspection):
- playlists.js POST item-add and PUT item-update in the playlist-detail
surface: the UI there doesn't expose zone editing, and their SQL
leaves zone_id NULL on insert / untouched on update. No regression.
- Other playlists.js SELECT projections (lines 141, 190, 240, 265, 334,
379, 419) all use SELECT pi.* and auto-pick zone_id once the column
exists.
- Kiosk-page assign at device-detail.js:1027 doesn't send zone_id;
separate pre-existing gap, not part of this regression.
Tests (all local, no push, no prod deploy):
- Migration boot on live local DB: clean, idempotent (second boot
skips the gated function).
- Migration boot on April 13 prod fixture (sandboxed copy at
/tmp/zone-fix-fixtures/test-run.db): cleanly runs the full migration
stack (multi-tenancy + 5 other phases the fixture predated) then
the new zone_id backfill. Live local DB untouched.
- 8 SQL-level route behavior tests pass: INSERT stores zone_id, PUT
changes/clears zone_id, ITEM_SELECT and buildSnapshotItems
projections include zone_id, copy-to preserves, discard-revert
restores from snapshot JSON, undefined zone_id in PUT leaves
existing value intact.
Not verified: end-to-end multi-zone playback on a real device. The
SQL + snapshot JSON layer is correct (player consumes
playlist.find(a => a.zone_id === zone.id) and now gets the right
zone_id back from the snapshot); confirming render-to-correct-zone
on actual hardware is the next step before prod deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wall editor: replaces the small grid with a Figma-style pan/zoom canvas. Each
display is a rectangle that can be dragged/resized to match its physical
arrangement; a separate semi-transparent player rect overlays the screens and
defines what content plays where. Drag empty space to pan, wheel to zoom,
"Center" button auto-fits content. Per-rect numeric x/y/w/h panel; arrow keys
nudge by 1px (10px with shift). Negative coordinates supported for screens
offset above/left of the origin. Coords rounded to integers on save.
Wall rendering: each device receives screen_rect + player_rect, maps the
player into its viewport with vw/vh and object-fit:fill so vertical position
of every source pixel is identical across devices that share viewport height.
Leader emits wall:sync at 4Hz with sent_at timestamp; followers apply
latency-adjusted target and use playbackRate ±3% for sub-300ms drift,
hard-seek for >300ms. Followers stay muted; leader unmutes via gesture with
AudioContext priming and pause+play retry to bypass Firefox autoplay.
"Tap to enable audio" overlay as a final fallback.
Reconnect handling: server re-evaluates leader on device:register so the
top-left tile reclaims leadership when it returns. Followers emit
wall:sync-request on entering wall mode (incl. reconnect) so they snap to
position immediately instead of drifting until the next periodic tick.
Group dissolve: removing a device from its last group clears its playlist
to mirror wall-leave semantics. Leaving a group with playlists on remaining
groups inherits the next group's playlist.
Dashboard: walls render as their own card section (hidden the device cards
they contain). Multi-select checkboxes on cards + "Create Video Wall" toolbar
action that creates the wall, removes devices from groups, and opens the
editor. dashboard:wall-changed broadcast triggers live re-render. Per-card
playback progress bar driven by play_start events forwarded from devices.
Security: PUT /walls/:id/devices verifies caller owns each device (or has
team-owner access via the widgets pattern), preventing cross-tenant device
takeover. wall:sync and wall:sync-request validate that the sending device
is a member of the named wall; relay re-stamps device_id with currentDeviceId
so clients can't spoof or shadow-exclude peers.
Schema: video_walls += player_x/y/width/height, playlist_id;
video_wall_devices += canvas_x/y/width/height. All idempotent migrations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CREATE INDEX on schedules(group_id) in schema.sql ran before the
phase4 migration added the group_id column, crashing on existing databases.
Move the index creation to the migration which already handles it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 4 group scheduling: schema migration adds group_id to schedules with
CHECK constraint, scheduler evaluates group+device schedules with priority,
group deletion converts schedules to per-device copies. Dashboard gets
playlist assignment dropdown and current playlist label on group headers.
Player persists audio unlock state in localStorage so version reloads
don't lose audio on unattended displays.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Schema: add status and published_snapshot columns to playlists table.
Migration snapshots all existing playlists as published (idempotent via schema_migrations).
Devices always receive the published_snapshot, not live playlist_items.
Edits from device-detail/groups auto-publish immediately (display updates instantly).
Edits from playlist detail page go to draft (requires explicit publish).
POST /playlists/:id/publish snapshots and pushes to all devices.
POST /playlists/:id/discard reverts playlist_items from published snapshot.
Content deletion scrubs references from all published snapshots.
Frontend: draft badge in playlist list, prominent yellow banner with publish/discard
buttons on playlist detail and device detail pages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New schema_migrations table (id TEXT PK, ran_at INTEGER) tracks which
one-time migrations have executed. The Phase 2 playlist migration now
checks for 'phase2_playlist_migration' in this table instead of
inferring state from devices.playlist_id. Records the migration ID
after successful completion. Eliminates ffprobe overhead on subsequent
startups.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every device will point to exactly one playlist. Schedules can temporarily
override a device's playlist. Auto-generated playlists (from migration) are
flagged so the UI can filter them.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 1 of playlist refactor: standalone playlist entities with ordered
items. No changes to existing tables or display behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ScreenTinker - open source digital signage management software.
MIT License, all features included, no license gates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
