mirror of https://github.com/screentinker/screentinker.git synced 2026-06-29 09:23:16 -06:00

History

screentinker 965920cd17 PiP overlay MVP: push image/web overlays to a device or group (#109 ) (#127 ) * PiP overlay MVP: push image/web overlays to a device or group (#109) Implements the #109 MVP from docs proposal: a floating overlay PUSHED to a device or group in real time, rendered above the playlist without disturbing it. Scope is the MVP only — video/RTSP, MQTT, offline-queue, and the priority/stacking system are deferred to follow-up PRs as the proposal specifies. Protocol (/device socket, player-agnostic): - device:pip-show { pip_id, type:image\|web, uri, position, width, height, duration, title?, title_color?, background_color?, opacity?, border_radius?, close_button? } - device:pip-clear { pip_id? } The player fetches uri itself (same trust model as remote_url content; server never proxies). type:web is full-trust by design, hence the 'full' token scope. Server (server/routes/pip.js, new; mounted in config/api-surface.js PUBLIC_ROUTERS): - POST /api/pip and POST /api/pip/clear + DELETE /api/pip, all requireScope('full'). - Resolves device_id to a device OR a group, expands a group to members, and emits per-device — reusing the group command route's room-size online check and {device_id, name, status: sent\|offline} result shape. Generates pip_id. - Validates type/position allowlists, uri http(s), numeric bounds on width/height/duration/opacity/border_radius, colors via the existing VALID_COLOR (#RRGGBB; transparency is the separate opacity field). - Workspace-isolated: every target query is scoped to req.workspaceId, so a token bound to workspace A can't address workspace B (404). Offline devices are reported, never queued (PiP is ephemeral). Player overlay layer (Tizen; tizen/js/pip-overlay.js, new): - A #pip sibling ABOVE #stage that PlaylistPlayer/ZoneRenderer never touch. - applyOrientation now applies the SAME transform to #pip as #stage, so corner positions track the visible CONTENT in all four orientations. - image -> <img>, web -> <iframe> (muted by default: empty allow= denies autoplay), sized/positioned/styled per payload, optional title bar. - Single overlay slot, last-show-wins; duration timer (0 = until cleared); pip-clear (id-aware) or timer tears down; teardown wrapped so a malformed payload can't wedge the layer. Reports show/clear over device:log (tag 'pip'). Dashboard: a minimal "Send overlay" / "Clear overlay" tester on the device-detail controls (device/group via the open device, type, uri, position, duration), calling POST /api/pip through the api helper. Tests (server suite green, 161/161): - api.test.js: PiP tier — authz (read/write 403, full passes), workspace isolation (wsA token -> wsB device 404), payload validation, device + group targeting, clear; plus the PUBLIC_ROUTERS snapshot-firewall updated for /api/pip. - pip-overlay.test.js: loads the real player.js + pip-overlay.js in a vm with a DOM shim; proves the overlay shows, auto-dismisses on the duration timer, and never changes the playlist signature / touches #stage; web->iframe, last-show-wins, id-aware clear, malformed-payload safety. Not in this PR (intentional): - Android player overlay — fast-follow. Protocol + server are player-agnostic; the Android layer (an overlay View above the player, orientation-matched to MainActivity's rootView rotation) is the same shape and lands next. - OpenAPI docs for POST /api/pip — the contract test's scope heuristic only treats 'command' paths as full-scope, so documenting a full-scope non-command route there needs that heuristic extended first; deferred with the docs item (proposal §8.6). - video/rtsp types, MQTT, offline queue-on-reconnect, priority/stacking, arbitrary (x,y)/selector positioning (proposal §6). Refs #109 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * PiP overlay: add Android + web players (#109) Extends the #109 PiP MVP to the other two players so the protocol (device:pip-show / device:pip-clear) is honored fleet-wide, not just on Tizen. No server/protocol changes — the route and socket messages are player-agnostic; these are the two missing surfaces. Web player (server/player/index.html): - New #pipContainer layer above #playerContainer, pointer-transparent, that the playlist render never touches. The same orientation transform is applied to it as to #playerContainer (extended to also reset width/height on landscape so a portrait->landscape switch realigns), so corner positions track the visible content. - Inline PiP logic mirroring tizen/js/pip-overlay.js: image -> <img>, web -> <iframe> (muted by default via empty allow=), position/size/bg/opacity/radius/title, single slot last-show-wins, duration timer (0 = until cleared), id-aware clear, wrapped teardown. - device:pip-show/clear handlers; reports show/clear over device:log (tag "pip"). Android player: - activity_main.xml: a pipLayout FrameLayout as the LAST child of rootLayout — it draws above the content AND inherits rootView's orientation rotation/translation, so corner positioning is orientation-matched for free. - PipOverlay.kt (new): builds the overlay box into pipLayout. image -> ImageView (decoded off-thread via ImageLoader, dropped if torn down mid-decode); web -> WebView with mediaPlaybackRequiresUserGesture=true (mute-by-default). Gravity-based corner/center placement with a 4% inset, GradientDrawable bg + corner radius, alpha=opacity, optional title bar. Single slot last-show-wins; duration timer; id-aware clear; teardown wrapped and also run on activity destroy (WebView cleanup). - WebSocketService: onPipShow/onPipClear callbacks + safeOn handlers posted to the main thread (they build Views) + a sendLog(tag, level, message) emitter for device:log. - MainActivity: instantiate PipOverlay (log -> wsService.sendLog("pip", ...)), wire the callbacks, tear down on destroy. Verified: Android assembleDebug builds clean; web player inline JS parses; server suite still 161/161 (no server changes this commit). Not yet validated on real hardware — four-orientation corner positioning mirrors the player container/rootView transform but should be eyeballed on a panel. Refs #109 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>		2026-06-18 14:54:44 -05:00
..
css	PiP overlay MVP: push image/web overlays to a device or group (#109 ) (#127 )	2026-06-18 14:54:44 -05:00
js	PiP overlay MVP: push image/web overlays to a device or group (#109 ) (#127 )	2026-06-18 14:54:44 -05:00
.gitignore	feat(tizen): Samsung Tizen TV web player (.wgt)	2026-06-09 19:01:58 -05:00
build-wgt.sh	Tizen player 1.9.1-beta3: bug fixes, multi-zone layouts, video walls	2026-06-18 13:28:08 -05:00
config.xml	Tizen player: real Samsung B2B fleet control (#125 ), folding in #126	2026-06-18 13:28:08 -05:00
icon.png	feat(tizen): Samsung Tizen TV web player (.wgt)	2026-06-09 19:01:58 -05:00
index.html	PiP overlay MVP: push image/web overlays to a device or group (#109 ) (#127 )	2026-06-18 14:54:44 -05:00
README.md	Tizen player: real Samsung B2B fleet control (#125 ), folding in #126	2026-06-18 13:28:08 -05:00

README.md

ScreenTinker — Tizen TV Player (`.wgt`)

A Samsung Tizen TV / signage web port of the ScreenTinker player. It speaks the exact same /device socket.io protocol as the Android player, so a Tizen display pairs and plays from the same dashboard with no server changes.

What it does

Enter a server URL → connects to {server}/device (socket.io v4).
Registers, shows a 6-digit pairing code; you claim it in the dashboard (Devices → Pair a display). On device:paired it switches to playback.
Reconnects automatically with a stored device_id + device_token.
Renders multi-zone layouts (matching the Android player) when a layout is assigned — each zone has its own percent geometry, z_index, fit_mode, background, and rotates its own assignments independently — and falls back to fullscreen single-zone when no layout is set, looping:
- image → shown for duration_sec (min 3s)
- video (/api/content/{id}/file or remote_url) → plays to end, then next; single item loops
- YouTube (mime video/youtube) → muted autoplay <iframe> embed
- widget → <iframe> of {server}/api/widgets/{id}/render
Sends device:heartbeat every 15s (with best-effort Tizen telemetry).
Keeps the screen awake (tizen.power / Samsung appcommon screensaver-off).
Video walls (mirrors the web player): when the device is a wall member the payload carries wall_config; the stage is positioned (in vw/vh) as this screen's slice of the wall, the leader broadcasts wall:sync and followers align index + drift-correct their video to the leader's clock. Per-tile rotation is not applied yet (matches the web player); video walls have no Android equivalent.

Files

config.xml          Tizen TV web-app manifest (privileges, profile, icon)
index.html          setup / pairing / stage screens
css/style.css
js/app.js           device protocol client (register, pair, heartbeat, state)
js/device-control.js Samsung B2B/system fleet control (device:command) — #125
js/player.js        fullscreen playlist renderer
js/socket.io.min.js socket.io-client v4.7.5 (bundled)
icon.png
build-wgt.sh        package (signed if Tizen CLI present, else unsigned)

Build

./build-wgt.sh            # -> ScreenTinker.wgt

Without the Tizen CLI this is an unsigned .wgt.

Why the released .wgt is unsigned: Samsung distributor certificates are locked to the DUID of the signer's own TVs, so a .wgt we signed would not install on your TV anyway. Releases therefore ship it unsigned (for inspection only). To actually run it, use path A (no signing) or sign it yourself with your own certificate (path B).

Deploy — two paths

A) URL Launcher / TV browser (easiest, no signing)

No package, no Tizen Studio. Point the TV's URL Launcher (or just its web browser) at your server's built-in web player: https://<your-instance>/player. The TV runs it as a web app on boot, pairs with a 6-digit code, and plays - best for Samsung B2B signage (SSSP). (You can instead self-host this tizen/ folder and point the URL Launcher at …/index.html for the Tizen-specific build.)

B) Signed `.wgt` (installed app)

A signing profile is already set up on the build box (Tizen Studio CLI 6.1):

Profile ScreenTinker = a self-signed author cert (~/tizen-studio-data/keystore/author/st_author.p12) + the default Tizen distributor cert. ./build-wgt.sh auto-detects the CLI and signs with it, producing a .wgt with author-signature.xml + signature1.xml.
This installs on developer-mode Samsung TVs and the Tizen emulator — the right path for a self-hosted fleet you control (enable Developer Mode on each TV once: Apps → enter 12345 → set the host IP).

Install onto a dev-mode TV:

sdb connect <tv-ip>
tizen install -n ScreenTinker.wgt -t <tv-device>

Production / retail (no developer mode): re-sign with a Samsung Partner or Public distributor certificate from the Tizen Certificate Manager (free Samsung account; distributor cert tied to each TV's DUID), then ./build-wgt.sh <thatProfile>. The self-signed author cert is not committed (it lives in ~/tizen-studio-data, password screentinker).

Validated (2026-06-09)

Protocol: headless test against the live server passed end-to-end — register(pairing_code) → device:registered → pair → reconnect(device_id+token) → device:playlist-update(2 items) → GET /api/content/{id}/file = 200.
Runtime: loads + renders in Chromium with no JS errors (setup screen verified).
Not yet on real Tizen hardware — needs signing + a TV (or URL Launcher).

Remote control & preview (#120 / #121 / #125)

The Tizen player listens for the same dashboard events as the web/Android player. device:command is handled by js/device-control.js, which drives the real Samsung fleet-control surface (webapis.systemcontrol on Tizen 6.5/7, else b2bapis.b2bcontrol on SSSP/Tizen 4) and reports each outcome back via device:log (tag command, shown live on the device-detail screen) plus a structured device:command-result:

Command (`device:command` type)	Tizen behaviour
`refresh` / `reload`	`location.reload()`
`launch` / `screen_on`	clears the screen-off overlay + re-asserts wake; `setPanelMute("OFF")` when the B2B surface is present
`screen_off`	`setPanelMute("ON")` (backlight off) on a B2B panel; black overlay fallback otherwise
`update`	reload to re-pull URL-Launcher content (no in-app OTA — see Updates)
`reboot`	`rebootDevice()` on a B2B panel; `unsupported` otherwise
`shutdown`	`setPanelMute("ON")` + note (SSSP web API has no true power-off)
unknown	reported as `unsupported`
`device:screenshot-request`	best-effort capture (see note)
`device:remote-start` / `-stop`	start/stop ~1 fps preview streaming

Partner-signing caveat (#125): the b2bcontrol / systemcontrol privileges in config.xml only take effect on a partner-signed .wgt on a real SSSP panel. On the dev/URL-Launcher/web build (or a consumer TV) those surfaces are absent, so reboot returns unsupported, screen_off uses the black overlay, and the startup capability log reports backend=none. Only a partner-signed build on real hardware fully validates reboot / panel power.

Screenshot/preview note: the TV decodes <video> onto a hardware overlay plane and plays YouTube in a cross-origin <iframe>, neither of which can be read back into a <canvas>. So images capture for real; video/YouTube fall back to a status card (device + timestamp). The dashboard preview shows a truthful frame rather than a dead button. Full-fidelity video preview isn't feasible on the sideloaded Tizen runtime.

Updates (#122)

There is no in-app OTA for a sideloaded, signed .wgt. Updating a screen means re-building and re-sideloading the .wgt (path B above), or — on Samsung B2B signage — pushing it through the URL Launcher refresh / MDM (MagicINFO / SSSP) channel. The dashboard update command therefore just tells the screen an update is pending; it cannot self-apply. If you run the URL Launcher path (A), a plain TV reboot re-fetches …/player and you're current with the server with no .wgt step.

Auto-launch on boot (#122)

Boot auto-start for a sideloaded consumer TV web app is a display setting, not an app setting — there's no config.xml autostart for the TV profile. Configure it on the panel:

URL Launcher path (A): set the URL Launcher as the boot app (it relaunches on power-up automatically) — the recommended signage setup.
Signed-app path (B): use the TV's kiosk / auto-start app setting (B2B/SSSP firmware) to launch ScreenTinker on boot; on dev-mode consumer TVs there's no guaranteed boot-launch, so the URL Launcher path is preferred for unattended screens.

Version reporting (#119)

app_version is sourced from config.xml's version="" — read at runtime via the Tizen application API, with a build-stamped constant fallback (build-wgt.sh stamps it from config.xml). The dashboard always shows the version actually installed.