Archive/screentinker
1
0
Fork 0
mirror of https://github.com/screentinker/screentinker.git synced 2026-08-14 22:33:12 -06:00
Code Issues Actions 10 Packages Projects Releases Wiki Activity

web player: declare capabilities at runtime, persist them, and audit all four players

Browse source 
The dashboard offered every control to every display, so a browser tab showed a
reboot button that could never work. server/lib/player-capabilities.js defines the
vocabulary; this makes the web player actually speak it.

The declaration is computed, not constant, because the same index.html is BOTH the
browser player and the BrightSign player. system.reboot / display.power /
display.resolution / system.self_update are claimed only when BS.hasHost() answers —
deliberately hasHost() and not isBrightSign(), since the UA check is also true for a
widget built without node integration, which can reach none of them. Screenshots,
offline cache, transitions and native sync are each probed the same way.

Capabilities were never persisted: the column and the handler did not exist, so a
declaration would have been sent and silently dropped. Added the migration and
applyCapabilities(). An ABSENT declaration leaves the column NULL so the baseline
still applies — several hundred fielded displays declare nothing and would otherwise
lose every control at once — while an EMPTY declaration is stored as '[]' and honoured.

docs/player-parity.md records every capability against all four players with a reason
for each "no", and flags three Tizen baseline errors found while verifying it.

Tests: 1109/1109. Both inline <script> blocks in index.html parse clean.
This commit is contained in:
ScreenTinker 2026-08-05 14:17:40 -05:00
parent 6bc709d2f7
commit c4ee7d008f
6 changed files with 412 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

118
docs/player-parity.md Normal file
View file

@ -0,0 +1,118 @@
# Player parity matrix
What each player can actually do, verified against the code rather than assumed. This is the
document that says where the remaining work is, so a wrong "yes" here is worse than a missing row:
it puts a control on the dashboard that cannot work.
Capability names come from `server/lib/player-capabilities.js`. Players declare their own set at
registration; a player that declares nothing falls back to the per-platform baseline in that file.
**Legend** — ✅ supported · ⚠️ partial/conditional (reason given) · ❌ not supported (reason given)
BrightSign runs the *same* `server/player/index.html` as the browser, so it differs only where the
`autorun.brs` host bridge adds something the browser cannot reach.
## Playback
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `playback.video` | ✅ ExoPlayer | ✅ `<video>` | ✅ AVPlay | ✅ hardware plane |
| `playback.image` | ✅ | ✅ | ✅ | ✅ |
| `playback.widget` | ✅ WebView | ✅ iframe | ✅ iframe | ✅ iframe |
| `playback.youtube` | ✅ WebView embed | ✅ IFrame API | ✅ iframe embed | ✅ IFrame API |
| `playback.zones` | ✅ | ✅ | ✅ | ✅ |
| `playback.transitions` | ✅ GL wipes (#204) | ⚠️ declared only when the bundle loads — a failed load hard-cuts rather than breaking playback | ✅ | ⚠️ as web |
| `playback.pip` | ✅ `PipOverlay` | ✅ `#pipContainer` | ✅ | ✅ |
## Audio
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `audio.mute` | ✅ incl. YouTube via IFrame bridge | ✅ | ✅ incl. YouTube via `postMessage` | ✅ as web |
| `audio.volume` | ✅ `set_volume` | ✅ `set_volume` | ❌ **no `set_volume` handler exists** — the dashboard slider does nothing today | ✅ as web |
## Display
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `display.rotation` | ✅ native `rootView.rotation` | ✅ CSS transform | ✅ CSS + AVPlay for video | ⚠️ host rotates the output via `roVideoMode`; CSS alone cannot turn the hardware video plane |
| `display.power` | ✅ `screen_off` / `lock_now` | ❌ a browser tab cannot power a panel — the overlay only paints black | ❌ `screen_off` draws a black overlay, deliberately, "so the command still does something visible" | ⚠️ media teardown always works; CEC is best-effort and absent on some units |
| `display.resolution` | ❌ no video-mode control in the app | ❌ not addressable from a browser | ❌ | ✅ `roVideoMode` via the host |
## Remote view and control
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `remote.screenshot` | ⚠️ view capture always; full-screen only with accessibility or MediaProjection | ⚠️ canvas only — same-origin content, and the alpha probe rejects frames where no pixels arrived | ✅ `captureAndSend` | ⚠️ host framebuffer capture **requires primary storage**; falls back to canvas, which cannot read the video plane |
| `remote.stream` | ✅ | ✅ 1fps | ✅ | ⚠️ as web |
| `remote.input` | ✅ | ✅ | ✅ | ✅ |
## Lifecycle
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `system.restart_player` | ✅ | ✅ `location.reload()` | ✅ | ✅ host rebuilds the widget — a page reload does not reliably return |
| `system.reboot` | ✅ device owner | ❌ a browser tab cannot reboot its host | ❌ no Tizen API exposed to the app | ✅ `RebootSystem()` via the host |
| `system.self_update` | ✅ APK OTA (`UpdateChecker`) | ❌ the server deploys the player; there is nothing for it to update | ❌ `.wgt` updates go through Tizen's own store/CLI | ✅ `autorun.zip` package update |
## Device management
Android device-owner territory. Everything here is ❌ elsewhere for the same reason — no equivalent
privilege model exists on those platforms — so the column is collapsed.
| capability | Android | Web / Tizen / BrightSign |
|---|---|---|
| `system.kiosk` | ✅ lock-task, now persisted across reboot | ❌ no device-owner concept |
| `system.brightness` | ✅ Tier 0/1 | ❌ |
| `system.screen_timeout` | ✅ Tier 1 | ❌ |
| `system.install_apk` | ✅ Tier 2 | ❌ not an APK platform |
| `system.shell` | ✅ Tier 2, handled in `WebSocketService` | ❌ |
| `system.time` | ✅ Tier 2 | ❌ |
## Synchronisation and resilience
| capability | Android | Web | Tizen | BrightSign |
|---|---|---|---|---|
| `sync.clock` | ✅ | ✅ | ✅ | ✅ |
| `sync.native` | ❌ no native protocol | ❌ | ❌ | ⚠️ SyncManager, BOS 8.2.10+; multicast so all members must share one L2 network |
| `offline.cache` | ✅ content downloaded to disk | ✅ service worker | ⚠️ **playlist payload only** — `st_payload_cache` replays the last renderable payload, but there is no service worker, so media still needs the network | ✅ service worker + 1GB storage quota |
---
## Real gaps worth closing
Ordered by how visible the failure is to an operator.
1. **Tizen `audio.volume` — dead control.** `set_volume` has no handler in `tizen/js/app.js`; the
only volume path is the on-device `KEYCODE_VOLUME_*` keys. The dashboard slider silently does
nothing. Either implement the handler or let the capability hide the control.
2. **Tizen `offline.cache` is partial.** The playlist survives a reboot; the media does not. A
Tizen panel that loses its uplink keeps its schedule and cannot play it. This is the largest
functional gap in the table.
3. **BrightSign `remote.screenshot` needs primary storage.** Reachable today only via the canvas
fallback, which cannot read the video plane, so screenshots show everything except the video.
Resolves itself when a card or SSD is fitted.
4. **`display.resolution` is BrightSign-only.** Fine, but the dashboard should not offer it
elsewhere.
## Correctly impossible — do not "fix" these
- **`system.reboot` on web/Tizen.** No API exists. A browser tab rebooting its host would be a
browser vulnerability.
- **`display.power` on web.** The overlay is the honest maximum; the panel stays lit.
- **All of device management off Android.** No equivalent privilege model exists on Tizen or
BrightSign, and a web player has no device to manage.
- **`system.self_update` on web.** The player *is* the deployment; there is nothing to update.
- **`sync.native` off BrightSign.** It is BrightSign's own protocol, and the clock-derived one is
the cross-platform answer that already works everywhere.
## ⚠️ Corrections needed in `player-capabilities.js`
Found while verifying this table. The baselines only apply to displays that declare nothing, so
these are wrong for the existing fleet until each player ships its declaration:
- **`tizen` claims `audio.volume`** — no handler exists (gap 1 above). Should be removed.
- **`tizen` omits `remote.screenshot` and `remote.stream`** — both are implemented
(`captureAndSend`, `startStreaming`). Should be added.
- **`tizen` claims `offline.cache`** — true only for the playlist payload, not media. Either keep
it with the partial meaning documented, or split the capability.

6
server/db/database.js
View file

@ -508,6 +508,12 @@ const migrations = [
('z-pp-1', 'tpl-p-pip', 'Background', 0, 0, 100, 100, 0, 0),
('z-pp-2', 'tpl-p-pip', 'PiP Window', 58, 4, 38, 20, 1, 1)`,
// What each player declares it can do (JSON array), so the dashboard can hide controls a
// display cannot honour. NULL means "never declared" and falls back to a per-platform baseline
// in server/lib/player-capabilities.js — distinct from '[]', which is a player genuinely saying
// it can do nothing and must be respected.
'ALTER TABLE devices ADD COLUMN capabilities TEXT',
];
// Apply each ALTER idempotently. A "duplicate column name" / "already exists"
// error means the column is already present (expected on a migrated DB) - benign.

90
server/player/index.html
View file

@ -1634,6 +1634,93 @@
if (had) pipReport('info', 'pip cleared' + (pipId ? ' ' + pipId : ''));
}
/*
* What THIS instance can actually do, decided at registration time.
*
* The dashboard used to offer every control to every display, so buttons that could not
* possibly work were shown and did nothing — a browser tab cannot reboot its host. The server
* falls back to a per-platform baseline when a player declares nothing, so this list only has
* to be RIGHT, not complete.
*
* It is computed rather than constant because the same index.html is BOTH the browser player
* and the BrightSign player: identical code, wildly different powers depending on whether a
* host bridge answered. Declaring a static set here would either lie for the browser or
* under-sell BrightSign.
*
* Everything conditional is probed, never assumed. A capability we cannot verify is left out:
* an absent control is a smaller failure than one that reports success and changes nothing.
*/
function declaredCapabilities() {
const caps = [
// Render paths that exist in this file unconditionally.
'playback.video', 'playback.image', 'playback.widget', 'playback.youtube',
'playback.zones', 'playback.pip',
// #129 per-item mute plus the operator volume added alongside it.
'audio.mute', 'audio.volume',
// CSS transform in a browser; the host rotates the output on BrightSign. Both real.
'display.rotation',
'remote.input',
'remote.stream',
// restartPlayer() always resolves to something: the host rebuilds the widget, or the page
// reloads. It is the one lifecycle action a browser tab can genuinely perform.
'system.restart_player',
'sync.clock',
];
// Transitions are a progressive enhancement — the bundle failing to load means hard cuts,
// not a broken player — so this reflects whether the runtime actually loaded.
//
// Deliberately the player's OWN predicate rather than a second guess at the global names.
// An earlier draft here tested window.STTransitions, which does not exist: the capability
// would simply never have been declared, hiding the transitions UI for every display, and
// nothing would have failed loudly enough to notice.
try {
if (typeof transitionRuntimeReady === 'function' && transitionRuntimeReady()) {
caps.push('playback.transitions');
}
} catch (e) { /* bundle absent: hard cuts, and we do not claim the capability */ }
// Offline caching is the service worker. Reported on support rather than on an active
// controller: the first load registers it and has no controller yet, and a display that
// re-registers on every boot would otherwise flap this capability on and off.
try {
if ('serviceWorker' in navigator) caps.push('offline.cache');
} catch (e) { /* locked-down browser */ }
// Screenshots need somewhere to draw. Same-origin content and a 2d context are the real
// requirements; a host bridge is a second, better path (it captures the framebuffer
// including the hardware video plane, which a canvas cannot read at all).
try {
const probe = document.createElement('canvas');
probe.width = 2; probe.height = 2;
if (probe.getContext('2d') && typeof probe.toDataURL === 'function') {
caps.push('remote.screenshot');
}
} catch (e) { /* no canvas => no screenshots, and we do not claim them */ }
// --- everything below exists ONLY behind a live host bridge ---
// hasHost() is specifically NOT isBrightSign(): the UA check alone is true for a widget
// built without node integration, which can reach none of this. Claiming reboot on one of
// those is exactly the dead button this whole exercise removes.
let host = false;
try { host = !!(BS && BS.hasHost()); } catch (e) { host = false; }
if (host) {
caps.push('system.reboot'); // RebootSystem() in autorun.brs
caps.push('display.power'); // media teardown + CEC; blanking works even where CEC does not
caps.push('display.resolution'); // roVideoMode via the bridge
caps.push('system.self_update'); // the autorun.zip package path
}
// Native frame-accurate sync needs the SyncManager module to have actually resolved, which
// is a firmware-version question (BOS 8.2.10+), not a platform one.
try {
if (window.ScreenTinkerBSSync && window.ScreenTinkerBSSync.available()) caps.push('sync.native');
} catch (e) { /* not on BrightSign */ }
return caps;
}
function register() {
const data = {};
// #163: always send device identity when we have it, regardless of paired
@ -1681,6 +1768,9 @@
data.sync_backend = BS.syncBackend();
} catch (e) { /* identity extras are additive — never block registration */ }
}
// What this instance can actually do. The server persists it and the dashboard hides the
// controls it does not contain, so a display never shows a button that cannot work.
data.capabilities = declaredCapabilities();
data.contract_version = 'v4';
// Device identity. `fingerprint` is per-INSTALL and is what the server matches on.
// `hw_fingerprint` is the old hardware-only value, sent alongside so a panel whose storage

57
server/test/device-capabilities-persist.test.js Normal file
View file

@ -0,0 +1,57 @@
'use strict';
// A player declares what it can do; the dashboard hides the controls it cannot honour.
//
// The failure this guards is the one that would hit hardest: several hundred displays are already
// in the field and declare NOTHING. If an absent declaration were persisted as "supports nothing"
// they would all lose their controls the moment this shipped. So absent must leave the column NULL
// (baseline applies) while an EMPTY declaration is stored as '[]' and honoured — a real statement
// from, say, a BrightSign widget with no host bridge.
//
// Those two cases differ by one character in the payload and by an entire dashboard in effect.
const { test } = require('node:test');
const assert = require('node:assert/strict');
const caps = require('../lib/player-capabilities');
// The exact filter the socket handler applies before writing.
const persistedValue = (raw) => {
const declared = caps.parseDeclared(raw);
return declared === null ? null : JSON.stringify(declared);
};
test('THE FLEET CASE: an absent declaration is not persisted, so the baseline still applies', () => {
assert.equal(persistedValue(undefined), null);
assert.equal(persistedValue(null), null);
const legacy = { client_type: 'apk' }; // column stays NULL
assert.ok(caps.supports(legacy, 'system.reboot'), 'legacy Android keeps its controls');
});
test('an EMPTY declaration IS persisted and is honoured as "nothing"', () => {
assert.equal(persistedValue([]), '[]');
assert.deepEqual(caps.capabilitiesFor({ client_type: 'apk', capabilities: '[]' }), []);
});
test('a hostile or malformed declaration never reaches the dashboard', () => {
// Not persisted at all -> the device keeps its baseline rather than gaining anything.
for (const bad of ['not json', '{"a":1}', 42, ' ']) assert.equal(persistedValue(bad), null);
});
test('unknown capability names are dropped, known ones survive', () => {
// A newer player declaring something this server has never heard of must not lose the rest.
assert.equal(persistedValue(['playback.video', 'quantum.teleport']), '["playback.video"]');
});
test('a stored declaration overrides the baseline in both directions', () => {
const stripped = { client_type: 'apk', capabilities: persistedValue(['playback.video']) };
assert.equal(caps.supports(stripped, 'system.reboot'), false, 'declared set wins over baseline');
const hosted = { platform: 'brightsign', capabilities: persistedValue(['system.reboot', 'sync.native']) };
assert.ok(caps.supports(hosted, 'sync.native'), 'a BrightSign with SyncManager can declare it');
});
test('the round trip is stable — persisted output re-parses to the same set', () => {
const declared = ['playback.video', 'audio.mute', 'system.reboot'];
const stored = persistedValue(declared);
assert.deepEqual(caps.capabilitiesFor({ capabilities: stored }), declared);
});

119
server/test/player-declared-capabilities.test.js Normal file
View file

@ -0,0 +1,119 @@
'use strict';
// The same index.html is BOTH the browser player and the BrightSign player. Identical code,
// wildly different powers depending on whether a host bridge answered.
//
// That is why the declaration is computed rather than constant, and why it is worth testing: a
// browser tab claiming `system.reboot` puts a button on the dashboard that cannot ever work, and
// a BrightSign failing to claim it hides one that would. Both are the same bug in opposite
// directions.
//
// The real function is extracted from index.html and run against fake globals, so this asserts
// what the player will actually send rather than a reimplementation of it.
const { test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const vm = require('node:vm');
const { CAP_SET } = require('../lib/player-capabilities');
const HTML = fs.readFileSync(path.join(__dirname, '..', 'player', 'index.html'), 'utf8');
/** Pull declaredCapabilities() out of the player and run it in a controlled world. */
function declare({ host = false, sync = false, transitions = false, canvas = true, sw = true } = {}) {
const start = HTML.indexOf(' function declaredCapabilities() {');
assert.notEqual(start, -1, 'declaredCapabilities() must exist in the player');
let depth = 0, end = -1;
for (let i = HTML.indexOf('{', start); i < HTML.length; i++) {
if (HTML[i] === '{') depth++;
else if (HTML[i] === '}' && --depth === 0) { end = i + 1; break; }
}
const src = HTML.slice(start, end);
const sandbox = {
console: { log() {}, warn() {} },
navigator: sw ? { serviceWorker: {} } : {},
document: {
createElement: () => (canvas
? { width: 0, height: 0, getContext: () => ({ drawImage() {} }), toDataURL: () => 'data:,' }
: { width: 0, height: 0, getContext: () => null }),
},
BS: host ? { hasHost: () => true } : null,
};
sandbox.window = sandbox;
if (sync) sandbox.ScreenTinkerBSSync = { available: () => true };
// The real runtime globals the player's own transitionRuntimeReady() checks. An earlier draft
// faked a global that does not exist, which passed a test while declaring nothing.
if (transitions) {
sandbox.TransitionRenderer = {};
sandbox.TransitionParams = {};
sandbox.__TRANSITION_SHADERS = {};
}
vm.createContext(sandbox);
// declaredCapabilities() calls transitionRuntimeReady(), which lives elsewhere in the player.
const predicate = 'function transitionRuntimeReady() { return !!(window.TransitionRenderer && window.TransitionParams && window.__TRANSITION_SHADERS); }';
vm.runInContext(`${predicate}\n${src}\nvar __out = declaredCapabilities();`, sandbox);
return sandbox.__out;
}
test('every declared name is a real capability the server understands', () => {
// A typo here is silently dropped by parseDeclared, so the control just never appears.
for (const c of declare({ host: true, sync: true, transitions: true })) {
assert.ok(CAP_SET.has(c), `player declares unknown capability "${c}"`);
}
});
test('THE LIE WE ARE REMOVING: a browser tab does not claim host-only powers', () => {
const web = declare({ host: false });
for (const cap of ['system.reboot', 'display.power', 'display.resolution', 'system.self_update']) {
assert.ok(!web.includes(cap), `a browser tab must not claim ${cap}`);
}
});
test('the same code behind a live host DOES claim them', () => {
const bs = declare({ host: true });
for (const cap of ['system.reboot', 'display.power', 'display.resolution', 'system.self_update']) {
assert.ok(bs.includes(cap), `a hosted player must claim ${cap}`);
}
});
test('what both share is declared by both', () => {
const web = declare({ host: false });
const bs = declare({ host: true });
for (const cap of ['playback.video', 'playback.zones', 'audio.mute', 'audio.volume',
'display.rotation', 'remote.input', 'system.restart_player', 'sync.clock']) {
assert.ok(web.includes(cap), `web should declare ${cap}`);
assert.ok(bs.includes(cap), `hosted should declare ${cap}`);
}
});
test('native sync depends on the module resolving, not on being a BrightSign', () => {
// BOS 8.2.10+ is a firmware question. A hosted player on older firmware has no SyncManager and
// must not offer frame-accurate sync it cannot perform.
assert.ok(!declare({ host: true, sync: false }).includes('sync.native'));
assert.ok(declare({ host: true, sync: true }).includes('sync.native'));
});
test('screenshots are claimed only when there is something to draw on', () => {
assert.ok(declare({ canvas: true }).includes('remote.screenshot'));
assert.ok(!declare({ canvas: false }).includes('remote.screenshot'));
});
test('offline cache follows service-worker support', () => {
assert.ok(declare({ sw: true }).includes('offline.cache'));
assert.ok(!declare({ sw: false }).includes('offline.cache'));
});
test('transitions are declared only when the bundle actually loaded', () => {
// It is a progressive enhancement — a failed load means hard cuts, not a broken player.
assert.ok(!declare({ transitions: false }).includes('playback.transitions'));
assert.ok(declare({ transitions: true }).includes('playback.transitions'));
});
test('a hostile environment degrades instead of throwing during registration', () => {
// This runs inside register(). Throwing here would cost the display its registration entirely,
// which is a far worse outcome than an under-declared capability set.
assert.doesNotThrow(() => declare({ canvas: false, sw: false, host: false }));
});

22
server/ws/deviceSocket.js
View file

@ -17,6 +17,7 @@ const flapLimiter = require('../lib/flap-limiter');
const sessionSettle = require('../lib/session-settle'); // #148 patch2: eviction-storm debounce
const { resolveIdentity } = require('../lib/device-identity');
const { resolveSyncBackend } = require('../lib/sync-backend');
const playerCapabilities = require('../lib/player-capabilities');
const logCoalescer = require('../lib/log-coalescer');
const loopLag = require('../services/loop-lag');
const deviceSettings = require('../lib/device-settings'); // #150 delete+re-pair settings restore
@ -141,6 +142,26 @@ function applyHardwareIdentity(deviceId, data) {
.run(model, serial, osVersion, output, deviceId);
}
/*
* Persist what the player says it can do.
*
* Routed through parseDeclared so a malformed or hostile declaration can never reach the
* dashboard: unknown capability names are dropped, and anything that is not an array is treated
* as no declaration at all.
*
* The null/empty distinction is the whole point and is easy to lose. A player that declares
* NOTHING (old build, absent field) must keep its per-platform baseline, or the several hundred
* displays already in the field lose their controls the moment this ships. A player that declares
* an EMPTY set is making a real statement — a BrightSign widget with no host bridge, say — and is
* stored as '[]' so it is honoured rather than silently upgraded.
*/
function applyCapabilities(deviceId, data) {
const declared = playerCapabilities.parseDeclared(data && data.capabilities);
if (declared === null) return; // no usable declaration: leave the baseline in charge
db.prepare('UPDATE devices SET capabilities = ? WHERE id = ?')
.run(JSON.stringify(declared), deviceId);
}
function generateDeviceToken() {
return crypto.randomBytes(32).toString('hex');
}
@ -866,6 +887,7 @@ module.exports = function setupDeviceSocket(io) {
// register payload, not device_info, so the emptiness guard above does not apply to
// them. The function no-ops when the panel reports none of them.
applyHardwareIdentity(device_id, data);
applyCapabilities(device_id, data);
heartbeat.registerConnection(device_id, socket.id);
// #134: a same-socket re-register is a playlist REFRESH (~45-60s), NOT a reconnect and NOT