Archive/screentinker
1
0
Fork 0
mirror of https://github.com/screentinker/screentinker.git synced 2026-06-29 09:23:16 -06:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
405 commits 4 branches 21 tags 7.4 MiB
Commit graph

2 commits

Author SHA1 Message Date
screentinker 7660d7433e
fix(#109): render Android PiP overlay above the YouTube WebView video plane (#135) 
* fix(#109): render Android PiP overlay above the YouTube WebView video plane

The PiP overlay (#109) returned sent:1 and showed its title in `uiautomator
dump`, but nothing painted on screen while YouTube was playing. By elimination
(YouTube-specific, landscape so no off-screen transform, real on-screen bounds
in the dump) the cause is surface occlusion: pipLayout sat as the last child of
rootLayout — the SAME compositing band as R.id.youtubeWebView — so the playing
video surface drew over it.

Fix (task option 1a): reparent pipLayout out of rootLayout to the window
content (android.R.id.content) as a top-level sibling drawn after rootLayout, so
it composites above the WebView. MainActivity.mirrorTransformToPip() copies
rootView's orientation/wall transform onto it so corner positions still track
the rotated content (web/Tizen parity). show() also bringToFront()+
requestLayout()+invalidate() on attach (covers the cause-3 measure/visibility
path). Remote-view screenshots now capture the content root so the PiP is still
included.

Instrumentation (Phase 1, default OFF): PipOverlay.pipDebug paints a solid
magenta box + border with media on top (box paints even if media never loads)
and logs box/pipLayout/rootView/youtubeWebView geometry over device:log tag
"pip"; loadImageInto also logs on success. Toggled via device:command
{type:"pip_debug"} (routed through MainActivity.onCommand).

Server: POST /api/pip and the clear handler log one concise [pip] dispatch line
(target + sent/offline) so journalctl shows PiP activity.

Validated end-to-end on an emulator (pixel10/API34) paired to an isolated local
server with YouTube playing: no crash, the PiP box composites above the live
video frame (center + top-right), clear removes it, and the portrait transform
mirror rotates the overlay with the stage (no off-screen). The Fire TV
hardware-overlay punch-through still needs real hardware (emulator composites
video inline); pipDebug + docs/109-android-pip-visibility.md cover that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#109): image PiPs never painted — set slot token before decode

Emulator e2e of an image PiP (a QR PNG) found the image area always blank (box
background + title only). Pre-existing defect, also on main, independent of the
occlusion reparent.

Root cause in PipOverlay.show(): teardown() clears `current` to null, then
loadImageInto() captured `token = current` (null) as its drop-if-replaced guard,
but `current` was set to the new pip_id AFTER the media was built. The image
decode finishes on a background thread and posts back after show() returns, so
`token != current` (null != pip_id) was always true and every decoded bitmap was
dropped. Web PiPs and the box/title were unaffected, which masked it.

Fix: set `current = pip_id` before building media so loadImageInto's token
matches. Verified on emulator — a QR image PiP now renders over both a static
image and live YouTube (hardware screencap + the app's software view.draw
capture both show it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(#109): record web PiP (HTML+JS) verification on emulator

Web PiP type loads its WebView and executes JS (a page stamping JS OK · <time>
rendered over live YouTube). No code change — web PiPs don't use the image path
that had the token bug. Completes the image/web/box content-type verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(#109): implement PiP close_button on Android (was a documented no-op)

The server forwarded close_button (routes/pip.js) and it's in openapi.yaml, but
no player rendered it — Tizen deferred "close-button focus" as non-MVP, the web
player has none, and Android's PipOverlay never read the flag. So the documented
field did nothing on any device.

Implement it on Android: when close_button:true, a tappable ✕ floats at the box's
top-right in a FrameLayout wrapper that is a SIBLING of the box — so it isn't
clipped by the box outline or dimmed by the overlay opacity. Tapping it clears
THIS overlay (id-matched via the captured token). Only the ✕ is clickable; the
rest of the full-screen pipLayout stays touch-transparent, so taps elsewhere
fall through to the playing content (no input regression).

Verified on the emulator over live YouTube: the ✕ renders at the corner, and
tapping it removes the overlay while the video keeps playing.

Parity note: web/Tizen players still don't implement close_button; D-pad focus
of the ✕ on non-touch TV hardware is intentionally not wired (MVP = touch/pointer,
matching the Tizen focus deferral).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-19 14:49:20 -05:00
screentinker 965920cd17
PiP overlay MVP: push image/web overlays to a device or group (#109) (#127) 
* PiP overlay MVP: push image/web overlays to a device or group (#109)

Implements the #109 MVP from docs proposal: a floating overlay PUSHED to a device or
group in real time, rendered above the playlist without disturbing it. Scope is the
MVP only — video/RTSP, MQTT, offline-queue, and the priority/stacking system are
deferred to follow-up PRs as the proposal specifies.

Protocol (/device socket, player-agnostic):
- device:pip-show { pip_id, type:image|web, uri, position, width, height, duration,
  title?, title_color?, background_color?, opacity?, border_radius?, close_button? }
- device:pip-clear { pip_id? }
The player fetches uri itself (same trust model as remote_url content; server never
proxies). type:web is full-trust by design, hence the 'full' token scope.

Server (server/routes/pip.js, new; mounted in config/api-surface.js PUBLIC_ROUTERS):
- POST /api/pip and POST /api/pip/clear + DELETE /api/pip, all requireScope('full').
- Resolves device_id to a device OR a group, expands a group to members, and emits
  per-device — reusing the group command route's room-size online check and
  {device_id, name, status: sent|offline} result shape. Generates pip_id.
- Validates type/position allowlists, uri http(s), numeric bounds on
  width/height/duration/opacity/border_radius, colors via the existing VALID_COLOR
  (#RRGGBB; transparency is the separate opacity field).
- Workspace-isolated: every target query is scoped to req.workspaceId, so a token
  bound to workspace A can't address workspace B (404). Offline devices are reported,
  never queued (PiP is ephemeral).

Player overlay layer (Tizen; tizen/js/pip-overlay.js, new):
- A #pip sibling ABOVE #stage that PlaylistPlayer/ZoneRenderer never touch.
- applyOrientation now applies the SAME transform to #pip as #stage, so corner
  positions track the visible CONTENT in all four orientations.
- image -> <img>, web -> <iframe> (muted by default: empty allow= denies autoplay),
  sized/positioned/styled per payload, optional title bar.
- Single overlay slot, last-show-wins; duration timer (0 = until cleared); pip-clear
  (id-aware) or timer tears down; teardown wrapped so a malformed payload can't wedge
  the layer. Reports show/clear over device:log (tag 'pip').

Dashboard: a minimal "Send overlay" / "Clear overlay" tester on the device-detail
controls (device/group via the open device, type, uri, position, duration), calling
POST /api/pip through the api helper.

Tests (server suite green, 161/161):
- api.test.js: PiP tier — authz (read/write 403, full passes), workspace isolation
  (wsA token -> wsB device 404), payload validation, device + group targeting, clear;
  plus the PUBLIC_ROUTERS snapshot-firewall updated for /api/pip.
- pip-overlay.test.js: loads the real player.js + pip-overlay.js in a vm with a DOM
  shim; proves the overlay shows, auto-dismisses on the duration timer, and never
  changes the playlist signature / touches #stage; web->iframe, last-show-wins,
  id-aware clear, malformed-payload safety.

Not in this PR (intentional):
- Android player overlay — fast-follow. Protocol + server are player-agnostic; the
  Android layer (an overlay View above the player, orientation-matched to MainActivity's
  rootView rotation) is the same shape and lands next.
- OpenAPI docs for POST /api/pip — the contract test's scope heuristic only treats
  'command' paths as full-scope, so documenting a full-scope non-command route there
  needs that heuristic extended first; deferred with the docs item (proposal §8.6).
- video/rtsp types, MQTT, offline queue-on-reconnect, priority/stacking, arbitrary
  (x,y)/selector positioning (proposal §6).

Refs #109

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PiP overlay: add Android + web players (#109)

Extends the #109 PiP MVP to the other two players so the protocol (device:pip-show /
device:pip-clear) is honored fleet-wide, not just on Tizen. No server/protocol changes —
the route and socket messages are player-agnostic; these are the two missing surfaces.

Web player (server/player/index.html):
- New #pipContainer layer above #playerContainer, pointer-transparent, that the playlist
  render never touches. The same orientation transform is applied to it as to
  #playerContainer (extended to also reset width/height on landscape so a
  portrait->landscape switch realigns), so corner positions track the visible content.
- Inline PiP logic mirroring tizen/js/pip-overlay.js: image -> <img>, web -> <iframe>
  (muted by default via empty allow=), position/size/bg/opacity/radius/title, single slot
  last-show-wins, duration timer (0 = until cleared), id-aware clear, wrapped teardown.
- device:pip-show/clear handlers; reports show/clear over device:log (tag "pip").

Android player:
- activity_main.xml: a pipLayout FrameLayout as the LAST child of rootLayout — it draws
  above the content AND inherits rootView's orientation rotation/translation, so corner
  positioning is orientation-matched for free.
- PipOverlay.kt (new): builds the overlay box into pipLayout. image -> ImageView (decoded
  off-thread via ImageLoader, dropped if torn down mid-decode); web -> WebView with
  mediaPlaybackRequiresUserGesture=true (mute-by-default). Gravity-based corner/center
  placement with a 4% inset, GradientDrawable bg + corner radius, alpha=opacity, optional
  title bar. Single slot last-show-wins; duration timer; id-aware clear; teardown wrapped
  and also run on activity destroy (WebView cleanup).
- WebSocketService: onPipShow/onPipClear callbacks + safeOn handlers posted to the main
  thread (they build Views) + a sendLog(tag, level, message) emitter for device:log.
- MainActivity: instantiate PipOverlay (log -> wsService.sendLog("pip", ...)), wire the
  callbacks, tear down on destroy.

Verified: Android assembleDebug builds clean; web player inline JS parses; server suite
still 161/161 (no server changes this commit). Not yet validated on real hardware —
four-orientation corner positioning mirrors the player container/rootView transform but
should be eyeballed on a panel.

Refs #109

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-18 14:54:44 -05:00