mirror of
https://github.com/screentinker/screentinker.git
synced 2026-08-13 13:53:12 -06:00
Offline media on every player, and a revision so the cache can still be updated
Two halves of the same problem. A screen has to keep playing when the link is gone, and it must not keep playing the wrong thing once the link is back. CACHING FOR OFFLINE, on the players that could not: - Tizen cached nothing but the playlist, so a panel came back from a reboot knowing exactly what to show and fetched every frame of it from a server that was not there. tizen/js/media-cache.js caches the media itself to wgt-private (the store Tizen documents as surviving reboots), resumable via Range and If-Range, with the transfer async so a stalled chunk cannot freeze the player. offline.cache moves from "absent" to a runtime claim: a build with no writable private storage still says nothing. - The web player's worker stored only what a single fetch() happened to complete, which on a marginal link is nothing at all — a 200MB asset never finishes in one go and every retry starts from zero. It now accumulates in resumable chunks, driven by the player's playlist rather than by playback, so the prefetch is not competing with the video that is currently on screen for the same scarce bandwidth. BrightSign inherits this. STILL UPDATING, which caching quietly breaks: PUT /api/content/:id/replace changes an asset's bytes under a stable id. Every cache keys on that id, so before this the new bytes could not reach a panel that already held the old ones — not until the next refresh, but never. Content now carries a revision, stamped onto each item at send time like widget revs, and every player keys its cache on it. The same send-time refresh fixes a second bug: a replace writes a new randomly-named file and unlinks the old one, so the filepath in a published snapshot pointed at a deleted file and web panels 404'd on the item until somebody republished the playlist. The route now also pushes to affected devices, which it never did. Bytes are kept only where they can be built upon: no validator means no safe resume, so the partial is discarded and the attempt backs off as the failure it is rather than re-fetching the same prefix forever. Server needed no new transfer support — res.sendFile already does Range, If-Range and 416. The Tizen cache and the service worker are both driven in Node against fakes, because neither can be exercised without hardware and "the chunks assemble correctly" is not something to discover from a panel showing a corrupt video. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uaeo9MvzKoyXuN6ZsbhtkL
This commit is contained in:
ScreenTinker
parent
cf3b2e62af
commit
684e60fc55
|
|
@ -237,7 +237,7 @@ class MainActivity : AppCompatActivity() {
|
|||
// a widget, a remote stream, or a fully-downloaded local file. A not-yet/failed download is
|
||||
// skipped (kept in the background) instead of blanking the screen on a loading state.
|
||||
playlistController.setContentReadyCheck { item ->
|
||||
item.isWidget || item.isRemote || contentCache.isContentCached(item.contentId)
|
||||
item.isWidget || item.isRemote || contentCache.isContentCached(item.contentId, item.contentRev)
|
||||
}
|
||||
|
||||
// feat/transition-engine: full-screen GLES2 overlay that plays image/video wipes. Inserted just
|
||||
|
|
@ -668,6 +668,10 @@ class MainActivity : AppCompatActivity() {
|
|||
val contentId = if (item.isNull("content_id")) "" else item.optString("content_id", "")
|
||||
if (contentId.isEmpty()) continue
|
||||
val filename = item.optString("filename", "content")
|
||||
// Bumped when the bytes behind this id change. A cached copy at a different revision is
|
||||
// a MISS: the id, the filename and the URL are all identical after a replace, so this
|
||||
// is the only thing that can tell a panel its copy is out of date.
|
||||
val contentRev = item.optLong("content_rev", 0L)
|
||||
// org.json's optString(key, null) returns the STRING "null" when the value is JSON
|
||||
// null (not the fallback) — so a local item with "remote_url": null was being
|
||||
// misclassified as a remote stream, ack'd "ready", and NEVER downloaded, stranding
|
||||
|
|
@ -687,7 +691,7 @@ class MainActivity : AppCompatActivity() {
|
|||
// defers when the socket is down (watchdog owns recovery), respects backoff, and
|
||||
// downloads at most once. It acks ready/failed itself (deduped via onAck).
|
||||
if (contentChanged) downloadCoordinator.resetBackoff(contentId) // #170: retry now, don't wait out a stale backoff
|
||||
downloadCoordinator.ensure(contentId, filename)
|
||||
downloadCoordinator.ensure(contentId, filename, contentRev)
|
||||
}
|
||||
|
||||
// Start/resume playback immediately — do NOT wait on downloads (they're async now).
|
||||
|
|
@ -962,7 +966,7 @@ class MainActivity : AppCompatActivity() {
|
|||
val file = contentCache.getCachedFile(item.contentId)
|
||||
if (file == null) {
|
||||
Log.i("MainActivity", "Content not ready at play time (${item.filename}) — keeping screen, advancing (bg download continues)")
|
||||
downloadCoordinator.ensure(item.contentId, item.filename) // ensure it's being fetched (single-flight)
|
||||
downloadCoordinator.ensure(item.contentId, item.filename, item.contentRev) // ensure it's being fetched (single-flight)
|
||||
handler.post { playlistController.next() }
|
||||
return
|
||||
}
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ class ContentCache internal constructor(
|
|||
// cross-matching. `contains` rather than `endsWith` for the temp check because the resume
|
||||
// validator sidecar is "<id>.<ext>.part.tag" — it does not END with ".part", and returning
|
||||
// THAT as the cached asset would hand the player a short ETag file to play.
|
||||
val files = cacheDir.listFiles { _, name -> name.startsWith("$contentId.") && !name.contains(PART_SUFFIX) }
|
||||
val files = cacheDir.listFiles { _, name -> name.startsWith("$contentId.") && !name.contains(PART_SUFFIX) && !name.endsWith(REV_SUFFIX) }
|
||||
val hit = files?.firstOrNull()?.takeIf { it.exists() && it.length() > 0 }
|
||||
com.remotedisplay.player.util.DebugLog.v("ContentCache", "getCachedFile($contentId): dir=${cacheDir.absolutePath} listFiles=${files?.size ?: -1} hit=${hit?.name}")
|
||||
return hit
|
||||
|
|
@ -76,11 +76,35 @@ class ContentCache internal constructor(
|
|||
return getCachedFile(contentId) != null
|
||||
}
|
||||
|
||||
/**
|
||||
* Cached AND holding the revision the playlist is asking for.
|
||||
*
|
||||
* The dashboard can replace an asset's bytes under a stable content id, which is the one way a
|
||||
* cached copy can be permanently wrong: the id does not change, the filename does not change,
|
||||
* and nothing about a plain existence check can tell. A panel would keep playing last month's
|
||||
* video until somebody deleted and re-added the item. Comparing the revision is what makes
|
||||
* "cached for offline" compatible with "and it still updates".
|
||||
*
|
||||
* A revision of 0 means the server never sent one (an older build): fall back to existence, so
|
||||
* an upgrade does not re-download the entire playlist over the link least able to afford it.
|
||||
*/
|
||||
fun isContentCached(contentId: String, rev: Long): Boolean {
|
||||
val file = getCachedFile(contentId) ?: return false
|
||||
if (rev <= 0L) return true
|
||||
return readRev(file) == rev
|
||||
}
|
||||
|
||||
private fun revFile(file: File) = File(file.absolutePath + REV_SUFFIX)
|
||||
|
||||
private fun readRev(file: File): Long =
|
||||
try { revFile(file).takeIf { it.exists() }?.readText()?.trim()?.toLongOrNull() ?: 0L }
|
||||
catch (e: Exception) { 0L }
|
||||
|
||||
/**
|
||||
* Fetch (or continue fetching) [contentId]. Safe to call repeatedly: each call transfers what
|
||||
* the link allows and leaves the rest for the next one.
|
||||
*/
|
||||
fun fetch(serverUrl: String, contentId: String, filename: String): Result {
|
||||
fun fetch(serverUrl: String, contentId: String, filename: String, rev: Long = 0L): Result {
|
||||
val ext = filename.substringAfterLast('.', "mp4")
|
||||
val finalFile = File(cacheDir, "$contentId.$ext")
|
||||
val partFile = File(cacheDir, "$contentId.$ext$PART_SUFFIX")
|
||||
|
|
@ -94,7 +118,11 @@ class ContentCache internal constructor(
|
|||
if (resumeFrom == 0L) { partFile.delete(); tagFile.delete() }
|
||||
|
||||
try {
|
||||
val builder = Request.Builder().url("$serverUrl/api/content/$contentId/file")
|
||||
// The revision rides in the URL as well as in the sidecar: an intermediary caching
|
||||
// /api/content/<id>/file would otherwise happily serve the superseded bytes to every
|
||||
// panel behind it, and no amount of client-side bookkeeping could tell.
|
||||
val url = "$serverUrl/api/content/$contentId/file" + (if (rev > 0L) "?rev=$rev" else "")
|
||||
val builder = Request.Builder().url(url)
|
||||
if (resumeFrom > 0) {
|
||||
builder.header("Range", "bytes=$resumeFrom-")
|
||||
builder.header("If-Range", validator!!)
|
||||
|
|
@ -186,6 +214,12 @@ class ContentCache internal constructor(
|
|||
return Result.Failed
|
||||
}
|
||||
tagFile.delete()
|
||||
// Record WHICH revision these bytes are, so a later replace is detectable. Written
|
||||
// after the rename: a revision marker next to a file that is not there yet would
|
||||
// claim a cached asset that does not exist.
|
||||
try {
|
||||
if (rev > 0L) revFile(finalFile).writeText(rev.toString()) else revFile(finalFile).delete()
|
||||
} catch (e: Exception) { /* the file is still usable; worst case it re-downloads */ }
|
||||
Log.i("ContentCache", "Downloaded: $filename -> ${finalFile.absolutePath} ($onDisk bytes)")
|
||||
return Result.Done(finalFile)
|
||||
}
|
||||
|
|
@ -202,8 +236,8 @@ class ContentCache internal constructor(
|
|||
}
|
||||
|
||||
/** Complete-or-nothing wrapper: returns the cached file only when the asset is whole. */
|
||||
fun downloadContent(serverUrl: String, contentId: String, filename: String): File? =
|
||||
(fetch(serverUrl, contentId, filename) as? Result.Done)?.file
|
||||
fun downloadContent(serverUrl: String, contentId: String, filename: String, rev: Long = 0L): File? =
|
||||
(fetch(serverUrl, contentId, filename, rev) as? Result.Done)?.file
|
||||
|
||||
fun deleteContent(contentId: String) {
|
||||
// Exact-prefix (with the dot) so we don't delete a different id's file — and this also
|
||||
|
|
@ -248,6 +282,7 @@ class ContentCache internal constructor(
|
|||
companion object {
|
||||
private const val PART_SUFFIX = ".part"
|
||||
private const val TAG_SUFFIX = ".tag"
|
||||
private const val REV_SUFFIX = ".rev"
|
||||
|
||||
/**
|
||||
* "bytes <start>-<end>/<total>" -> (start, total). Null for anything else, including the
|
||||
|
|
|
|||
|
|
@ -49,16 +49,16 @@ class DownloadCoordinator(
|
|||
* sweep — the 60s refresh and every post-reconnect re-register included. Idempotent and
|
||||
* non-blocking: it enqueues at most ONE download per contentId and returns immediately.
|
||||
*/
|
||||
fun ensure(contentId: String, filename: String) {
|
||||
fun ensure(contentId: String, filename: String, rev: Long = 0L) {
|
||||
if (contentId.isEmpty()) return
|
||||
if (cache.isContentCached(contentId)) { DebugLog.v("DownloadCoordinator", "ensure($contentId): SEED-A cached -> ack ready"); onAck(contentId, "ready"); return } // already have it — re-ack (SEED-A)
|
||||
if (cache.isContentCached(contentId, rev)) { DebugLog.v("DownloadCoordinator", "ensure($contentId): SEED-A cached -> ack ready"); onAck(contentId, "ready"); return } // already have it — re-ack (SEED-A)
|
||||
// Socket down => the WATCHDOG owns recovery; don't hammer downloads over a dead connection.
|
||||
if (!socketAlive()) { DebugLog.v("DownloadCoordinator", "ensure($contentId): socket not alive -> skip"); return }
|
||||
if (now() < (nextAttemptAt[contentId] ?: 0L)) { DebugLog.v("DownloadCoordinator", "ensure($contentId): in backoff until ${nextAttemptAt[contentId]} -> skip"); return } // in failure backoff — don't storm
|
||||
if (!inFlight.add(contentId)) { DebugLog.v("DownloadCoordinator", "ensure($contentId): already inFlight -> skip"); return } // single-flight: already downloading
|
||||
DebugLog.v("DownloadCoordinator", "ensure($contentId): dispatching download '$filename'")
|
||||
try {
|
||||
executor.execute { runDownload(contentId, filename) }
|
||||
executor.execute { runDownload(contentId, filename, rev) }
|
||||
} catch (e: Throwable) {
|
||||
inFlight.remove(contentId) // executor rejected (shut down) — don't leak the guard
|
||||
}
|
||||
|
|
@ -76,12 +76,12 @@ class DownloadCoordinator(
|
|||
* inFlight is held across the whole chain, so this is still exactly one writer per `.part` and
|
||||
* a concurrent sweep still finds the item busy rather than starting a duplicate.
|
||||
*/
|
||||
private fun runDownload(contentId: String, filename: String) {
|
||||
private fun runDownload(contentId: String, filename: String, rev: Long) {
|
||||
try {
|
||||
var link = 0
|
||||
while (link < MAX_RESUME_CHAIN) {
|
||||
link++
|
||||
val result = cache.fetch(serverUrl(), contentId, filename)
|
||||
val result = cache.fetch(serverUrl(), contentId, filename, rev)
|
||||
when (result) {
|
||||
is ContentCache.Result.Done -> {
|
||||
attempts.remove(contentId); nextAttemptAt.remove(contentId)
|
||||
|
|
|
|||
|
|
@ -22,6 +22,10 @@ data class PlaylistItem(
|
|||
// Changes whenever the widget is edited. Carried into the render URL so an edited widget gets
|
||||
// a URL the player has not seen, which is what defeats the deliberate same-URL WebView reuse.
|
||||
val widgetRev: Long = 0L,
|
||||
// Bumped by the server when an asset's BYTES change under a stable content id (the dashboard's
|
||||
// "replace file"). The cache is keyed on it: without one, a replaced asset would keep playing
|
||||
// the copy already on disk forever, because nothing about the id or the URL would differ.
|
||||
val contentRev: Long = 0L,
|
||||
val widgetType: String? = null,
|
||||
val schedules: List<ScheduleEval.Block> = emptyList(),
|
||||
// feat/transition-engine: the resolved GL transition this item plays INTO (null = hard cut).
|
||||
|
|
@ -173,6 +177,7 @@ class PlaylistController(
|
|||
muted = obj.optInt("muted", 0) == 1,
|
||||
widgetId = if (obj.isNull("widget_id")) null else obj.optString("widget_id", "").ifEmpty { null },
|
||||
widgetRev = obj.optLong("widget_rev", 0L),
|
||||
contentRev = obj.optLong("content_rev", 0L),
|
||||
widgetType = if (obj.isNull("widget_type")) null else obj.optString("widget_type", "").ifEmpty { null },
|
||||
schedules = parseSchedules(obj.optJSONArray("schedules")),
|
||||
transition = Transitions.parse(obj.optJSONObject("transition"))
|
||||
|
|
|
|||
|
|
@ -291,6 +291,45 @@ class ContentDownloadTest {
|
|||
assertNull("and nothing incomplete is ever served as cached", cache.getCachedFile("noval"))
|
||||
}
|
||||
|
||||
|
||||
// ---- the UPDATE half: caching for offline must not make a screen permanently wrong ----
|
||||
@Test fun `an asset replaced under a stable id is a cache MISS at the new revision`() {
|
||||
// The trap that offline caching creates. PUT /api/content/:id/replace changes the bytes and
|
||||
// nothing else — same id, same filename, same URL path — so a plain "do I have this file?"
|
||||
// check says yes forever and the panel keeps playing last month's video.
|
||||
val v1 = ByteArray(50) { 'a'.code.toByte() }
|
||||
val url = serveFlaky({ v1 }, bytesPerCall = 500, etagOf = { "\"v1\"" })
|
||||
|
||||
assertTrue(cache.fetch(url, "swap", "clip.bin", rev = 100L) is ContentCache.Result.Done)
|
||||
assertTrue("cached at the revision we asked for", cache.isContentCached("swap", 100L))
|
||||
assertTrue("...and NOT at a newer one", !cache.isContentCached("swap", 200L))
|
||||
}
|
||||
|
||||
@Test fun `a player with no revision from the server still uses whatever it has`() {
|
||||
// Older servers send no content_rev. Treating that as a permanent miss would re-download the
|
||||
// entire playlist on every sweep, over the link least able to afford it.
|
||||
val url = serveOnce { it.writeHttp(4, "abcd".toByteArray()) }
|
||||
assertNotNull(cache.downloadContent(url, "norev", "clip.bin"))
|
||||
assertTrue(cache.isContentCached("norev", 0L))
|
||||
}
|
||||
|
||||
@Test fun `the revision marker is never served as the cached asset`() {
|
||||
// "<id>.<ext>.rev" starts with the id, so a prefix match would hand the player a few bytes
|
||||
// of ASCII digits to decode as a video.
|
||||
java.io.File(dir, "marker.bin.rev").writeText("12345")
|
||||
assertNull(cache.getCachedFile("marker"))
|
||||
}
|
||||
|
||||
@Test fun `the request carries the revision, so an intermediary cannot serve the old bytes`() {
|
||||
seen.clear()
|
||||
val url = serveFlaky({ ByteArray(20) }, bytesPerCall = 500, etagOf = { "\"v1\"" })
|
||||
cache.fetch(url, "cdn", "clip.bin", rev = 777L)
|
||||
// The request line is not captured by the header sniffer, so assert via the effect: a
|
||||
// revisioned fetch completes and records that revision.
|
||||
assertTrue(cache.isContentCached("cdn", 777L))
|
||||
assertTrue(!cache.isContentCached("cdn", 778L))
|
||||
}
|
||||
|
||||
// ---- prefix cross-match guard: an id that prefixes another must not match ----
|
||||
@Test fun `getCachedFile does not cross-match an id that is a prefix of another`() {
|
||||
serveOnce { it.writeHttp(3, "abc".toByteArray()) }.let { url ->
|
||||
|
|
|
|||
|
|
@ -75,7 +75,7 @@ privilege model exists on those platforms — so the column is collapsed.
|
|||
|---|---|---|---|---|
|
||||
| `sync.clock` | ✅ | ✅ | ✅ | ✅ |
|
||||
| `sync.native` | ❌ no native protocol | ❌ | ❌ | ⚠️ SyncManager, BOS 8.2.10+; multicast so all members must share one L2 network |
|
||||
| `offline.cache` | ✅ content downloaded to disk | ✅ service worker | ⚠️ **playlist payload only** — `st_payload_cache` replays the last renderable payload, but there is no service worker, so media still needs the network | ✅ service worker + 1GB storage quota |
|
||||
| `offline.cache` | ✅ content downloaded to disk, **resumable** (Range + If-Range), revision-keyed | ✅ service worker, **resumable chunked prefetch**, revision-keyed | ✅ **media cached to `wgt-private`** (`js/media-cache.js`), resumable, revision-keyed — declared at runtime, since a build with no writable private storage must not claim it | ✅ inherits the web player's service worker |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -86,9 +86,11 @@ Ordered by how visible the failure is to an operator.
|
|||
1. **Tizen `audio.volume` — dead control.** `set_volume` has no handler in `tizen/js/app.js`; the
|
||||
only volume path is the on-device `KEYCODE_VOLUME_*` keys. The dashboard slider silently does
|
||||
nothing. Either implement the handler or let the capability hide the control.
|
||||
2. **Tizen `offline.cache` is partial.** The playlist survives a reboot; the media does not. A
|
||||
Tizen panel that loses its uplink keeps its schedule and cannot play it. This is the largest
|
||||
functional gap in the table.
|
||||
2. ~~**Tizen `offline.cache` is partial.**~~ **Closed.** `tizen/js/media-cache.js` caches the
|
||||
media itself to `wgt-private` — resumable, so a panel on a bad link accumulates an asset
|
||||
across attempts instead of restarting from zero, and revision-keyed, so a replaced asset is
|
||||
still a miss. The capability is declared at runtime rather than assumed: a build that cannot
|
||||
write to private storage keeps quiet about it.
|
||||
3. **BrightSign `remote.screenshot` needs primary storage.** Reachable today only via the canvas
|
||||
fallback, which cannot read the video plane, so screenshots show everything except the video.
|
||||
Resolves itself when a card or SSD is fitted.
|
||||
|
|
@ -114,5 +116,6 @@ these are wrong for the existing fleet until each player ships its declaration:
|
|||
- **`tizen` claims `audio.volume`** — no handler exists (gap 1 above). Should be removed.
|
||||
- **`tizen` omits `remote.screenshot` and `remote.stream`** — both are implemented
|
||||
(`captureAndSend`, `startStreaming`). Should be added.
|
||||
- **`tizen` claims `offline.cache`** — true only for the playlist payload, not media. Either keep
|
||||
it with the partial meaning documented, or split the capability.
|
||||
- **`tizen` declares `offline.cache` itself now** — the server baseline still omits it, which is
|
||||
correct: a fielded panel that has not been updated genuinely cannot hold media, and the
|
||||
baseline describes what an un-updated one can do.
|
||||
|
|
|
|||
|
|
@ -92,6 +92,11 @@ const migrations = [
|
|||
// pre-v4 clients that send no identity block). No logic is built on these yet.
|
||||
'ALTER TABLE devices ADD COLUMN client_type TEXT',
|
||||
'ALTER TABLE devices ADD COLUMN client_version TEXT',
|
||||
// Content revision. SQLite cannot ADD COLUMN with a non-constant default, so this lands as 0 and
|
||||
// is backfilled from created_at below — a row that has never been replaced is at its birth
|
||||
// revision, which is exactly right.
|
||||
'ALTER TABLE content ADD COLUMN updated_at INTEGER NOT NULL DEFAULT 0',
|
||||
'UPDATE content SET updated_at = created_at WHERE updated_at = 0',
|
||||
'ALTER TABLE devices ADD COLUMN platform TEXT',
|
||||
'ALTER TABLE devices ADD COLUMN contract_version TEXT',
|
||||
// Exit-signal contract v1 — manner-of-death annotation on Offline (additive; NEVER alters offline
|
||||
|
|
|
|||
|
|
@ -107,7 +107,12 @@ CREATE TABLE IF NOT EXISTS content (
|
|||
width INTEGER,
|
||||
height INTEGER,
|
||||
remote_url TEXT,
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now'))
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
-- Bumped whenever the BYTES change (PUT /:id/replace). Players cache media by id, and an id
|
||||
-- whose bytes changed underneath them is the one way a cached asset can be stale forever: the
|
||||
-- URL is identical, so every offline cache we have would keep serving the old file. This is
|
||||
-- what makes the URL differ exactly when the content differs.
|
||||
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS assignments (
|
||||
|
|
|
|||
|
|
@ -138,11 +138,128 @@
|
|||
return (usedBytes + incomingBytes) > (quotaBytes * headroom);
|
||||
}
|
||||
|
||||
/*
|
||||
* ---------------------------------------------------------------------------------------------
|
||||
* RESUMABLE TRANSFER
|
||||
*
|
||||
* Storing a complete 200 is only safe once you can GET a complete 200. A single fetch() of a
|
||||
* 200MB asset over a one-bar link does not finish, and every retry starts again from nothing —
|
||||
* which is how a site ends up with an empty cache and a screen showing the waiting state. The
|
||||
* Android player hit exactly this and was fixed by resuming; these helpers are the same idea for
|
||||
* the browser-based players, where there is no file handle to append to and progress has to be
|
||||
* accumulated as separate cache entries instead.
|
||||
* ---------------------------------------------------------------------------------------------
|
||||
*/
|
||||
|
||||
// 4MB. Small enough that a marginal link finishes one inside a stall timeout, large enough that a
|
||||
// 200MB asset is 50 requests rather than 1600 — per-request overhead on a slow uplink is not free.
|
||||
var CHUNK_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
/*
|
||||
* The byte ranges an asset of [total] bytes decomposes into. Inclusive ends, because that is what
|
||||
* both the Range header and Content-Range use, and converting between the two conventions is
|
||||
* where off-by-one corruption comes from.
|
||||
*/
|
||||
function chunkRanges(total, chunkSize) {
|
||||
var size = chunkSize > 0 ? chunkSize : CHUNK_BYTES;
|
||||
var out = [];
|
||||
if (!(total > 0)) return out;
|
||||
for (var start = 0; start < total; start += size) {
|
||||
out.push({ start: start, end: Math.min(start + size, total) - 1 });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/*
|
||||
* "bytes <start>-<end>/<total>" -> {start, end, total}. Null for anything we cannot verify a
|
||||
* total from, including the "*" form: without the total there is nothing to decide completeness
|
||||
* against, and guessing produces a cache entry that is confidently short.
|
||||
*/
|
||||
function parseContentRange(header) {
|
||||
if (!header || typeof header !== 'string') return null;
|
||||
var m = /^\s*bytes\s+(\d+)-(\d+)\/(\d+)\s*$/.exec(header);
|
||||
if (!m) return null;
|
||||
return { start: Number(m[1]), end: Number(m[2]), total: Number(m[3]) };
|
||||
}
|
||||
|
||||
/*
|
||||
* The validator to send back as If-Range on the next chunk.
|
||||
*
|
||||
* Without one, a resume splices the tail of a NEW asset onto the head of an old one and produces
|
||||
* a file of exactly the right length that is nonetheless corrupt — it would pass every
|
||||
* completeness check and be played. ETag first (it changes on any byte change); Last-Modified is
|
||||
* the weaker fallback, and no validator at all means the transfer is not resumable.
|
||||
*/
|
||||
function validatorOf(headers) {
|
||||
if (!headers || typeof headers.get !== 'function') return null;
|
||||
return headers.get('ETag') || headers.get('Last-Modified') || null;
|
||||
}
|
||||
|
||||
/*
|
||||
* Is a stored partial still usable, given what the server just said?
|
||||
*
|
||||
* A 206 at the offset we asked for, with a matching validator and the same total, continues. A
|
||||
* 200 means the server declined the range (changed asset, or no range support) and the transfer
|
||||
* restarts. Anything else is discarded rather than reasoned about.
|
||||
*/
|
||||
function resumeVerdict(status, contentRange, expectStart, expectTotal, validatorSent, validatorNow) {
|
||||
if (status === 200) return 'restart';
|
||||
if (status === 416) return 'discard';
|
||||
if (status !== 206) return 'discard';
|
||||
var cr = parseContentRange(contentRange);
|
||||
if (!cr) return 'discard';
|
||||
if (cr.start !== expectStart) return 'discard';
|
||||
if (expectTotal > 0 && cr.total !== expectTotal) return 'discard';
|
||||
// A server that honours If-Range should not answer 206 with a different validator, but the
|
||||
// check costs nothing and the failure it prevents is a silent splice.
|
||||
if (validatorSent && validatorNow && validatorSent !== validatorNow) return 'discard';
|
||||
return 'continue';
|
||||
}
|
||||
|
||||
/*
|
||||
* The asset a URL refers to, ignoring the revision.
|
||||
*
|
||||
* Cache entries are keyed with the revision in the query string so that replacing an asset is a
|
||||
* MISS everywhere rather than a copy nobody can invalidate. The flip side is that the superseded
|
||||
* entries would sit there until the quota evicted them — on a 1GB panel quota, a few replaced
|
||||
* videos is the whole budget. This is what lets a store sweep its own predecessors.
|
||||
*/
|
||||
function assetKey(url) {
|
||||
try {
|
||||
var u = typeof url === 'string' ? new URL(url, 'http://x') : url;
|
||||
return u.origin + u.pathname;
|
||||
} catch (e) {
|
||||
return String(url).split('?')[0];
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Is [url] a chunk/bookkeeping entry rather than a playable asset? Those must never be returned
|
||||
* to a media element, and must never be mistaken for "the asset is cached".
|
||||
*/
|
||||
function isInternalKey(url) {
|
||||
return String(url).indexOf(INTERNAL_MARK) !== -1;
|
||||
}
|
||||
|
||||
var INTERNAL_MARK = '__st_part';
|
||||
|
||||
function chunkKey(url, start) {
|
||||
return url + (url.indexOf('?') === -1 ? '?' : '&') + INTERNAL_MARK + '=' + start;
|
||||
}
|
||||
|
||||
return {
|
||||
isCacheableContent: isCacheableContent,
|
||||
parseRange: parseRange,
|
||||
partialHeaders: partialHeaders,
|
||||
isStorable: isStorable,
|
||||
needsEviction: needsEviction
|
||||
needsEviction: needsEviction,
|
||||
CHUNK_BYTES: CHUNK_BYTES,
|
||||
chunkRanges: chunkRanges,
|
||||
parseContentRange: parseContentRange,
|
||||
validatorOf: validatorOf,
|
||||
resumeVerdict: resumeVerdict,
|
||||
assetKey: assetKey,
|
||||
isInternalKey: isInternalKey,
|
||||
chunkKey: chunkKey
|
||||
};
|
||||
});
|
||||
|
|
|
|||
|
|
@ -403,6 +403,45 @@
|
|||
// Cache the layout alongside the playlist so a cold start renders the correct
|
||||
// zone layout on the FIRST pass, instead of rendering fullscreen and only
|
||||
// switching to zones once the server payload arrives.
|
||||
/*
|
||||
* Where an item's bytes live.
|
||||
*
|
||||
* The revision is in the URL on purpose. Media is cached for offline playback keyed by URL, and
|
||||
* PUT /api/content/:id/replace changes an asset's BYTES without changing its id — so without a
|
||||
* revision a replaced asset would keep playing the old copy on every panel that already held
|
||||
* it, with no way for the player to find out. The revision changes exactly when the bytes do,
|
||||
* which makes a replace a cache miss and an untouched asset a hit.
|
||||
*/
|
||||
function mediaUrl(it) {
|
||||
if (!it) return '';
|
||||
if (it.remote_url) return it.remote_url;
|
||||
const base = `${config.serverUrl}/uploads/content/${it.filepath}`;
|
||||
return it.content_rev ? `${base}?rev=${encodeURIComponent(it.content_rev)}` : base;
|
||||
}
|
||||
|
||||
/*
|
||||
* Ask the service worker to hold this playlist's media for offline use.
|
||||
*
|
||||
* The worker stores whatever a playback fetch happens to complete, which on a healthy link is
|
||||
* everything and on a marginal one is nothing: a large asset never finishes in one go, so the
|
||||
* panel is left with an empty cache exactly where it needs a full one. Handing it the list lets
|
||||
* it fetch in resumable chunks, on its own schedule, instead of racing the video that is
|
||||
* currently playing for the same scarce bandwidth.
|
||||
*
|
||||
* Best-effort by construction: no service worker (an insecure origin, an older BrightSign
|
||||
* build) simply means no prefetch, and playback is unchanged.
|
||||
*/
|
||||
function requestOfflineCache(items) {
|
||||
try {
|
||||
const sw = navigator.serviceWorker && navigator.serviceWorker.controller;
|
||||
if (!sw) return;
|
||||
const urls = (items || [])
|
||||
.filter((it) => it && it.filepath && !it.remote_url)
|
||||
.map((it) => mediaUrl(it));
|
||||
if (urls.length) sw.postMessage({ type: 'st-cache-playlist', urls });
|
||||
} catch (e) { /* never let caching break playback */ }
|
||||
}
|
||||
|
||||
const LAYOUT_CACHE_KEY = 'rd_layout_cache' + SCREEN_SUFFIX;
|
||||
function saveLayoutCache(l) {
|
||||
try { localStorage.setItem(LAYOUT_CACHE_KEY, JSON.stringify(l || null)); } catch {}
|
||||
|
|
@ -2063,7 +2102,7 @@
|
|||
if (!item) return;
|
||||
const isVid = item.mime_type && item.mime_type.indexOf('video/') === 0 && item.mime_type !== 'video/youtube';
|
||||
if (!isVid) { groupPreloadIdx = idx; groupPreloadEl = null; return; } // mark handled, nothing to warm
|
||||
const url = item.remote_url || (config.serverUrl + '/uploads/content/' + item.filepath);
|
||||
const url = mediaUrl(item);
|
||||
try {
|
||||
if (groupPreloadEl) { try { groupPreloadEl.remove(); } catch (e) {} }
|
||||
const v = document.createElement('video');
|
||||
|
|
@ -2365,6 +2404,7 @@
|
|||
playlist = newItems;
|
||||
imgPreloadCache = {}; // playlist changed — drop stale one-ahead preloads (feat/player-image-preload)
|
||||
savePlaylistCache(playlist);
|
||||
requestOfflineCache(playlist);
|
||||
// #157: a fresh structural update supersedes any pending deferred rotation; the branches
|
||||
// below re-arm it only if the current item was removed while live in solo playback.
|
||||
deferredRotation = false;
|
||||
|
|
@ -2875,7 +2915,7 @@
|
|||
// feat/player-image-preload: warm the NEXT scheduled image (decoded) during the current dwell,
|
||||
// and swap decode-gated so a slow panel never shows a blank/half-painted frame mid-decode.
|
||||
function imgSrcFor(it) {
|
||||
return it.remote_url || `${config.serverUrl}/uploads/content/${it.filepath}`;
|
||||
return mediaUrl(it);
|
||||
}
|
||||
// A remote image gets a proxy fallback (/media/proxy/:contentId) so it can be textured for a
|
||||
// transition when its origin sends no CORS header. Local /uploads content is already CORS-enabled,
|
||||
|
|
@ -3101,7 +3141,7 @@
|
|||
// + the wipe), and a playlist push mid-window must not let a stale clip tear down the newer content that
|
||||
// already took over.
|
||||
function renderVideoBuffered(item) {
|
||||
const src = item.remote_url || `${config.serverUrl}/uploads/content/${item.filepath}`;
|
||||
const src = mediaUrl(item);
|
||||
const from = currentTexturableFrame(); // capture the outgoing frame NOW, before any teardown
|
||||
const t = item.transition;
|
||||
const mySeq = renderSeq;
|
||||
|
|
@ -3301,7 +3341,7 @@
|
|||
const isImage = item.mime_type?.startsWith('image/');
|
||||
const remoteUrl = item.remote_url;
|
||||
const serverUrl = config.serverUrl;
|
||||
const src = remoteUrl || `${serverUrl}/uploads/content/${item.filepath}`;
|
||||
const src = mediaUrl(item);
|
||||
|
||||
if (layout && layout.zones && layout.zones.length > 1 && !wallConfig) {
|
||||
renderZones(container, item);
|
||||
|
|
@ -3489,7 +3529,7 @@
|
|||
|
||||
const isYoutube = a.mime_type === 'video/youtube';
|
||||
const isVideo = !isYoutube && a.mime_type?.startsWith('video/');
|
||||
const src = a.remote_url || `${config.serverUrl}/uploads/content/${a.filepath}`;
|
||||
const src = mediaUrl(a);
|
||||
const dur = (a.duration_sec || 10) * 1000;
|
||||
|
||||
// Render based on what the ASSIGNMENT is (widget_id), not the zone's type:
|
||||
|
|
|
|||
|
|
@ -1,9 +1,12 @@
|
|||
// v21: chunked resumable content prefetch + revision-keyed media URLs — index.html gained
|
||||
// mediaUrl()/requestOfflineCache() and this worker gained the message handler, so an old shell
|
||||
// cache would pair a new worker with a player that never posts it a playlist.
|
||||
// v20: rc3 changed the fetch strategy AND the shipped player assets. The activate handler deletes
|
||||
// every cache whose name does not match, so leaving this at v19 kept the previous shell cache alive
|
||||
// — a player then ran a new index.html against a stale st-bridge.js and threw on every heartbeat.
|
||||
// Bump whenever a shipped /player asset changes shape; content lives in its own cache, so this
|
||||
// costs a small re-download and never re-fetches the playlist.
|
||||
const CACHE_NAME = 'rd-player-v20';
|
||||
const CACHE_NAME = 'rd-player-v21';
|
||||
// Content lives in its own cache so the shell can be re-versioned (the activate handler deletes
|
||||
// every cache that is not CACHE_NAME) WITHOUT throwing away megabytes of media that are still
|
||||
// perfectly valid. Rolling the shell used to mean a player re-downloaded its entire playlist.
|
||||
|
|
@ -108,6 +111,159 @@ self.addEventListener('fetch', (event) => {
|
|||
// Returning without event.respondWith lets the browser handle it natively.
|
||||
});
|
||||
|
||||
/*
|
||||
* PREFETCH — the half that makes the cache fill on a link that cannot carry a whole asset.
|
||||
*
|
||||
* handleContent below stores an asset when a single fetch() of it happens to succeed. On a good
|
||||
* link that is everything. On a marginal one (the one-bar 5G site this came from) a 200MB fetch
|
||||
* never completes, every retry starts from nothing, and the cache stays empty — so the panel has
|
||||
* nothing to fall back on the moment the uplink drops. The Android player had the identical bug and
|
||||
* was fixed by resuming; a service worker has no file handle to append to, so progress accumulates
|
||||
* as separate cache entries and is assembled once every piece is present.
|
||||
*
|
||||
* Driven by the player rather than by playback: it posts its current media URLs after each playlist
|
||||
* update, and this works through them ONE AT A TIME. Deliberately not started from the fetch
|
||||
* handler — that would put the accumulator in competition with the playing video for the same
|
||||
* scarce bandwidth, which is worse than either alone.
|
||||
*/
|
||||
const prefetching = new Set();
|
||||
let prefetchChain = Promise.resolve();
|
||||
|
||||
self.addEventListener('message', (event) => {
|
||||
const data = event.data;
|
||||
if (!data || data.type !== 'st-cache-playlist' || !Array.isArray(data.urls)) return;
|
||||
for (const url of data.urls) {
|
||||
if (typeof url !== 'string' || !POLICY || !POLICY.isCacheableContent(url, 'GET')) continue;
|
||||
if (prefetching.has(url)) continue; // single-flight: a 60s playlist sweep must not restart it
|
||||
prefetching.add(url);
|
||||
// Serialised. Three concurrent chunk streams on a link that cannot finish one is how you get
|
||||
// three unfinished downloads instead of one finished one.
|
||||
prefetchChain = prefetchChain
|
||||
.then(() => ensureCached(url))
|
||||
.catch(() => {})
|
||||
.then(() => { prefetching.delete(url); });
|
||||
}
|
||||
});
|
||||
|
||||
/*
|
||||
* Fetch [url] into the content cache, one chunk at a time, resuming across calls.
|
||||
*
|
||||
* Returns when the asset is whole OR when a chunk fails — the caller does not retry, because the
|
||||
* player will ask again on its next playlist sweep and whatever landed is still on disk. That is
|
||||
* the entire point: attempts accumulate instead of restarting.
|
||||
*/
|
||||
async function ensureCached(url) {
|
||||
const cache = await caches.open(CONTENT_CACHE);
|
||||
if (await cache.match(url, { ignoreVary: true })) { await sweepOldRevisions(cache, url); return; }
|
||||
|
||||
const metaKey = POLICY.chunkKey(url, 'meta');
|
||||
let meta = null;
|
||||
const metaHit = await cache.match(metaKey);
|
||||
if (metaHit) { try { meta = await metaHit.json(); } catch (e) { meta = null; } }
|
||||
|
||||
// Learn the size and validator from the first ranged request, or trust what a previous call
|
||||
// already learned. A server that answers 200 here has no range support: fall back to storing it
|
||||
// whole, which is the pre-existing behaviour and is correct, just not resumable.
|
||||
if (!meta) {
|
||||
const probe = await fetch(new Request(url, { headers: { Range: 'bytes=0-' + (POLICY.CHUNK_BYTES - 1) } }));
|
||||
if (probe.status === 200) {
|
||||
if (POLICY.isStorable(probe)) await storeContent(cache, new Request(url), probe);
|
||||
return;
|
||||
}
|
||||
const cr = POLICY.parseContentRange(probe.headers.get('Content-Range'));
|
||||
if (probe.status !== 206 || !cr || !(cr.total > 0)) return;
|
||||
meta = { total: cr.total, validator: POLICY.validatorOf(probe.headers), type: probe.headers.get('Content-Type') || '' };
|
||||
if (!meta.validator) {
|
||||
// Nothing to detect a changed asset with, so a resume would be a guess. Store this one whole
|
||||
// if it happens to fit in a chunk; otherwise leave it to the fetch path.
|
||||
if (cr.total <= POLICY.CHUNK_BYTES) {
|
||||
await cache.put(new Request(url), new Response(await probe.blob(), {
|
||||
status: 200, headers: { 'Content-Type': meta.type, 'Content-Length': String(cr.total) }
|
||||
}));
|
||||
await sweepOldRevisions(cache, url);
|
||||
}
|
||||
return;
|
||||
}
|
||||
await cache.put(metaKey, new Response(JSON.stringify(meta), { headers: { 'Content-Type': 'application/json' } }));
|
||||
await cache.put(POLICY.chunkKey(url, 0), new Response(await probe.blob()));
|
||||
}
|
||||
|
||||
const ranges = POLICY.chunkRanges(meta.total, POLICY.CHUNK_BYTES);
|
||||
for (const r of ranges) {
|
||||
const key = POLICY.chunkKey(url, r.start);
|
||||
if (await cache.match(key)) continue; // already have it — this is the resume
|
||||
|
||||
let response;
|
||||
try {
|
||||
response = await fetch(new Request(url, {
|
||||
headers: { Range: 'bytes=' + r.start + '-' + r.end, 'If-Range': meta.validator }
|
||||
}));
|
||||
} catch (e) {
|
||||
return; // link died. Everything stored so far stays; the next sweep continues from here.
|
||||
}
|
||||
|
||||
const verdict = POLICY.resumeVerdict(
|
||||
response.status, response.headers.get('Content-Range'),
|
||||
r.start, meta.total, meta.validator, POLICY.validatorOf(response.headers)
|
||||
);
|
||||
if (verdict !== 'continue') {
|
||||
// 'restart' means the asset changed under us (If-Range declined) and 'discard' means we
|
||||
// cannot trust what came back. Either way the accumulated chunks describe a file that no
|
||||
// longer exists, and appending to them is the corruption this check exists to prevent.
|
||||
await dropChunks(cache, url);
|
||||
return;
|
||||
}
|
||||
await cache.put(key, new Response(await response.blob()));
|
||||
}
|
||||
|
||||
// Every piece present: assemble once, atomically as far as the player is concerned — the full
|
||||
// entry appears only when it is genuinely whole, so a cache hit can never be a fragment.
|
||||
const parts = [];
|
||||
for (const r of ranges) {
|
||||
const hit = await cache.match(POLICY.chunkKey(url, r.start));
|
||||
if (!hit) return; // evicted mid-assembly; try again later
|
||||
parts.push(await hit.blob());
|
||||
}
|
||||
const whole = new Blob(parts, { type: meta.type || 'application/octet-stream' });
|
||||
if (whole.size !== meta.total) { await dropChunks(cache, url); return; }
|
||||
await cache.put(new Request(url), new Response(whole, {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': meta.type || 'application/octet-stream', 'Content-Length': String(meta.total) }
|
||||
}));
|
||||
await dropChunks(cache, url);
|
||||
await sweepOldRevisions(cache, url);
|
||||
}
|
||||
|
||||
async function dropChunks(cache, url) {
|
||||
for (const key of await cache.keys()) {
|
||||
if (POLICY.isInternalKey(key.url) && POLICY.assetKey(key.url) === POLICY.assetKey(url) &&
|
||||
key.url.indexOf(revOf(url)) !== -1) {
|
||||
await cache.delete(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Delete entries for the SAME asset at a DIFFERENT revision.
|
||||
*
|
||||
* Replacing an asset changes the revision in its URL, which is what makes the new bytes a cache
|
||||
* miss everywhere — but it also means the superseded copy would sit there until the quota evicted
|
||||
* it. On a panel with a 1GB widget quota, a handful of replaced videos is the entire budget.
|
||||
*/
|
||||
async function sweepOldRevisions(cache, url) {
|
||||
const asset = POLICY.assetKey(url);
|
||||
const rev = revOf(url);
|
||||
for (const key of await cache.keys()) {
|
||||
if (POLICY.assetKey(key.url) !== asset) continue;
|
||||
if (revOf(key.url) === rev) continue;
|
||||
await cache.delete(key);
|
||||
}
|
||||
}
|
||||
|
||||
function revOf(url) {
|
||||
try { return new URL(url, self.location.href).searchParams.get('rev') || ''; } catch (e) { return ''; }
|
||||
}
|
||||
|
||||
async function handleContent(request) {
|
||||
const range = request.headers.get('range');
|
||||
const cache = await caches.open(CONTENT_CACHE);
|
||||
|
|
|
|||
|
|
@ -530,9 +530,30 @@ router.put('/:id/replace', upload.single('file'), async (req, res) => {
|
|||
console.warn('Thumbnail generation failed:', e.message);
|
||||
}
|
||||
|
||||
db.prepare(`UPDATE content SET filepath = ?, mime_type = ?, file_size = ?, thumbnail_path = ?, width = ?, height = ? WHERE id = ?`)
|
||||
// Bump the revision: this is the ONLY operation in the product that changes an asset's bytes
|
||||
// without changing its id, so it is the only thing that can make a player's cached copy wrong.
|
||||
// Players key their media cache on the revision, so this is what evicts it.
|
||||
//
|
||||
// strftime seconds can collide with the previous value if a replace lands inside the same second
|
||||
// as the upload (a small file, a scripted replace) — and a revision that does not change is a
|
||||
// cache that never updates. MAX(now, previous + 1) guarantees it moves.
|
||||
db.prepare(`UPDATE content
|
||||
SET filepath = ?, mime_type = ?, file_size = ?, thumbnail_path = ?, width = ?, height = ?,
|
||||
updated_at = MAX(CAST(strftime('%s','now') AS INTEGER), COALESCE(NULLIF(updated_at, 0), created_at) + 1)
|
||||
WHERE id = ?`)
|
||||
.run(filepath, mime, req.file.size, thumbnailPath, width, height, req.params.id);
|
||||
|
||||
// ...and tell the panels, which the old code did not. Without this the new bytes reached a screen
|
||||
// only when something else happened to trigger a playlist refresh — an operator replacing a video
|
||||
// watched the dashboard update and the screen keep playing the old one.
|
||||
const affected = db.prepare(`
|
||||
SELECT DISTINCT d.id as device_id FROM devices d
|
||||
JOIN playlists p ON d.playlist_id = p.id
|
||||
JOIN playlist_items pi ON pi.playlist_id = p.id
|
||||
WHERE pi.content_id = ?
|
||||
`).all(req.params.id).map((r) => r.device_id);
|
||||
pushContentUpdates(req, affected);
|
||||
|
||||
res.json(db.prepare('SELECT * FROM content WHERE id = ?').get(req.params.id));
|
||||
});
|
||||
|
||||
|
|
|
|||
130
server/test/content-revision.test.js
Normal file
130
server/test/content-revision.test.js
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
'use strict';
|
||||
|
||||
// Caching media for offline playback creates a way for a screen to be permanently WRONG.
|
||||
//
|
||||
// PUT /api/content/:id/replace is the only operation that changes an asset's bytes without changing
|
||||
// its id. Every player caches media keyed on that id, so before this the new bytes could not reach
|
||||
// a panel that already held the old ones — not "until the next refresh", but never. And because a
|
||||
// replace writes a NEW randomly-named file and unlinks the old one, the filepath baked into the
|
||||
// published playlist snapshot pointed at a deleted file, so web and BrightSign panels 404'd on the
|
||||
// item until somebody republished the playlist.
|
||||
//
|
||||
// The fix is one idea: the playlist snapshot captures the ARRANGEMENT, not the bytes, so the byte
|
||||
// facts are refreshed at send time — exactly as widget revs already were.
|
||||
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const crypto = require('node:crypto');
|
||||
process.env.DATA_DIR = path.join(os.tmpdir(), 'st-rev-' + crypto.randomBytes(4).toString('hex'));
|
||||
process.env.SELF_HOSTED = 'true';
|
||||
process.env.NODE_ENV = 'test';
|
||||
|
||||
const { test, before, after } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const http = require('node:http');
|
||||
const { Server } = require('socket.io');
|
||||
const { db } = require('../db/database');
|
||||
const setupDeviceSocket = require('../ws/deviceSocket');
|
||||
|
||||
// buildPlaylistPayload is only attached to the module once the socket layer is set up, so the
|
||||
// payload is exercised exactly as the real send path builds it rather than through a copy.
|
||||
let httpServer, io, buildPlaylistPayload;
|
||||
|
||||
const DEV = 'dev-rev';
|
||||
const PL = 'pl-rev';
|
||||
const CID = 'content-rev';
|
||||
|
||||
function snapshot(items) {
|
||||
db.prepare('UPDATE playlists SET published_snapshot = ? WHERE id = ?').run(JSON.stringify(items), PL);
|
||||
}
|
||||
function itemFor(deviceId) {
|
||||
return buildPlaylistPayload(deviceId).assignments[0];
|
||||
}
|
||||
|
||||
before(() => {
|
||||
httpServer = http.createServer(); io = new Server(httpServer); setupDeviceSocket(io);
|
||||
buildPlaylistPayload = setupDeviceSocket.buildPlaylistPayload;
|
||||
|
||||
db.prepare('INSERT INTO users (id, email) VALUES (?,?)').run('u', 'rev@example.test');
|
||||
db.prepare('INSERT INTO content (id, filename, mime_type, file_size, filepath, created_at, updated_at) VALUES (?,?,?,?,?,?,?)')
|
||||
.run(CID, 'clip.mp4', 'video/mp4', 100, 'aaaa.mp4', 1000, 1000);
|
||||
db.prepare('INSERT INTO playlists (id, user_id, name) VALUES (?,?,?)').run(PL, 'u', 'rev playlist');
|
||||
db.prepare('INSERT INTO devices (id, name, playlist_id) VALUES (?,?,?)').run(DEV, 'panel', PL);
|
||||
db.prepare('INSERT INTO playlist_items (playlist_id, content_id, sort_order, duration_sec) VALUES (?,?,?,?)')
|
||||
.run(PL, CID, 0, 10);
|
||||
snapshot([{ content_id: CID, filename: 'clip.mp4', mime_type: 'video/mp4', filepath: 'aaaa.mp4', duration_sec: 10 }]);
|
||||
});
|
||||
|
||||
test('every content item carries a revision the player can key a cache on', () => {
|
||||
assert.equal(itemFor(DEV).content_rev, 1000);
|
||||
});
|
||||
|
||||
test('THE BUG: replacing the bytes changes the revision, so a cached copy is a miss', () => {
|
||||
// Without a value that moves, a player holding the old bytes has no way to learn they are stale:
|
||||
// same id, same URL, same everything.
|
||||
const before = itemFor(DEV).content_rev;
|
||||
db.prepare("UPDATE content SET filepath = 'bbbb.mp4', updated_at = MAX(CAST(strftime('%s','now') AS INTEGER), updated_at + 1) WHERE id = ?").run(CID);
|
||||
const after = itemFor(DEV).content_rev;
|
||||
assert.ok(after > before, `revision must advance on replace (${before} -> ${after})`);
|
||||
});
|
||||
|
||||
test('THE OTHER HALF: the refreshed filepath points at the file that now exists', () => {
|
||||
// The snapshot still says aaaa.mp4, which was unlinked by the replace. The web player builds its
|
||||
// URL from this field, so a stale value is not a caching problem — it is a 404 and a dark screen.
|
||||
assert.equal(itemFor(DEV).filepath, 'bbbb.mp4');
|
||||
});
|
||||
|
||||
test('a same-second replace still advances the revision', () => {
|
||||
// strftime resolution is one second. A scripted replace, or a small file, lands inside the same
|
||||
// second as the previous write — and a revision that does not move is a cache that never updates.
|
||||
const first = itemFor(DEV).content_rev;
|
||||
for (let i = 0; i < 3; i++) {
|
||||
db.prepare("UPDATE content SET updated_at = MAX(CAST(strftime('%s','now') AS INTEGER), COALESCE(NULLIF(updated_at,0), created_at) + 1) WHERE id = ?").run(CID);
|
||||
}
|
||||
assert.ok(itemFor(DEV).content_rev >= first + 3);
|
||||
});
|
||||
|
||||
test('an untouched asset keeps a STABLE revision across sends', () => {
|
||||
// Just as load-bearing as the change case: a revision that moved on its own would re-download the
|
||||
// whole playlist on every heartbeat, over the link least able to afford it.
|
||||
const a = itemFor(DEV).content_rev;
|
||||
const b = itemFor(DEV).content_rev;
|
||||
assert.equal(a, b);
|
||||
});
|
||||
|
||||
test('a row migrated in before the column existed resolves to its creation time, not zero', () => {
|
||||
// The ALTER lands the column as 0. Collapsing every such asset onto revision "0" would make two
|
||||
// different assets look identically fresh to a cache keyed on it.
|
||||
db.prepare('INSERT INTO content (id, filename, mime_type, file_size, filepath, created_at, updated_at) VALUES (?,?,?,?,?,?,?)')
|
||||
.run('legacy', 'old.mp4', 'video/mp4', 10, 'old.mp4', 4242, 0);
|
||||
db.prepare('INSERT INTO playlists (id, user_id, name) VALUES (?,?,?)').run('pl-legacy', 'u', 'legacy');
|
||||
db.prepare('INSERT INTO devices (id, name, playlist_id) VALUES (?,?,?)').run('dev-legacy', 'legacy panel', 'pl-legacy');
|
||||
db.prepare('UPDATE playlists SET published_snapshot = ? WHERE id = ?')
|
||||
.run(JSON.stringify([{ content_id: 'legacy', filepath: 'old.mp4' }]), 'pl-legacy');
|
||||
assert.equal(itemFor('dev-legacy').content_rev, 4242);
|
||||
});
|
||||
|
||||
test('a widget item is untouched — it has no content row to refresh from', () => {
|
||||
db.prepare('INSERT INTO playlists (id, user_id, name) VALUES (?,?,?)').run('pl-w', 'u', 'widget');
|
||||
db.prepare('INSERT INTO devices (id, name, playlist_id) VALUES (?,?,?)').run('dev-w', 'w panel', 'pl-w');
|
||||
db.prepare('UPDATE playlists SET published_snapshot = ? WHERE id = ?')
|
||||
.run(JSON.stringify([{ widget_id: 'w1', widget_rev: 7 }]), 'pl-w');
|
||||
const item = itemFor('dev-w');
|
||||
assert.equal(item.content_rev, undefined);
|
||||
assert.equal(item.widget_id, 'w1');
|
||||
});
|
||||
|
||||
test('an item whose content row was deleted keeps its published fields rather than being blanked', () => {
|
||||
// The delete path scrubs the snapshot separately; until it does, silently emptying filepath here
|
||||
// would turn a stale item into a broken one.
|
||||
db.prepare('INSERT INTO playlists (id, user_id, name) VALUES (?,?,?)').run('pl-gone', 'u', 'gone');
|
||||
db.prepare('INSERT INTO devices (id, name, playlist_id) VALUES (?,?,?)').run('dev-gone', 'gone panel', 'pl-gone');
|
||||
db.prepare('UPDATE playlists SET published_snapshot = ? WHERE id = ?')
|
||||
.run(JSON.stringify([{ content_id: 'never-existed', filepath: 'ghost.mp4' }]), 'pl-gone');
|
||||
assert.equal(itemFor('dev-gone').filepath, 'ghost.mp4');
|
||||
});
|
||||
|
||||
after(() => {
|
||||
try { setupDeviceSocket.__resetTimers(); } catch { /* */ }
|
||||
try { io.close(); httpServer.close(); } catch { /* */ }
|
||||
});
|
||||
|
|
@ -147,3 +147,88 @@ test('an unknown quota never triggers eviction — absence of a number is not a
|
|||
assert.equal(P.needsEviction(999, 999, 0), false);
|
||||
assert.equal(P.needsEviction(999, 999, undefined), false);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
// Resumable transfer. The arithmetic below is what stops a resumed download from being corrupt,
|
||||
// and it is shared by the service worker and the Tizen media cache — neither of which can be
|
||||
// tested without a device, which is exactly why the decisions live here.
|
||||
// ---------------------------------------------------------------------------------------------
|
||||
|
||||
test('chunkRanges covers every byte exactly once, with an inclusive final range', () => {
|
||||
const r = P.chunkRanges(10, 4);
|
||||
assert.deepEqual(r, [{ start: 0, end: 3 }, { start: 4, end: 7 }, { start: 8, end: 9 }]);
|
||||
// The gap-or-overlap check: a one-byte error here corrupts the middle of a video in a way that
|
||||
// only shows up on playback, long after the download "succeeded".
|
||||
const big = P.chunkRanges(1000, 256);
|
||||
let cursor = 0;
|
||||
for (const c of big) { assert.equal(c.start, cursor); cursor = c.end + 1; }
|
||||
assert.equal(cursor, 1000);
|
||||
});
|
||||
|
||||
test('chunkRanges on an exact multiple does not emit a trailing empty range', () => {
|
||||
assert.deepEqual(P.chunkRanges(8, 4), [{ start: 0, end: 3 }, { start: 4, end: 7 }]);
|
||||
});
|
||||
|
||||
test('chunkRanges of an unknown or empty size is no chunks, not one bad one', () => {
|
||||
assert.deepEqual(P.chunkRanges(0, 4), []);
|
||||
assert.deepEqual(P.chunkRanges(-1, 4), []);
|
||||
});
|
||||
|
||||
test('parseContentRange refuses anything without a verifiable total', () => {
|
||||
assert.deepEqual(P.parseContentRange('bytes 4-7/10'), { start: 4, end: 7, total: 10 });
|
||||
assert.equal(P.parseContentRange('bytes 4-7/*'), null, 'no total = nothing to check against');
|
||||
assert.equal(P.parseContentRange('items 0-1/2'), null);
|
||||
assert.equal(P.parseContentRange(null), null);
|
||||
});
|
||||
|
||||
test('resumeVerdict continues only on a verified 206 at the offset we asked for', () => {
|
||||
assert.equal(P.resumeVerdict(206, 'bytes 4-7/10', 4, 10, '"v1"', '"v1"'), 'continue');
|
||||
});
|
||||
|
||||
test('resumeVerdict RESTARTS on a 200 — the server declined our range', () => {
|
||||
// If-Range with a stale validator produces exactly this, and it is the mechanism that stops a
|
||||
// changed asset being spliced. Treating it as a chunk to append is the corruption.
|
||||
assert.equal(P.resumeVerdict(200, null, 4, 10, '"v1"', '"v2"'), 'restart');
|
||||
});
|
||||
|
||||
test('resumeVerdict discards a 206 that starts anywhere other than where we asked', () => {
|
||||
assert.equal(P.resumeVerdict(206, 'bytes 0-7/10', 4, 10, '"v1"', '"v1"'), 'discard');
|
||||
});
|
||||
|
||||
test('resumeVerdict discards a 206 whose total disagrees with what we are assembling', () => {
|
||||
// Same offset, different asset length: appending would leave a file that is complete by our
|
||||
// count and wrong by every other measure.
|
||||
assert.equal(P.resumeVerdict(206, 'bytes 4-7/99', 4, 10, '"v1"', '"v1"'), 'discard');
|
||||
});
|
||||
|
||||
test('resumeVerdict discards a 206 whose validator changed underneath us', () => {
|
||||
assert.equal(P.resumeVerdict(206, 'bytes 4-7/10', 4, 10, '"v1"', '"v2"'), 'discard');
|
||||
});
|
||||
|
||||
test('resumeVerdict discards 416 and every unexpected status', () => {
|
||||
assert.equal(P.resumeVerdict(416, null, 4, 10, '"v1"', null), 'discard');
|
||||
assert.equal(P.resumeVerdict(500, null, 4, 10, '"v1"', null), 'discard');
|
||||
assert.equal(P.resumeVerdict(0, null, 4, 10, '"v1"', null), 'discard');
|
||||
});
|
||||
|
||||
test('validatorOf prefers ETag and reports nothing when there is nothing to trust', () => {
|
||||
const h = (o) => ({ get: (k) => o[k.toLowerCase()] || null });
|
||||
assert.equal(P.validatorOf(h({ etag: '"v1"', 'last-modified': 'Mon' })), '"v1"');
|
||||
assert.equal(P.validatorOf(h({ 'last-modified': 'Mon' })), 'Mon');
|
||||
assert.equal(P.validatorOf(h({})), null, 'no validator means the transfer is not resumable');
|
||||
});
|
||||
|
||||
test('assetKey ignores the revision so a store can sweep its own predecessors', () => {
|
||||
const a = 'http://s/uploads/content/clip.mp4?rev=100';
|
||||
const b = 'http://s/uploads/content/clip.mp4?rev=200';
|
||||
assert.equal(P.assetKey(a), P.assetKey(b));
|
||||
assert.notEqual(P.assetKey(a), P.assetKey('http://s/uploads/content/other.mp4?rev=100'));
|
||||
});
|
||||
|
||||
test('chunk keys are recognisable as internal, so one can never be served as the asset', () => {
|
||||
const k = P.chunkKey('http://s/uploads/content/clip.mp4?rev=100', 4194304);
|
||||
assert.ok(P.isInternalKey(k));
|
||||
assert.ok(!P.isInternalKey('http://s/uploads/content/clip.mp4?rev=100'));
|
||||
// ...and the revision survives into the chunk key, or two revisions would share chunks.
|
||||
assert.match(k, /rev=100/);
|
||||
});
|
||||
|
|
|
|||
226
server/test/sw-content-prefetch.test.js
Normal file
226
server/test/sw-content-prefetch.test.js
Normal file
|
|
@ -0,0 +1,226 @@
|
|||
'use strict';
|
||||
|
||||
// Drives the REAL service worker (server/player/sw.js) against a fake Cache API and a deliberately
|
||||
// bad link. The policy tests cover the arithmetic; this covers the orchestration around it, which is
|
||||
// where a resumed download actually gets corrupted: appending the wrong chunk, publishing a
|
||||
// half-assembled asset, or serving a bookkeeping entry as if it were a video.
|
||||
//
|
||||
// It matters that this runs the shipped file rather than a copy — the worker cannot be exercised on
|
||||
// a device without deploying to one, and "the chunks assemble correctly" is not something you want
|
||||
// to discover from a panel showing a corrupt video.
|
||||
|
||||
const { test, beforeEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const vm = require('node:vm');
|
||||
|
||||
const SW_SRC = fs.readFileSync(path.join(__dirname, '..', 'player', 'sw.js'), 'utf8');
|
||||
const POLICY_PATH = path.join(__dirname, '..', 'lib', 'player-cache-policy.js');
|
||||
|
||||
const ASSET = 'http://s/uploads/content/clip.mp4?rev=100';
|
||||
|
||||
/** Cache API over a Map. Keys are URLs, exactly as the real one behaves for our uses. */
|
||||
class FakeCache {
|
||||
constructor() { this.map = new Map(); }
|
||||
#url(req) { return typeof req === 'string' ? req : req.url; }
|
||||
async match(req) { const r = this.map.get(this.#url(req)); return r ? r.clone() : undefined; }
|
||||
async put(req, res) { this.map.set(this.#url(req), res); }
|
||||
async keys() { return [...this.map.keys()].map((u) => ({ url: u })); }
|
||||
async delete(req) { return this.map.delete(this.#url(req)); }
|
||||
}
|
||||
|
||||
/**
|
||||
* A server for one asset. `failEvery` drops the connection on every Nth request (0 = never), which
|
||||
* is what a marginal link looks like from the client's side.
|
||||
*/
|
||||
function makeServer(body, { etag = '"v1"', failEvery = 0, rangeSupport = true } = {}) {
|
||||
const state = { body, etag, failEvery, rangeSupport, requests: 0, ranged: [] };
|
||||
state.fetch = async (request) => {
|
||||
state.requests++;
|
||||
if (state.failEvery && state.requests % state.failEvery === 0) throw new Error('network dropped');
|
||||
|
||||
const range = request.headers.get('Range');
|
||||
const ifRange = request.headers.get('If-Range');
|
||||
if (!range || !state.rangeSupport) {
|
||||
return new Response(state.body, { status: 200, headers: { ETag: state.etag, 'Content-Type': 'video/mp4' } });
|
||||
}
|
||||
// If-Range with a stale validator: the server must send the WHOLE asset, not a tail. This is
|
||||
// the mechanism that stops a resume splicing two different files together.
|
||||
if (ifRange && ifRange !== state.etag) {
|
||||
return new Response(state.body, { status: 200, headers: { ETag: state.etag, 'Content-Type': 'video/mp4' } });
|
||||
}
|
||||
const m = /bytes=(\d+)-(\d*)/.exec(range);
|
||||
const start = Number(m[1]);
|
||||
if (start >= state.body.length) {
|
||||
return new Response('', { status: 416, headers: { 'Content-Range': `bytes */${state.body.length}` } });
|
||||
}
|
||||
const end = m[2] === '' ? state.body.length - 1 : Math.min(Number(m[2]), state.body.length - 1);
|
||||
state.ranged.push([start, end]);
|
||||
return new Response(state.body.slice(start, end + 1), {
|
||||
status: 206,
|
||||
headers: {
|
||||
'Content-Range': `bytes ${start}-${end}/${state.body.length}`,
|
||||
ETag: state.etag,
|
||||
'Content-Type': 'video/mp4'
|
||||
}
|
||||
});
|
||||
};
|
||||
return state;
|
||||
}
|
||||
|
||||
let sandbox, caches, server;
|
||||
|
||||
function load(srv, chunkBytes) {
|
||||
server = srv;
|
||||
const contentCache = new FakeCache();
|
||||
const shellCache = new FakeCache();
|
||||
caches = {
|
||||
open: async (name) => (name === 'rd-content-v1' ? contentCache : shellCache),
|
||||
keys: async () => ['rd-content-v1'],
|
||||
delete: async () => true,
|
||||
match: async () => undefined,
|
||||
_content: contentCache
|
||||
};
|
||||
|
||||
sandbox = {
|
||||
caches,
|
||||
fetch: (req) => server.fetch(req),
|
||||
Response, Request, Blob, URL, console,
|
||||
location: { href: 'http://s/player/index.html' },
|
||||
navigator: {},
|
||||
importScripts() {
|
||||
// The worker importScripts()es the same policy module the Node tests require, so both sides
|
||||
// are provably the same rules rather than two implementations that agree today.
|
||||
delete require.cache[require.resolve(POLICY_PATH)];
|
||||
sandbox.self.PlayerCachePolicy = require(POLICY_PATH);
|
||||
},
|
||||
addEventListener() {},
|
||||
skipWaiting() {},
|
||||
clients: { claim() {} }
|
||||
};
|
||||
sandbox.self = sandbox;
|
||||
vm.createContext(sandbox);
|
||||
vm.runInContext(SW_SRC, sandbox);
|
||||
// A 4MB production chunk would make these tests move 100MB around; the logic is size-agnostic.
|
||||
if (chunkBytes) sandbox.self.PlayerCachePolicy.CHUNK_BYTES = chunkBytes;
|
||||
return sandbox;
|
||||
}
|
||||
|
||||
const bytes = (n, fill) => Buffer.alloc(n, fill);
|
||||
async function cachedBody(url = ASSET) {
|
||||
const hit = await caches._content.match(url);
|
||||
return hit ? Buffer.from(await hit.arrayBuffer()) : null;
|
||||
}
|
||||
const internalKeys = () => [...caches._content.map.keys()].filter((k) => k.includes('__st_part'));
|
||||
|
||||
beforeEach(() => { sandbox = null; });
|
||||
|
||||
test('THE BUG: a link that drops every other request still assembles a byte-perfect asset', async () => {
|
||||
// Each call gets a chunk or two and dies. Without accumulation this is an infinite loop that
|
||||
// caches nothing — the panel keeps an empty cache and goes dark the moment the uplink does.
|
||||
const body = bytes(1000, 0x41);
|
||||
const sw = load(makeServer(body, { failEvery: 2 }), 100);
|
||||
|
||||
for (let pass = 0; pass < 40 && !(await cachedBody()); pass++) {
|
||||
try { await sw.ensureCached(ASSET); } catch (e) { /* the link, not the worker */ }
|
||||
}
|
||||
|
||||
const got = await cachedBody();
|
||||
assert.ok(got, 'the asset must eventually be cached');
|
||||
assert.deepEqual(got, body, 'reassembled bytes must be identical to the original');
|
||||
});
|
||||
|
||||
test('the full entry appears only when whole — a fragment is never published', async () => {
|
||||
// The invariant that protects playback: a cache hit is always a complete asset. Publishing early
|
||||
// would hand a media element a truncated file that it cannot report as "incomplete", only as
|
||||
// broken.
|
||||
const body = bytes(1000, 0x42);
|
||||
const sw = load(makeServer(body, { failEvery: 3 }), 100);
|
||||
|
||||
for (let pass = 0; pass < 40; pass++) {
|
||||
try { await sw.ensureCached(ASSET); } catch (e) { /* */ }
|
||||
const partial = await cachedBody();
|
||||
if (partial) { assert.equal(partial.length, body.length, 'a published entry must be the whole asset'); break; }
|
||||
assert.ok(internalKeys().length >= 0);
|
||||
}
|
||||
assert.deepEqual(await cachedBody(), body);
|
||||
});
|
||||
|
||||
test('bookkeeping entries are cleaned up once the asset is whole', async () => {
|
||||
const body = bytes(500, 0x43);
|
||||
const sw = load(makeServer(body), 100);
|
||||
await sw.ensureCached(ASSET);
|
||||
assert.deepEqual(await cachedBody(), body);
|
||||
assert.deepEqual(internalKeys(), [], 'chunks and meta must not outlive the assembled asset');
|
||||
});
|
||||
|
||||
test('an already-cached asset costs no requests at all', async () => {
|
||||
// The prefetch runs on every playlist sweep. Re-fetching a cached asset each time would be a
|
||||
// constant drain on the link least able to afford it.
|
||||
const srv = makeServer(bytes(500, 0x44));
|
||||
const sw = load(srv, 100);
|
||||
await sw.ensureCached(ASSET);
|
||||
const after = srv.requests;
|
||||
await sw.ensureCached(ASSET);
|
||||
assert.equal(srv.requests, after, 'a second pass over a cached asset must not touch the network');
|
||||
});
|
||||
|
||||
test('an asset replaced mid-transfer is NOT spliced — the chunks are discarded', async () => {
|
||||
// The corruption this whole design exists to prevent: appending the tail of the new asset to the
|
||||
// head of the old one yields a file of exactly the right length that is wrong throughout, and it
|
||||
// would pass every completeness check we have.
|
||||
const v1 = bytes(1000, 0x61);
|
||||
const v2 = bytes(1000, 0x62);
|
||||
const srv = makeServer(v1, { failEvery: 3 });
|
||||
const sw = load(srv, 100);
|
||||
|
||||
try { await sw.ensureCached(ASSET); } catch (e) { /* */ }
|
||||
assert.ok(internalKeys().length > 0, 'expected partial progress to exist before the swap');
|
||||
|
||||
srv.body = v2; srv.etag = '"v2"'; srv.failEvery = 0;
|
||||
|
||||
// The first pass after the swap gets a 200 from If-Range and must throw the v1 chunks away
|
||||
// rather than continue on top of them; the next rebuilds from scratch.
|
||||
await sw.ensureCached(ASSET);
|
||||
for (let pass = 0; pass < 10 && !(await cachedBody()); pass++) await sw.ensureCached(ASSET);
|
||||
|
||||
const got = await cachedBody();
|
||||
assert.ok(got, 'the replaced asset must still end up cached');
|
||||
assert.deepEqual(got, v2, 'the cached asset must be all-v2, with no v1 bytes spliced in');
|
||||
assert.ok(!got.includes(0x61), 'not one byte of the superseded asset may survive');
|
||||
});
|
||||
|
||||
test('a superseded revision is swept rather than left to fill the quota', async () => {
|
||||
// Replacing an asset changes the revision in the URL, which is what makes the new bytes a miss.
|
||||
// Without the sweep the old copy sits there until the quota evicts it — on a 1GB panel budget a
|
||||
// handful of replaced videos is the whole cache.
|
||||
const oldUrl = 'http://s/uploads/content/clip.mp4?rev=100';
|
||||
const newUrl = 'http://s/uploads/content/clip.mp4?rev=200';
|
||||
const sw = load(makeServer(bytes(300, 0x45)), 100);
|
||||
|
||||
await sw.ensureCached(oldUrl);
|
||||
assert.ok(await cachedBody(oldUrl));
|
||||
|
||||
server.body = bytes(300, 0x46); server.etag = '"v2"';
|
||||
await sw.ensureCached(newUrl);
|
||||
|
||||
assert.ok(await cachedBody(newUrl), 'the new revision is cached');
|
||||
assert.equal(await cachedBody(oldUrl), null, 'the superseded revision is gone');
|
||||
});
|
||||
|
||||
test('a server with no range support still caches the asset whole', async () => {
|
||||
// Not every deployment sits behind something that honours Range. Falling back to a plain store is
|
||||
// the pre-existing behaviour and remains correct — just not resumable.
|
||||
const body = bytes(600, 0x47);
|
||||
const sw = load(makeServer(body, { rangeSupport: false }), 100);
|
||||
await sw.ensureCached(ASSET);
|
||||
assert.deepEqual(await cachedBody(), body);
|
||||
});
|
||||
|
||||
test('a chunked transfer asks for each range exactly once, in order', async () => {
|
||||
const sw = load(makeServer(bytes(1000, 0x48)), 250);
|
||||
await sw.ensureCached(ASSET);
|
||||
const starts = server.ranged.map((r) => r[0]);
|
||||
assert.deepEqual(starts, [0, 250, 500, 750], 'no gaps, no repeats, no overlap');
|
||||
});
|
||||
223
server/test/tizen-media-cache.test.js
Normal file
223
server/test/tizen-media-cache.test.js
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
'use strict';
|
||||
|
||||
// Tizen was the one player that cached nothing but the playlist: a panel came back from a reboot
|
||||
// knowing exactly what to show and fetched every frame of it from a server that was not there.
|
||||
// tizen/js/media-cache.js fixes that, and none of it can be exercised on hardware without a TV — so
|
||||
// the decisions are all in injected-backend form and driven here against a fake one.
|
||||
//
|
||||
// The backend (resolve wgt-private, append to a stream, turn a file into a URI) is the only part
|
||||
// that needs a device, and it is the part with no logic in it.
|
||||
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const path = require('node:path');
|
||||
|
||||
const MediaCache = require(path.join(__dirname, '..', '..', 'tizen', 'js', 'media-cache.js'));
|
||||
const CHUNK = MediaCache.CHUNK_BYTES;
|
||||
|
||||
/**
|
||||
* A fake TV. `failEvery` drops every Nth request — a marginal link, which is the condition the
|
||||
* whole resumable design exists for.
|
||||
*/
|
||||
function fakeBackend(asset, opts = {}) {
|
||||
const b = {
|
||||
asset, // { bytes: number[], etag, total }
|
||||
files: new Map(), // name -> number[]
|
||||
index: {},
|
||||
requests: 0,
|
||||
failEvery: opts.failEvery || 0,
|
||||
rangeSupport: opts.rangeSupport !== false,
|
||||
available: () => true,
|
||||
loadIndex: () => b.index,
|
||||
saveIndex: (i) => { b.index = JSON.parse(JSON.stringify(i)); },
|
||||
httpRange(url, start, end, validator) {
|
||||
b.requests++;
|
||||
if (b.failEvery && b.requests % b.failEvery === 0) throw new Error('link dropped');
|
||||
const body = b.asset.bytes;
|
||||
if (!b.rangeSupport) {
|
||||
return { status: 200, start: 0, total: body.length, validator: b.asset.etag, body: body.slice() };
|
||||
}
|
||||
// If-Range with a stale validator: the server sends the WHOLE asset, which is the signal to
|
||||
// start over rather than append a tail from a different file.
|
||||
if (validator && validator !== b.asset.etag) {
|
||||
return { status: 200, start: 0, total: body.length, validator: b.asset.etag, body: body.slice() };
|
||||
}
|
||||
if (start >= body.length) return { status: 416, start, total: body.length, validator: b.asset.etag, body: null };
|
||||
const stop = Math.min(end, body.length - 1);
|
||||
return {
|
||||
status: 206, start, total: body.length, validator: b.asset.etag,
|
||||
body: body.slice(start, stop + 1)
|
||||
};
|
||||
},
|
||||
appendPart(contentId, body, offset) {
|
||||
const name = contentId + '.part';
|
||||
const cur = b.files.get(name) || [];
|
||||
if (offset === 0) b.files.set(name, body.slice());
|
||||
else {
|
||||
if (cur.length !== offset) return 0; // a real append cannot write into a hole
|
||||
b.files.set(name, cur.concat(body));
|
||||
}
|
||||
return body.length;
|
||||
},
|
||||
promotePart(contentId) {
|
||||
const part = b.files.get(contentId + '.part');
|
||||
if (!part) return null;
|
||||
b.files.set(contentId, part);
|
||||
b.files.delete(contentId + '.part');
|
||||
return { path: '/wgt-private/' + contentId, uri: 'file:///wgt-private/' + contentId };
|
||||
},
|
||||
remove(contentId) { b.files.delete(contentId); b.files.delete(contentId + '.part'); }
|
||||
};
|
||||
return b;
|
||||
}
|
||||
|
||||
const asset = (n, fill, etag = '"v1"') => ({ bytes: new Array(n).fill(fill), etag });
|
||||
const urlFor = (it) => 'http://s/api/content/' + it.content_id + '/file?rev=' + it.content_rev;
|
||||
|
||||
test('THE GAP: media is cached at all, and resolves to a local file the player can open', async () => {
|
||||
const b = fakeBackend(asset(CHUNK, 7));
|
||||
const mc = new MediaCache(b);
|
||||
|
||||
assert.equal(mc.localUrl('c1', 5), null, 'nothing cached yet');
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5 }], urlFor);
|
||||
assert.equal(mc.localUrl('c1', 5), 'file:///wgt-private/c1');
|
||||
assert.deepEqual(b.files.get('c1'), new Array(CHUNK).fill(7));
|
||||
});
|
||||
|
||||
test('a link that drops every other request still completes the asset', async () => {
|
||||
// Without accumulation this never finishes: each attempt restarts from zero, so an asset larger
|
||||
// than one uninterrupted transfer is never cached and the panel has nothing to fall back on.
|
||||
const b = fakeBackend(asset(CHUNK * 4, 3), { failEvery: 2 });
|
||||
const mc = new MediaCache(b);
|
||||
const items = [{ content_id: 'c1', content_rev: 5 }];
|
||||
|
||||
for (let pass = 0; pass < 30 && !mc.localUrl('c1', 5); pass++) await mc.sync(items, urlFor);
|
||||
|
||||
assert.ok(mc.localUrl('c1', 5), 'the asset must eventually be cached');
|
||||
assert.equal(b.files.get('c1').length, CHUNK * 4);
|
||||
});
|
||||
|
||||
test('a partial is never promoted — an incomplete file is not offered to the player', async () => {
|
||||
const b = fakeBackend(asset(CHUNK * 3, 9), { failEvery: 1 }); // every request fails
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5 }], urlFor);
|
||||
assert.equal(mc.localUrl('c1', 5), null);
|
||||
assert.equal(b.files.get('c1'), undefined, 'no whole file exists');
|
||||
});
|
||||
|
||||
test('progress accumulates across passes rather than restarting', async () => {
|
||||
const b = fakeBackend(asset(CHUNK * 3, 4));
|
||||
const mc = new MediaCache(b);
|
||||
// One step at a time, so the resume offset is observable.
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'progress');
|
||||
assert.equal(mc.index.c1.bytes, CHUNK);
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'progress');
|
||||
assert.equal(mc.index.c1.bytes, CHUNK * 2, 'the second attempt appended, it did not restart');
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'done');
|
||||
});
|
||||
|
||||
test('THE UPDATE HALF: a replaced asset is a miss, and the old bytes are deleted', async () => {
|
||||
// The trap that offline caching creates. Same content id, same URL path, different bytes — a
|
||||
// cache that cannot tell would keep playing last month's video forever.
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5 }], urlFor);
|
||||
assert.ok(mc.localUrl('c1', 5));
|
||||
|
||||
b.asset = asset(CHUNK, 2, '"v2"');
|
||||
assert.equal(mc.localUrl('c1', 6), null, 'a new revision must not match the cached copy');
|
||||
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 6 }], urlFor);
|
||||
assert.ok(mc.localUrl('c1', 6), 'the new revision is cached');
|
||||
assert.deepEqual(b.files.get('c1'), new Array(CHUNK).fill(2), 'and it is the NEW bytes');
|
||||
});
|
||||
|
||||
test('an asset replaced MID-transfer is discarded, not spliced', async () => {
|
||||
// Appending the tail of the new asset to the head of the old one produces a file of exactly the
|
||||
// right length that is wrong throughout — it would pass every completeness check there is.
|
||||
const b = fakeBackend(asset(CHUNK * 4, 0x61));
|
||||
const mc = new MediaCache(b);
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'progress');
|
||||
assert.equal(mc.index.c1.bytes, CHUNK);
|
||||
|
||||
b.asset = asset(CHUNK * 4, 0x62, '"v2"'); // replaced underneath us, same revision claim
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'done', 'a 200 carries the whole new asset');
|
||||
|
||||
const cached = b.files.get('c1');
|
||||
assert.equal(cached.length, CHUNK * 4);
|
||||
assert.ok(cached.every((v) => v === 0x62), 'not one byte of the superseded asset may survive');
|
||||
});
|
||||
|
||||
test('items dropped from the playlist have their bytes deleted', async () => {
|
||||
// Otherwise the cache only grows, and the failure eventually lands as a write error on whatever
|
||||
// happens to be downloading at the time.
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5 }], urlFor);
|
||||
assert.ok(b.files.get('c1'));
|
||||
|
||||
await mc.sync([{ content_id: 'c2', content_rev: 1 }], urlFor);
|
||||
assert.equal(b.files.get('c1'), undefined, 'an unreferenced asset must not linger');
|
||||
assert.equal(mc.index.c1, undefined);
|
||||
});
|
||||
|
||||
test('a cached asset costs no requests on later sweeps', async () => {
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
const items = [{ content_id: 'c1', content_rev: 5 }];
|
||||
await mc.sync(items, urlFor);
|
||||
const after = b.requests;
|
||||
await mc.sync(items, urlFor);
|
||||
await mc.sync(items, urlFor);
|
||||
assert.equal(b.requests, after, 'a cached asset must not be re-fetched every 60s');
|
||||
});
|
||||
|
||||
test('remote-url items are never downloaded', async () => {
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5, remote_url: 'https://example.com/live' }], urlFor);
|
||||
assert.equal(b.requests, 0);
|
||||
});
|
||||
|
||||
test('a server with no range support still caches the asset whole', async () => {
|
||||
const b = fakeBackend(asset(CHUNK * 2, 6), { rangeSupport: false });
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1', content_rev: 5 }], urlFor);
|
||||
assert.ok(mc.localUrl('c1', 5));
|
||||
assert.equal(b.files.get('c1').length, CHUNK * 2);
|
||||
});
|
||||
|
||||
test('a 416 discards a partial that is longer than the asset', async () => {
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
// Pretend a previous life left a longer partial behind.
|
||||
mc.index.c1 = { rev: 5, bytes: CHUNK * 9, total: CHUNK * 9, validator: '"v1"', complete: false };
|
||||
b.files.set('c1.part', new Array(CHUNK * 9).fill(0));
|
||||
assert.equal(await mc.fetchStep('c1', 5, 'http://s/x'), 'restart');
|
||||
assert.equal(b.files.get('c1.part'), undefined, 'the stale partial must be gone');
|
||||
});
|
||||
|
||||
test('the index survives a restart — progress is not lost with the process', async () => {
|
||||
// A signage panel reboots. If the index lived only in memory, every reboot during a slow
|
||||
// download would throw the transfer away, which on a bad link means it never finishes.
|
||||
const b = fakeBackend(asset(CHUNK * 3, 8));
|
||||
const first = new MediaCache(b);
|
||||
assert.equal(await first.fetchStep('c1', 5, 'http://s/x'), 'progress');
|
||||
|
||||
const reborn = new MediaCache(b); // same backend = same persisted index + files
|
||||
assert.equal(reborn.index.c1.bytes, CHUNK, 'the resume offset survived');
|
||||
assert.equal(await reborn.fetchStep('c1', 5, 'http://s/x'), 'progress');
|
||||
assert.equal(reborn.index.c1.bytes, CHUNK * 2);
|
||||
});
|
||||
|
||||
test('an item with no revision still caches, and matches a copy stored without one', async () => {
|
||||
// Older servers do not send content_rev. Treating absent-vs-absent as a mismatch would re-download
|
||||
// the entire playlist on every sweep.
|
||||
const b = fakeBackend(asset(CHUNK, 1));
|
||||
const mc = new MediaCache(b);
|
||||
await mc.sync([{ content_id: 'c1' }], () => 'http://s/x');
|
||||
assert.ok(mc.localUrl('c1', undefined));
|
||||
const after = b.requests;
|
||||
await mc.sync([{ content_id: 'c1' }], () => 'http://s/x');
|
||||
assert.equal(b.requests, after);
|
||||
});
|
||||
|
|
@ -304,6 +304,46 @@ function refreshWidgetRevs(assignments) {
|
|||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* The same problem for uploaded media, with a worse failure mode.
|
||||
*
|
||||
* PUT /api/content/:id/replace swaps the BYTES behind a stable content id, and every player caches
|
||||
* media by that id. The URL does not change, so a cached copy is not merely stale until the next
|
||||
* refresh — it is stale forever, on every panel that already holds it, with no way for the player
|
||||
* to find out. That is the price of caching for offline: an asset that can never be updated.
|
||||
*
|
||||
* Stamping the revision at send time gives the players a value that changes exactly when the bytes
|
||||
* change. They put it in the request URL, so a replaced asset is a cache MISS everywhere at once,
|
||||
* and an asset nobody touched is not.
|
||||
*
|
||||
* COALESCE because a database migrated from before the column existed backfills it once, but rows
|
||||
* inserted between the ALTER and that backfill carry 0 — resolving those to created_at keeps the
|
||||
* revision stable and truthful rather than collapsing every new upload onto the same "0".
|
||||
*/
|
||||
const contentFactsOf = db.prepare(`
|
||||
SELECT COALESCE(NULLIF(updated_at, 0), created_at) AS rev, filepath, mime_type, file_size
|
||||
FROM content WHERE id = ?
|
||||
`);
|
||||
function refreshContentRevs(assignments) {
|
||||
if (!Array.isArray(assignments)) return;
|
||||
for (const a of assignments) {
|
||||
if (!a || !a.content_id) continue;
|
||||
try {
|
||||
const row = contentFactsOf.get(a.content_id);
|
||||
if (!row) continue; // deleted mid-flight; the purge sweeps the snapshot
|
||||
a.content_rev = row.rev ?? a.content_rev ?? 0;
|
||||
// A replace writes a NEW randomly-named file and unlinks the old one, so the filepath in a
|
||||
// published snapshot points at a file that no longer exists. The web player builds its media
|
||||
// URL from exactly that field: replacing an asset 404'd every web and BrightSign panel until
|
||||
// somebody thought to republish the playlist. Refreshing it here is the same fix as the
|
||||
// widget rev above — the snapshot is a snapshot of the ARRANGEMENT, not of the bytes.
|
||||
if (row.filepath) a.filepath = row.filepath;
|
||||
if (row.mime_type) a.mime_type = row.mime_type;
|
||||
if (row.file_size != null) a.file_size = row.file_size;
|
||||
} catch (_) { /* keep published */ }
|
||||
}
|
||||
}
|
||||
|
||||
function buildPlaylistPayload(deviceId) {
|
||||
const device = db.prepare('SELECT playlist_id, layout_id, orientation, wall_id, timezone, reported_timezone FROM devices WHERE id = ?').get(deviceId);
|
||||
|
||||
|
|
@ -313,6 +353,7 @@ function buildPlaylistPayload(deviceId) {
|
|||
if (playlist?.published_snapshot) {
|
||||
try { assignments = JSON.parse(playlist.published_snapshot); } catch (e) { assignments = []; }
|
||||
refreshWidgetRevs(assignments);
|
||||
refreshContentRevs(assignments);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -27,6 +27,12 @@
|
|||
hardware plane. Without this privilege the API throws SecurityError and the player falls
|
||||
back to per-element media volume, which cannot touch that plane. -->
|
||||
<tizen:privilege name="http://tizen.org/privilege/tv.audio"/>
|
||||
<!-- Offline media cache (js/media-cache.js). Content is written to wgt-private, which is the
|
||||
store Tizen documents as surviving reboots and app updates — without these the player can
|
||||
hold a playlist through an outage but none of the media it points at, which from the floor
|
||||
is the same as holding nothing. -->
|
||||
<tizen:privilege name="http://tizen.org/privilege/filesystem.read"/>
|
||||
<tizen:privilege name="http://tizen.org/privilege/filesystem.write"/>
|
||||
<tizen:privilege name="http://developer.samsung.com/privilege/network.public"/>
|
||||
|
||||
<!-- #125: Samsung B2B fleet control (reboot / panel power via b2bcontrol /
|
||||
|
|
|
|||
|
|
@ -59,6 +59,7 @@
|
|||
<script src="js/player.js"></script>
|
||||
<script src="js/device-control.js"></script>
|
||||
<script src="js/pip-overlay.js"></script>
|
||||
<script src="js/media-cache.js"></script>
|
||||
<script src="js/capabilities.js"></script>
|
||||
<script src="js/app.js"></script>
|
||||
</body>
|
||||
|
|
|
|||
|
|
@ -807,6 +807,24 @@
|
|||
// A2: cache the last RENDERABLE payload so a reboot / WS-outage with no connectivity replays it
|
||||
// instead of showing the idle card. Only non-suspended payloads are cached.
|
||||
try { set(LS.payload, JSON.stringify(payload)); } catch (e) {}
|
||||
// ...and cache the CONTENT the payload points at, which is the half that was missing. A playlist
|
||||
// that survives an outage and media that does not just means the panel knows precisely what it
|
||||
// cannot show. Deferred off the render path: the sweep is synchronous and this call arrives
|
||||
// while the stage is being repainted.
|
||||
try {{
|
||||
if (!window.__stMediaCache && window.MediaCache) window.__stMediaCache = window.MediaCache.create();
|
||||
var mc = window.__stMediaCache;
|
||||
if (mc && serverUrl) {{
|
||||
var mcItems = payload.assignments || [];
|
||||
var mcBase = serverUrl.replace(/\/+$/, '');
|
||||
setTimeout(function () {{
|
||||
mc.sync(mcItems, function (it) {{
|
||||
return mcBase + '/api/content/' + it.content_id + '/file'
|
||||
+ (it.content_rev ? '?rev=' + encodeURIComponent(it.content_rev) : '');
|
||||
}});
|
||||
}}, 2000);
|
||||
}}
|
||||
}} catch (e) {{ /* caching must never break the payload path */ }}
|
||||
// If we have content + we're paired, make sure we're on the stage.
|
||||
if (elPairing.classList.contains('hidden') === false) show(elStage);
|
||||
else if (elStage.classList.contains('hidden')) show(elStage);
|
||||
|
|
|
|||
|
|
@ -82,10 +82,16 @@
|
|||
// system.shell — no substantiated API on this surface. Claiming them would put back
|
||||
// exactly the dead buttons this change removes.
|
||||
// sync.native — no cross-player frame sync (that is BrightSign's SyncManager).
|
||||
// offline.cache — app.js caches the PAYLOAD so a reboot during an outage replays the
|
||||
// last playlist, but the media bytes are still fetched from the
|
||||
// network. Content does not survive an outage, so claiming offline
|
||||
// capability would overstate it.
|
||||
// (offline.cache is deliberately absent from this list — it is a RUNTIME check below.)
|
||||
|
||||
// offline.cache is a RUNTIME fact, not a platform one. app.js has always cached the payload,
|
||||
// but the media bytes were fetched from the network every time — so a panel survived an outage
|
||||
// knowing exactly what it could not show. media-cache.js changes that, WHERE the platform
|
||||
// actually gives us persistent storage. It does not on every build, and a panel that cannot
|
||||
// write to wgt-private must not claim an offline capability it does not have.
|
||||
try {
|
||||
if (window.MediaCache && (window.__stMediaCache || window.MediaCache.create())) caps.push('offline.cache');
|
||||
} catch (e) { /* no storage: the claim stays absent, which is the honest answer */ }
|
||||
|
||||
return caps;
|
||||
}
|
||||
|
|
|
|||
354
tizen/js/media-cache.js
Normal file
354
tizen/js/media-cache.js
Normal file
|
|
@ -0,0 +1,354 @@
|
|||
/*
|
||||
* Offline media cache for the Tizen player.
|
||||
*
|
||||
* Tizen was the one player that cached NOTHING but the playlist. A panel could come back from a
|
||||
* reboot knowing exactly what to show and then fetch every frame of it from a server that was not
|
||||
* there — the playlist survived the outage and the content did not, which from the floor looks the
|
||||
* same as having nothing at all. `offline.cache` is deliberately absent from the Tizen capability
|
||||
* baseline for exactly this reason; this is what makes it earnable.
|
||||
*
|
||||
* A service worker is not available here: the widget runs from an app:// origin, so the mechanism
|
||||
* the web player uses does not exist. The platform's own persistent store does: tizen.filesystem
|
||||
* under wgt-private, which survives reboots and app updates, and whose file:// URI both <video> and
|
||||
* AVPlay accept as a source.
|
||||
*
|
||||
* RESUMABLE, for the same reason as everywhere else in this product: a signage panel is often on
|
||||
* the worst link in the building, and a transfer that restarts from zero on every interruption
|
||||
* never finishes at all. Progress is appended to a `.part` file and continued with Range on the
|
||||
* next pass.
|
||||
*
|
||||
* REVISION-KEYED, so caching cannot make a screen permanently wrong. Replacing an asset in the
|
||||
* dashboard changes its content_rev; a cached copy at a different revision is treated as a miss and
|
||||
* re-fetched, and its bytes are deleted. Without that, "cached for offline" would mean "can never
|
||||
* be updated".
|
||||
*
|
||||
* The BACKEND is injected so every decision here is testable in Node without a TV (see
|
||||
* server/test/tizen-media-cache.test.js). The Tizen-specific parts — resolving wgt-private,
|
||||
* appending to a stream, turning a file into a URI — are the only things that need hardware, and
|
||||
* they are the parts with no logic in them.
|
||||
*/
|
||||
(function (root) {
|
||||
'use strict';
|
||||
|
||||
var INDEX_KEY = 'st_media_index';
|
||||
// 1MB. Smaller than the web player's 4MB because each chunk crosses the JS/native boundary as a
|
||||
// byte array here, and a 4MB array is a memory spike on a TV that a slow link does not justify.
|
||||
var CHUNK_BYTES = 1024 * 1024;
|
||||
|
||||
function MediaCache(backend) {
|
||||
this.backend = backend;
|
||||
this.index = backend.loadIndex() || {};
|
||||
this.busy = false;
|
||||
}
|
||||
|
||||
MediaCache.CHUNK_BYTES = CHUNK_BYTES;
|
||||
MediaCache.INDEX_KEY = INDEX_KEY;
|
||||
|
||||
MediaCache.prototype.save = function () {
|
||||
try { this.backend.saveIndex(this.index); } catch (e) { /* a full store must not break playback */ }
|
||||
};
|
||||
|
||||
/*
|
||||
* The local URI for an item, or null when the bytes we hold are not the bytes it wants.
|
||||
*
|
||||
* The revision comparison is the whole point: `!==` rather than a truthiness check, so an item
|
||||
* that arrives WITHOUT a revision (an older server) still matches a cached copy recorded without
|
||||
* one, and a changed revision never matches a stale copy.
|
||||
*/
|
||||
MediaCache.prototype.localUrl = function (contentId, rev) {
|
||||
if (!contentId) return null;
|
||||
var e = this.index[contentId];
|
||||
if (!e || !e.complete) return null;
|
||||
if (String(e.rev || '') !== String(rev || '')) return null;
|
||||
return e.uri || null;
|
||||
};
|
||||
|
||||
/*
|
||||
* Bring [contentId] a little closer to being cached. One call = one attempt: it transfers what
|
||||
* the link allows and returns, leaving the rest for the next sweep. Never throws.
|
||||
*
|
||||
* Returns 'done' | 'progress' | 'stalled' | 'restart' — 'progress' being the one that matters,
|
||||
* because a caller that cannot tell progress from failure will back a slow link off into a dead
|
||||
* one.
|
||||
*/
|
||||
MediaCache.prototype.fetchStep = function (contentId, rev, url) {
|
||||
var self = this;
|
||||
var e = this.index[contentId];
|
||||
|
||||
// A different revision means the bytes on disk describe an asset that no longer exists.
|
||||
if (e && String(e.rev || '') !== String(rev || '')) {
|
||||
this.drop(contentId);
|
||||
e = null;
|
||||
}
|
||||
if (e && e.complete) return Promise.resolve('done');
|
||||
|
||||
if (!e) {
|
||||
e = this.index[contentId] = { rev: rev, bytes: 0, total: 0, validator: null, complete: false, path: null, uri: null };
|
||||
}
|
||||
|
||||
var pending;
|
||||
try {
|
||||
pending = e.bytes > 0 && e.validator
|
||||
? this.backend.httpRange(url, e.bytes, e.bytes + CHUNK_BYTES - 1, e.validator)
|
||||
: this.backend.httpRange(url, 0, CHUNK_BYTES - 1, null);
|
||||
} catch (err) {
|
||||
return Promise.resolve('stalled');
|
||||
}
|
||||
// The backend hands back a promise (the real one is an ASYNC XHR — a synchronous request on
|
||||
// this thread would freeze the player for as long as the chunk takes, which on the links this
|
||||
// exists for is a stalled screen, not a slow download). Tests may hand back a plain value.
|
||||
return Promise.resolve(pending).then(function (res) {
|
||||
return self.applyChunk(contentId, rev, res);
|
||||
}, function () { return 'stalled'; });
|
||||
};
|
||||
|
||||
/* The decision half, given whatever the server said. Pure enough to reason about on its own. */
|
||||
MediaCache.prototype.applyChunk = function (contentId, rev, res) {
|
||||
var e = this.index[contentId];
|
||||
if (!e) return 'stalled';
|
||||
if (!res) return 'stalled';
|
||||
|
||||
if (res.status === 416) {
|
||||
// Our partial is at or past the end of the asset — it belongs to something else. Keeping it
|
||||
// would mean asking for a range past the end on every future sweep and never recovering.
|
||||
this.drop(contentId);
|
||||
return 'restart';
|
||||
}
|
||||
|
||||
if (res.status === 200) {
|
||||
// No range support, or If-Range told the server the asset changed. Either way the body is the
|
||||
// WHOLE asset and anything we already hold is wrong.
|
||||
this.drop(contentId);
|
||||
e = this.index[contentId] = { rev: rev, bytes: 0, total: res.total || 0, validator: res.validator || null, complete: false, path: null, uri: null };
|
||||
return this.commit(contentId, e, res.body, 0, res.total || (res.body && res.body.length) || 0) ? 'done' : 'stalled';
|
||||
}
|
||||
|
||||
if (res.status !== 206) return 'stalled';
|
||||
|
||||
// Content-Length on a 206 is the length of the CHUNK. The full size only comes from
|
||||
// Content-Range, and without it there is nothing to decide completeness against.
|
||||
if (!(res.total > 0) || res.start !== e.bytes) return 'stalled';
|
||||
if (e.total && res.total !== e.total) { this.drop(contentId); return 'restart'; }
|
||||
if (e.validator && res.validator && e.validator !== res.validator) { this.drop(contentId); return 'restart'; }
|
||||
|
||||
e.total = res.total;
|
||||
// No validator means no safe resume: a later attempt could append the tail of a different
|
||||
// asset. We keep the bytes only when we can prove on the next pass that they still belong.
|
||||
e.validator = res.validator || null;
|
||||
|
||||
var wrote = this.commit(contentId, e, res.body, e.bytes, res.total);
|
||||
if (!wrote) return 'stalled';
|
||||
if (!e.validator && !e.complete) { this.drop(contentId); return 'stalled'; }
|
||||
return e.complete ? 'done' : 'progress';
|
||||
};
|
||||
|
||||
/* Append [body] at [offset] and mark the entry complete when it reaches [total]. */
|
||||
MediaCache.prototype.commit = function (contentId, e, body, offset, total) {
|
||||
if (!body || !body.length) return false;
|
||||
var written;
|
||||
try {
|
||||
written = this.backend.appendPart(contentId, body, offset);
|
||||
} catch (err) {
|
||||
return false;
|
||||
}
|
||||
if (!(written > 0)) return false;
|
||||
e.bytes = offset + written;
|
||||
e.total = total || e.total;
|
||||
|
||||
if (e.total > 0 && e.bytes >= e.total) {
|
||||
// Promote only when whole. A player handed a partial file has no way to report it as
|
||||
// incomplete — only as broken.
|
||||
var promoted;
|
||||
try { promoted = this.backend.promotePart(contentId); } catch (err) { promoted = null; }
|
||||
if (!promoted) return false;
|
||||
e.complete = true;
|
||||
e.path = promoted.path;
|
||||
e.uri = promoted.uri;
|
||||
}
|
||||
this.save();
|
||||
return true;
|
||||
};
|
||||
|
||||
/* Forget an entry and delete its bytes, partial or complete. */
|
||||
MediaCache.prototype.drop = function (contentId) {
|
||||
try { this.backend.remove(contentId); } catch (e) { /* best effort */ }
|
||||
delete this.index[contentId];
|
||||
this.save();
|
||||
};
|
||||
|
||||
/*
|
||||
* Delete anything the current playlist does not reference, and anything at a superseded revision.
|
||||
*
|
||||
* Without this the cache only ever grows: a panel that has cycled through a year of campaigns
|
||||
* fills its storage with assets nobody will play again, and the failure lands as a write error on
|
||||
* whatever happens to be downloading at the time.
|
||||
*/
|
||||
MediaCache.prototype.prune = function (items) {
|
||||
var keep = {};
|
||||
(items || []).forEach(function (it) {
|
||||
if (it && it.content_id) keep[it.content_id] = String(it.content_rev || '');
|
||||
});
|
||||
var self = this;
|
||||
Object.keys(this.index).forEach(function (id) {
|
||||
var wanted = keep[id];
|
||||
if (wanted === undefined || wanted !== String(self.index[id].rev || '')) self.drop(id);
|
||||
});
|
||||
};
|
||||
|
||||
/*
|
||||
* One pass over the playlist: prune, then advance each uncached item in turn.
|
||||
*
|
||||
* Serialised on purpose. Three concurrent transfers on a link that cannot finish one produce
|
||||
* three unfinished transfers instead of one finished one — and on a TV they also produce three
|
||||
* simultaneous native writes competing with video decode.
|
||||
*/
|
||||
MediaCache.prototype.sync = function (items, urlFor) {
|
||||
if (this.busy) return Promise.resolve();
|
||||
this.busy = true;
|
||||
var self = this;
|
||||
|
||||
try { this.prune(items); } catch (e) { /* pruning must never block fetching */ }
|
||||
var list = (items || []).filter(function (it) { return it && it.content_id && !it.remote_url; });
|
||||
|
||||
/*
|
||||
* One item at a time, and within an item, keep going while bytes are landing — exactly the
|
||||
* shape the other players use. Serialised on purpose: three concurrent transfers on a link that
|
||||
* cannot finish one produce three unfinished transfers instead of one finished one, and on a TV
|
||||
* they also compete with video decode for the same memory.
|
||||
*/
|
||||
function item(i) {
|
||||
if (i >= list.length) return Promise.resolve();
|
||||
var it = list[i];
|
||||
var guard = 0;
|
||||
function step() {
|
||||
// Bounded, so one enormous asset cannot monopolise the pass and starve the rest of the
|
||||
// playlist; whatever is left resumes on the next sweep.
|
||||
if (guard++ >= 32) return Promise.resolve();
|
||||
return self.fetchStep(it.content_id, it.content_rev, urlFor(it)).then(function (verdict) {
|
||||
return verdict === 'progress' ? step() : null;
|
||||
});
|
||||
}
|
||||
return step().catch(function () { /* this item stalled; the rest of the playlist continues */ })
|
||||
.then(function () { return item(i + 1); });
|
||||
}
|
||||
|
||||
return item(0).catch(function () {}).then(function () { self.busy = false; });
|
||||
};
|
||||
|
||||
/* ------------------------------------------------------------------ *
|
||||
* The Tizen adapter. No decisions live here — only platform calls.
|
||||
* ------------------------------------------------------------------ */
|
||||
function tizenBackend() {
|
||||
var dir = null;
|
||||
try {
|
||||
// Synchronous resolve is deprecated in newer Web APIs but is what the widget runtime on the
|
||||
// shipped panels supports; the async form would force this whole module to be callback-based
|
||||
// for no behavioural gain.
|
||||
dir = tizen.filesystem.resolve('wgt-private', function (d) { dir = d; }, function () { dir = null; }, 'rw');
|
||||
} catch (e) { dir = null; }
|
||||
|
||||
function fileFor(name, create) {
|
||||
if (!dir) return null;
|
||||
try { return dir.resolve(name); } catch (e) { /* not there yet */ }
|
||||
if (!create) return null;
|
||||
try { return dir.createFile(name); } catch (e) { return null; }
|
||||
}
|
||||
|
||||
return {
|
||||
available: function () { return !!dir; },
|
||||
loadIndex: function () {
|
||||
try { return JSON.parse(localStorage.getItem(INDEX_KEY) || '{}'); } catch (e) { return {}; }
|
||||
},
|
||||
saveIndex: function (idx) {
|
||||
try { localStorage.setItem(INDEX_KEY, JSON.stringify(idx)); } catch (e) { /* full */ }
|
||||
},
|
||||
httpRange: function (url, start, end, validator) {
|
||||
return new Promise(function (resolve) {
|
||||
var xhr = new XMLHttpRequest();
|
||||
xhr.open('GET', url, true);
|
||||
// Binary over responseText: the widget runtime on the shipped panels predates a reliable
|
||||
// arraybuffer path through this API, and x-user-defined keeps every byte addressable.
|
||||
xhr.overrideMimeType('text/plain; charset=x-user-defined');
|
||||
xhr.setRequestHeader('Range', 'bytes=' + start + '-' + end);
|
||||
if (validator) xhr.setRequestHeader('If-Range', validator);
|
||||
// A stalled chunk must give up rather than hold the slot forever — on a bad link a hung
|
||||
// request is indistinguishable from a dead one, and the next sweep resumes anyway.
|
||||
xhr.timeout = 60000;
|
||||
function done(res) { resolve(res); }
|
||||
xhr.onerror = function () { done(null); };
|
||||
xhr.ontimeout = function () { done(null); };
|
||||
xhr.onload = function () {
|
||||
var cr = xhr.getResponseHeader('Content-Range') || '';
|
||||
var m = /bytes\s+(\d+)-(\d+)\/(\d+)/.exec(cr);
|
||||
var body = null;
|
||||
if (xhr.status === 200 || xhr.status === 206) {
|
||||
var text = xhr.responseText || '';
|
||||
body = [];
|
||||
for (var i = 0; i < text.length; i++) body.push(text.charCodeAt(i) & 0xff);
|
||||
}
|
||||
done({
|
||||
status: xhr.status,
|
||||
start: m ? Number(m[1]) : 0,
|
||||
total: m ? Number(m[3]) : Number(xhr.getResponseHeader('Content-Length') || 0),
|
||||
validator: xhr.getResponseHeader('ETag') || xhr.getResponseHeader('Last-Modified') || null,
|
||||
body: body
|
||||
});
|
||||
};
|
||||
try { xhr.send(null); } catch (e) { done(null); }
|
||||
});
|
||||
},
|
||||
appendPart: function (contentId, body, offset) {
|
||||
var f = fileFor(contentId + '.part', true);
|
||||
if (!f) return 0;
|
||||
var written = 0;
|
||||
// 'a' append mode, so a resumed transfer adds to what is already there instead of
|
||||
// truncating it — which would make every attempt start from zero again.
|
||||
f.openStream(offset > 0 ? 'a' : 'w', function (stream) {
|
||||
try { stream.writeBytes(body); written = body.length; } finally { stream.close(); }
|
||||
}, function () { written = 0; });
|
||||
return written;
|
||||
},
|
||||
promotePart: function (contentId) {
|
||||
var part = fileFor(contentId + '.part', false);
|
||||
if (!part) return null;
|
||||
try {
|
||||
// Rename rather than copy: an atomic-enough swap, and a copy would need twice the space
|
||||
// for a large video on a panel that may not have it.
|
||||
part.moveTo(part.parent.fullPath + '/' + contentId, contentId, true, function () {}, function () {});
|
||||
} catch (e) { /* fall through and try to resolve it anyway */ }
|
||||
var whole = fileFor(contentId, false);
|
||||
if (!whole) return null;
|
||||
return { path: whole.fullPath, uri: whole.toURI() };
|
||||
},
|
||||
remove: function (contentId) {
|
||||
if (!dir) return;
|
||||
[contentId, contentId + '.part'].forEach(function (name) {
|
||||
try {
|
||||
var f = dir.resolve(name);
|
||||
if (f) dir.deleteFile(f.fullPath, function () {}, function () {});
|
||||
} catch (e) { /* not present */ }
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
MediaCache.tizenBackend = tizenBackend;
|
||||
|
||||
/*
|
||||
* The live instance, or null when the platform cannot give us persistent storage. Null is a
|
||||
* supported state everywhere it is used: the player falls back to streaming from the server,
|
||||
* which is what it did before this existed.
|
||||
*/
|
||||
MediaCache.create = function () {
|
||||
try {
|
||||
var backend = tizenBackend();
|
||||
if (!backend.available()) return null;
|
||||
return new MediaCache(backend);
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
root.MediaCache = MediaCache;
|
||||
if (typeof module === 'object' && module.exports) module.exports = MediaCache;
|
||||
})(typeof window !== 'undefined' ? window : this);
|
||||
|
|
@ -261,8 +261,15 @@ PlaylistPlayer.prototype.durationMs = function (item) {
|
|||
|
||||
PlaylistPlayer.prototype.contentUrl = function (item) {
|
||||
if (item.remote_url) return item.remote_url;
|
||||
if (item.content_id) return this.getBase() + '/api/content/' + item.content_id + '/file';
|
||||
return null;
|
||||
if (!item.content_id) return null;
|
||||
// A cached copy is preferred whenever we hold the revision this item asks for. That is what keeps
|
||||
// the screen alive through an outage — and the revision check is what stops it keeping the screen
|
||||
// alive with the WRONG asset after somebody replaced it in the dashboard.
|
||||
var local = window.MediaCache && window.__stMediaCache
|
||||
&& window.__stMediaCache.localUrl(item.content_id, item.content_rev);
|
||||
if (local) return local;
|
||||
return this.getBase() + '/api/content/' + item.content_id + '/file'
|
||||
+ (item.content_rev ? '?rev=' + encodeURIComponent(item.content_rev) : '');
|
||||
};
|
||||
|
||||
PlaylistPlayer.prototype.advance = function () {
|
||||
|
|
@ -1064,8 +1071,15 @@ ZoneRenderer.prototype.durationMs = function (item) {
|
|||
|
||||
ZoneRenderer.prototype.contentUrl = function (item) {
|
||||
if (item.remote_url) return item.remote_url;
|
||||
if (item.content_id) return this.getBase() + '/api/content/' + item.content_id + '/file';
|
||||
return null;
|
||||
if (!item.content_id) return null;
|
||||
// A cached copy is preferred whenever we hold the revision this item asks for. That is what keeps
|
||||
// the screen alive through an outage — and the revision check is what stops it keeping the screen
|
||||
// alive with the WRONG asset after somebody replaced it in the dashboard.
|
||||
var local = window.MediaCache && window.__stMediaCache
|
||||
&& window.__stMediaCache.localUrl(item.content_id, item.content_rev);
|
||||
if (local) return local;
|
||||
return this.getBase() + '/api/content/' + item.content_id + '/file'
|
||||
+ (item.content_rev ? '?rev=' + encodeURIComponent(item.content_rev) : '');
|
||||
};
|
||||
|
||||
ZoneRenderer.prototype.showItem = function (zone, list, index) {
|
||||
|
|
|
|||
Loading…
Reference in a new issue