Keep retrying an install for a working day, and flag a human straight away

Three attempts inside one hour, then a day of silence, was calibrated for the wrong
cost. The ~8.7MB re-download that throttle exists to prevent is already prevented by
the APK cache — downloadAndInstall reuses a previously verified file, so attempts
2..N pull no bytes. What actually blocks these installs is a confirm dialog waiting
for somebody to walk past, and giving up an hour in guarantees nobody has.

The cap is now 40, roughly a working day at the 30-minute cadence, before falling back
to the existing daily retry. Two things had to come with it, because raising the number
alone would have made things worse:

Telling the operator is now a SEPARATE threshold from giving up. It used to fire at
the cap, so a bare bump would have pushed "this panel needs attention" from about an
hour out to about twenty. It fires at ATTEMPTS_BEFORE_FLAGGING (3) instead, and
statusFor keys on the same threshold, so a device reports manual_update_required as
soon as a human is demonstrably needed and KEEPS reporting it while it retries.
Previously the status dropped back to 'pending' once the backoff window elapsed, so a
panel that needed hands looked healthy in between attempts.

PackageInstaller sessions are now abandoned before a new one is opened. Every attempt
stages a full copy of the APK via openWrite, and a session whose dialog is never
accepted holds onto it. At three that was a rounding error; at forty it would be
~350MB of staged installs on hardware without it to spare, and would eventually trip
the per-app session limit.

The warning text no longer promises a 24h backoff it is not about to take, and says
what would actually fix it — accept the prompt, or have the MDM delegate install
permission.

The three tests that broke encoded the old thresholds and were rewritten to the new
intent rather than retuned to pass.

android/app/src/main/java/com/remotedisplay/player/service/OtaThrottle.kt

@@ -13,7 +13,19 @@ package com.remotedisplay.player.service
  *  - the "entering backoff" signal fires on the crossing only (report-on-transition).
  */
 object OtaThrottle {
-    const val MAX_INSTALL_ATTEMPTS = 3
+    // Why this is 40 and not 3: an attempt is nearly free. The APK is downloaded and signature-
+    // verified ONCE and then reused from cache, so attempts 2..N pull no bytes — the ~8.7MB
+    // re-download this throttle was built to stop is already prevented by the cache, not by the
+    // cap. What actually blocks these installs is a confirm dialog waiting for a human, and a
+    // human may walk past at any hour. Three tries inside one hour, then a day of silence, gave up
+    // long before anyone had a chance; ~40 keeps trying across a working day (30-minute cadence
+    // ≈ 20 hours) before falling back to the daily retry.
+    const val MAX_INSTALL_ATTEMPTS = 40
+    // Telling the operator is a SEPARATE decision from giving up, and it has to stay early:
+    // flagging only at the cap would push "this panel needs attention" from ~1 hour out to ~20.
+    // After this many failed launches a human is demonstrably needed, so say so — and keep
+    // retrying anyway, because saying it costs nothing and stopping costs the update.
+    const val ATTEMPTS_BEFORE_FLAGGING = 3
     const val BACKOFF_MS = 24L * 60 * 60 * 1000
 
     /** Persisted OTA state for the version we are currently trying to install. */
@@ -50,9 +62,12 @@ object OtaThrottle {
     fun onInstallLaunched(state: State, now: Long): Pair<State, Boolean> {
         val attempts = state.attempts + 1
         var s = state.copy(attempts = attempts, lastAttemptAt = now)
-        val enteredBackoff = attempts >= MAX_INSTALL_ATTEMPTS && !s.backoffReported
-        if (enteredBackoff) s = s.copy(backoffReported = true)
-        return s to enteredBackoff
+        // Flag at ATTEMPTS_BEFORE_FLAGGING, not at the cap — the operator needs to know a human is
+        // required long before the device stops trying. Reported once (the latch is re-armed by a
+        // new target version or by a forced check, so a genuinely new situation is announced again).
+        val shouldFlag = attempts >= ATTEMPTS_BEFORE_FLAGGING && !s.backoffReported
+        if (shouldFlag) s = s.copy(backoffReported = true)
+        return s to shouldFlag
     }
 
     /**
@@ -100,7 +115,11 @@ object OtaThrottle {
      */
     fun statusFor(state: State, now: Long): String = when {
         state.targetVersion.isEmpty() -> "none"
-        state.attempts >= MAX_INSTALL_ATTEMPTS && now - state.lastAttemptAt < BACKOFF_MS -> "manual_update_required"
+        // Keyed on the FLAGGING threshold, not the cap: after this many launched installs failed to
+        // take, a human really is required, and that stays true while the device keeps retrying.
+        // Previously this also required being inside the backoff window, so a device that was still
+        // trying read as plain 'pending' and never surfaced.
+        state.attempts >= ATTEMPTS_BEFORE_FLAGGING -> "manual_update_required"
         else -> "pending"
     }
 }

android/app/src/main/java/com/remotedisplay/player/service/UpdateChecker.kt

@@ -267,7 +267,7 @@ class UpdateChecker(private val context: Context) {
                 "Force update: $latestVersion downloaded and verified, install launched — a confirm dialog must be accepted on the device unless an accessibility service does it")
         }
         if (enteredBackoff) {
-            report("warn", "Update $latestVersion available but not installing after ${afterLaunch.attempts} attempts — manual update required (backing off to one retry per ${OtaThrottle.BACKOFF_MS / 3_600_000L}h)")
+            report("warn", "Update $latestVersion downloaded and verified, but ${afterLaunch.attempts} install attempts have not completed — a human needs to accept the install prompt on this device (or the MDM needs to delegate install permission). Still retrying.")
             announceOtaStatus() // transition -> emits 'manual_update_required'
         }
     }
@@ -404,6 +404,17 @@ class UpdateChecker(private val context: Context) {
         try {
             if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
                 val installer = context.packageManager.packageInstaller
+                // Abandon our own leftover sessions first. Every attempt stages a FULL copy of the
+                // APK (~8.7MB) via openWrite, and a session whose confirm dialog is never accepted
+                // just sits there holding it. At three attempts that was a rounding error; at forty
+                // it would be ~350MB of staged installs on a panel nobody walks up to, on hardware
+                // that does not have it spare. Also keeps us clear of the per-app session limit,
+                // which would start throwing once enough accumulated.
+                try {
+                    for (s in installer.mySessions) {
+                        try { installer.abandonSession(s.sessionId) } catch (_: Throwable) { /* already gone */ }
+                    }
+                } catch (e: Throwable) { Log.w(TAG, "Session cleanup skipped: ${e.message}") }
                 val params = android.content.pm.PackageInstaller.SessionParams(
                     android.content.pm.PackageInstaller.SessionParams.MODE_FULL_INSTALL
                 )

android/app/src/test/java/com/remotedisplay/player/service/OtaBackoffCadenceTest.kt

@@ -38,22 +38,32 @@ class OtaBackoffCadenceTest {
         return attemptsAt
     }
 
-    @Test fun the_first_three_attempts_burn_fast_at_the_check_cadence() {
-        val at = simulate(hours = 2)
-        assertEquals(3, at.size)
-        // Back to back on consecutive 30-min checks, so the cap is reached within about an hour.
-        assertEquals(listOf(0L, CHECK_INTERVAL, CHECK_INTERVAL * 2), at)
+    @Test fun the_opening_burst_runs_at_the_check_cadence_until_the_cap() {
+        val MAX = OtaThrottle.MAX_INSTALL_ATTEMPTS
+        // Long enough to burn the whole budget: MAX attempts at one per check interval.
+        val at = simulate(hours = (MAX * CHECK_INTERVAL / 3_600_000L).toInt() + 2)
+        assertEquals(MAX, at.size)
+        // Back to back on consecutive checks — no artificial spacing before the cap.
+        assertEquals(listOf(0L, CHECK_INTERVAL, CHECK_INTERVAL * 2), at.take(3))
+        assertEquals(CHECK_INTERVAL * (MAX - 1), at.last())
     }
 
-    @Test fun THE_QUESTION_after_the_cap_it_is_ONE_attempt_per_24h_not_three() {
-        val at = simulate(hours = 24 * 3 + 2)   // three full days
-        // 3 in the opening burst, then one per 24h window.
-        assertEquals(6, at.size)
-        val afterBurst = at.drop(3)
-        assertEquals(3, afterBurst.size)
+    @Test fun the_burst_now_spans_a_working_day_not_an_hour() {
+        // The reason for raising the cap: a confirm dialog needs a human to walk past, and three
+        // tries inside one hour gave up long before anyone realistically would.
+        val spanMs = OtaThrottle.MAX_INSTALL_ATTEMPTS * CHECK_INTERVAL
+        assert(spanMs >= 12 * 60 * 60 * 1000L) { "burst should cover a working day, was ${spanMs / 3_600_000}h" }
+    }
+
+    @Test fun THE_QUESTION_after_the_cap_it_is_ONE_attempt_per_24h_not_a_fresh_budget() {
+        val MAX = OtaThrottle.MAX_INSTALL_ATTEMPTS
+        val burstHours = (MAX * CHECK_INTERVAL / 3_600_000L).toInt()
+        val at = simulate(hours = burstHours + 24 * 3 + 2)
+        val afterBurst = at.drop(MAX)
+        assertEquals(3, afterBurst.size)          // three further days -> three retries, not 3x3
         for (i in 1 until afterBurst.size) {
-            val gap = afterBurst[i] - afterBurst[i - 1]
-            assertEquals("post-cap retries must be 24h apart", OtaThrottle.BACKOFF_MS, gap)
+            assertEquals("post-cap retries must be 24h apart",
+                OtaThrottle.BACKOFF_MS, afterBurst[i] - afterBurst[i - 1])
         }
     }
 

android/app/src/test/java/com/remotedisplay/player/service/OtaThrottleTest.kt

@@ -80,19 +80,29 @@ class OtaThrottleTest {
         assertFalse(OtaThrottle.shouldClearOnUpToDate(OtaThrottle.State())) // nothing pending
     }
 
-    @Test fun statusForReflectsBackoffWindow() {
+    @Test fun statusForFlagsEarlyAndStaysFlaggedWhileStillRetrying() {
         val now = 10_000L
+        val FLAG = OtaThrottle.ATTEMPTS_BEFORE_FLAGGING
         // no target → none
         assertEquals("none", OtaThrottle.statusFor(OtaThrottle.State(), now))
-        // under the cap → pending
+        // below the flagging threshold → pending (it may still just work)
         assertEquals("pending", OtaThrottle.statusFor(
-            OtaThrottle.State(targetVersion = V, attempts = 1, lastAttemptAt = now), now))
-        // capped AND inside the window → manual update required
+            OtaThrottle.State(targetVersion = V, attempts = FLAG - 1, lastAttemptAt = now), now))
+        // at the threshold → a human is demonstrably needed, say so IMMEDIATELY. This is now far
+        // below the cap, so the dashboard flags in ~1h instead of waiting ~20h for the give-up.
         assertEquals("manual_update_required", OtaThrottle.statusFor(
-            OtaThrottle.State(targetVersion = V, attempts = MAX, lastAttemptAt = now), now + WINDOW - 1))
-        // capped but window elapsed (a retry is due) → pending, not stuck
-        assertEquals("pending", OtaThrottle.statusFor(
-            OtaThrottle.State(targetVersion = V, attempts = MAX, lastAttemptAt = now), now + WINDOW + 1))
+            OtaThrottle.State(targetVersion = V, attempts = FLAG, lastAttemptAt = now), now))
+        // ...and STAYS flagged while the device keeps retrying. It used to drop back to 'pending'
+        // once the backoff window elapsed, so a panel needing hands looked healthy between retries.
+        assertEquals("manual_update_required", OtaThrottle.statusFor(
+            OtaThrottle.State(targetVersion = V, attempts = FLAG, lastAttemptAt = now), now + WINDOW + 1))
+        assertEquals("manual_update_required", OtaThrottle.statusFor(
+            OtaThrottle.State(targetVersion = V, attempts = MAX, lastAttemptAt = now), now + WINDOW * 5))
+    }
+
+    @Test fun flagging_happens_far_before_giving_up() {
+        // The whole point of splitting the two thresholds.
+        assert(OtaThrottle.ATTEMPTS_BEFORE_FLAGGING < OtaThrottle.MAX_INSTALL_ATTEMPTS)
     }
 
     // ---- #166: managed stand-down (a foreign DPC owns installs on this panel) --------------------