Archive/screentinker
1
0
Fork 0
mirror of https://github.com/screentinker/screentinker.git synced 2026-08-16 23:33:10 -06:00
Code Issues Actions Packages Projects Releases Wiki Activity

Install statistics: send on opt-in, name a blocked firewall, and add an operator collector

Browse source 
Three changes, all about the same failure: sharing appears to be on while nothing
actually arrives.

SEND ON OPT-IN. Turning sharing on now reports immediately instead of waiting for
the next daily tick. Two reasons: the operator is standing right there, and
"nothing has been sent" for the next 24h reads as broken at exactly the moment
someone is checking whether it works. It also means an egress-filtered network
fails HERE, where we can name the host to unblock, rather than silently tonight
where nobody is watching.

NAME THE FAILURE. Failed attempts are now recorded separately from successes, so
Settings can say which address did not answer and why, instead of showing an empty
"nothing sent yet". A blocked outbound connection is the normal failure on a
self-hosted box and is otherwise completely invisible — the operator cannot tell a
firewall from a broken feature. A later success clears the complaint, so a stale
warning never outlives the problem it describes. Docs gained a section on it, and
the UI states plainly that nothing needs opening inbound.

OPERATOR COLLECTOR, ADDITIVE. TELEMETRY_EXTRA_ENDPOINT lets an operator post the
same three fields to their own collector.

The naming is the point. It replaces TELEMETRY_ENDPOINT, which was a true override
— and an override is the wrong shape here, because a variable called "endpoint"
that silently redirected the report someone agreed to SHARE would make the opt-in
mean something other than what the UI says. Our address is hard-wired and not
overridable; theirs is explicitly additional and named so it cannot be mistaken for
a replacement. Settings lists every destination a report goes to.

The operator collector is independent of the sharing switch, because it is their
server posting to their host and our opt-in has no business gating it. So an
operator who wants internal fleet numbers with nothing leaving for us sets it and
leaves sharing off — supported on purpose, and tested.

Destinations are attempted separately: one unreachable collector must not cost the
other its report.

1662/1662 pass. Tests pin the properties that matter: an operator collector never
replaces the shared report, sharing-off still sends nothing to us whatever else is
configured, one dead destination does not stop the other, and a failure records the
address actually tried.
This commit is contained in:
ScreenTinker 2026-08-13 18:32:27 -05:00
parent 13534d9b61
commit d87a53a6be
5 changed files with 281 additions and 25 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

39
docs/telemetry.md
View file

@ -50,8 +50,43 @@ remembered, so declining is permanent and you will not be asked again after an u
To change your mind at any time: **Settings → Install statistics**.
Reports are sent at most once a day. Nothing is queued or retried — if your server is offline or
the request fails, that day is simply skipped.
Reports are sent 5 minutes after the server starts, then once a day while it keeps running.
Nothing is queued or retried — if your server is offline or the request fails, that attempt is
simply skipped.
## If your outbound traffic is filtered
Reports are an ordinary HTTPS `POST` from your server to:
```
https://stats.screentinker.com/api/telemetry/report
```
Many self-hosted servers sit on networks that block outbound connections by default. **If yours
does, that address has to be allowed or the reports never arrive** — sharing will appear to be on
while nothing reaches us.
You do not have to guess whether that is happening. Turning sharing on sends a report immediately,
so a blocked connection is reported there and then, and **Settings → Install statistics** names the
failure and the address to allow.
Nothing needs to be opened *inbound*. This is an outbound connection from your server only.
## Keeping your own copy
If you want these numbers for your own fleet, set `TELEMETRY_EXTRA_ENDPOINT` to your own collector.
Your server then posts the same three fields there as well.
Two things to be clear about, because the naming is deliberate:
- **It is additional, not a redirect.** Setting it does not stop the shared report going to
ScreenTinker — that is why it is called `EXTRA` rather than `ENDPOINT`. Settings lists every
destination a report goes to, so what is configured is always visible.
- **It is independent of the sharing switch.** Your collector receives reports whether sharing is
on or off, because that is your server posting to your host. **If you want your own statistics
and nothing sent to us, set it and leave sharing off** — that combination is supported on purpose.
Each destination is attempted separately, so one being unreachable never stops the other.
## Why we ask

38
frontend/js/views/settings.js
View file

@ -570,7 +570,17 @@ export async function render(container) {
const on = info.state === 'on';
const sent = info.last_report
? `Last sent ${new Date(info.last_report.at * 1000).toLocaleString()}.`
: 'Nothing has been sent.';
: 'Nothing has been sent yet.';
// A blocked outbound connection is the normal failure on a self-hosted box, and it is
// otherwise invisible — the operator just sees nothing arriving. Name the failure and the
// host, so the fix is "allow this in the firewall" rather than "guess".
const failed = on && info.last_error;
const why = failed
? ({ network: 'the connection was refused or the address did not resolve',
timeout: 'the connection timed out' }[info.last_error.reason]
|| `the server replied ${esc(info.last_error.reason)}`)
: '';
box.innerHTML = `
<p style="color:var(--text-muted);font-size:13px;margin-bottom:12px">
@ -585,6 +595,24 @@ export async function render(container) {
Everything that would be sent, in full:
</p>
<pre style="background:var(--bg-input,rgba(0,0,0,.2));padding:10px;border-radius:var(--radius);font-size:12px;overflow-x:auto;margin-bottom:8px">${esc(JSON.stringify(info.payload, null, 2))}</pre>
<p style="color:var(--text-muted);font-size:12px;margin-bottom:${info.extra_endpoint ? '4' : '8'}px">
${on ? 'Sent once a day to' : 'When enabled, sent once a day to'}
<code style="font-size:11px">${esc(info.endpoint || '')}</code>. If this server's outbound
traffic is filtered, that address has to be allowed or the reports never arrive.
</p>
${info.extra_endpoint ? `
<p style="color:var(--text-muted);font-size:12px;margin-bottom:8px">
A second copy also goes to your own collector at
<code style="font-size:11px">${esc(info.extra_endpoint)}</code>, configured on this server
with <code style="font-size:11px">TELEMETRY_EXTRA_ENDPOINT</code>. That is in addition to
the above, not instead of it — turn the switch off if you want your own statistics without
sharing.
</p>` : ''}
${failed ? `
<p style="font-size:12px;color:var(--danger);margin-bottom:8px">
The last attempt (${esc(new Date(info.last_error.at * 1000).toLocaleString())}) did not get
through — ${why}. Check that outbound HTTPS to that address is permitted.
</p>` : ''}
<p style="color:var(--text-muted);font-size:12px">
No names, addresses, content, or user details. The ID is random and identifies the install
only so repeat reports aren't counted twice. ${esc(sent)}
@ -594,8 +622,12 @@ export async function render(container) {
document.getElementById('telemetryToggle')?.addEventListener('change', async (e) => {
const enabled = e.target.checked;
try {
await api.adminSetTelemetry(enabled);
showToast(enabled ? 'Sharing install statistics — thank you' : 'Install statistics off', 'success');
// Turning it on sends immediately, so a blocked firewall is reported here and now rather
// than failing quietly tonight — say so plainly instead of a cheerful success toast.
const r = await api.adminSetTelemetry(enabled);
if (!enabled) showToast('Install statistics off', 'success');
else if (r.first_report && r.first_report.sent) showToast('Shared — thank you', 'success');
else showToast('Saved, but the first report did not get through — see below', 'error');
loadTelemetry();
} catch {
e.target.checked = !enabled;

94
server/lib/telemetry.js
View file

@ -35,9 +35,24 @@ const appSettings = require('./app-settings');
const KEY_ID = 'telemetry_instance_id';
const KEY_ENABLED = 'telemetry_enabled'; // unset = never asked
const KEY_LAST = 'telemetry_last_report';
const KEY_LAST = 'telemetry_last_report'; // last SUCCESSFUL send
const KEY_LAST_ERROR = 'telemetry_last_error';// last FAILED attempt — see getLastError
const DEFAULT_ENDPOINT = 'https://stats.screentinker.com/api/telemetry/report';
/*
* Where reports go. TWO independent destinations, deliberately:
*
* SCREENTINKER_ENDPOINT hard-wired, and reached only when the operator has switched sharing on.
* Not overridable — an "override" that silently redirected the shared
* report would make the opt-in mean something different from what it says.
*
* TELEMETRY_EXTRA_ENDPOINT an operator's OWN collector, for their own fleet numbers. Additional,
* never a replacement, and named so it cannot be mistaken for one. It is
* sent independently of the sharing toggle: it is their server posting to
* their host, so our opt-in has no business gating it. An operator who
* wants internal statistics and nothing leaving for us sets this and
* leaves sharing off — that combination is supported on purpose.
*/
const SCREENTINKER_ENDPOINT = 'https://stats.screentinker.com/api/telemetry/report';
const REPORT_INTERVAL_MS = 24 * 60 * 60 * 1000; // daily; this is a count, not a metric
const FIRST_REPORT_DELAY_MS = 5 * 60 * 1000; // let boot settle before any outbound call
@ -88,6 +103,24 @@ function countScreens(db) {
}
}
/* The address an operator may need to allowlist for the shared report. Hard-wired. */
function endpoint() { return SCREENTINKER_ENDPOINT; }
/* The operator's own collector, if they configured one. Null when they have not. */
function extraEndpoint() { return process.env.TELEMETRY_EXTRA_ENDPOINT || null; }
/*
* Everywhere this report is going, right now, and why — so the UI can list every destination
* rather than implying there is only one. Sharing gates OUR endpoint alone.
*/
function destinations() {
const out = [];
if (state() === 'on') out.push({ url: endpoint(), kind: 'screentinker' });
const extra = extraEndpoint();
if (extra) out.push({ url: extra, kind: 'extra' });
return out;
}
/* What was last sent, and when. Surfaced in Settings so an operator can check rather than trust. */
function getLastReport() {
const raw = appSettings.get(KEY_LAST, null);
@ -95,30 +128,65 @@ function getLastReport() {
try { return JSON.parse(raw); } catch (_) { return null; }
}
/*
* The last FAILED attempt, kept separately from the last success.
*
* A self-hosted server frequently sits behind egress filtering, so "enabled but nothing arrives"
* is the normal failure and it is otherwise completely silent — the operator sees "nothing has
* been sent" and has no way to tell a blocked firewall from a broken feature. Recording the
* failure lets the UI name the host that needs unblocking instead.
*/
function getLastError() {
const raw = appSettings.get(KEY_LAST_ERROR, null);
if (!raw) return null; // '' is how a success clears it
try { return JSON.parse(raw); } catch (_) { return null; }
}
/*
* Send one report. Returns {sent:false, reason} rather than throwing — a stats call must never be
* able to affect the running server, so every failure path here is quiet and local.
*/
async function report(db, { endpoint = process.env.TELEMETRY_ENDPOINT || DEFAULT_ENDPOINT } = {}) {
if (state() !== 'on') return { sent: false, reason: 'not_enabled' };
const body = payload(db);
async function postTo(url, body) {
try {
const res = await fetch(endpoint, {
const res = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
signal: AbortSignal.timeout(10000),
});
if (!res.ok) return { sent: false, reason: `http_${res.status}`, body };
appSettings.set(KEY_LAST, JSON.stringify({ at: Math.floor(Date.now() / 1000), body }));
return { sent: true, body };
return res.ok ? { sent: true } : { sent: false, reason: `http_${res.status}` };
} catch (err) {
// Offline, DNS failure, blocked egress — all normal for a self-hosted box, none of them news.
return { sent: false, reason: err && err.name === 'TimeoutError' ? 'timeout' : 'network', body };
// Offline, DNS failure, blocked egress — all normal for a self-hosted box, none of them news
// in the log, but all worth surfacing in the UI so the operator can act on it.
return { sent: false, reason: err && err.name === 'TimeoutError' ? 'timeout' : 'network' };
}
}
async function report(db, { urls = null } = {}) {
// `urls` is a test seam. Normal callers get destinations() — sharing gates ours, an operator's
// own collector is independent of it.
const targets = urls || destinations();
if (!targets.length) return { sent: false, reason: 'not_enabled', results: [] };
const now = () => Math.floor(Date.now() / 1000);
const body = payload(db);
// Every destination is attempted, independently. One unreachable collector must not stop the
// other from receiving — a blocked corporate firewall on their host should not cost us the
// shared count, and our endpoint being down should not cost them their own fleet numbers.
const results = [];
for (const t of targets) results.push({ ...t, ...(await postTo(t.url, body)) });
const failed = results.filter(r => !r.sent);
if (results.some(r => r.sent)) appSettings.set(KEY_LAST, JSON.stringify({ at: now(), body, results }));
// Keep only a LIVE complaint: record what is still failing, and clear it once nothing is.
appSettings.set(KEY_LAST_ERROR, failed.length
? JSON.stringify({ at: now(), reason: failed[0].reason, url: failed[0].url, failed })
: '');
return { sent: failed.length === 0, results, body, reason: failed[0]?.reason };
}
function start(db) {
if (timer) return;
const tick = () => { report(db).catch(() => {}); };
@ -129,4 +197,4 @@ function start(db) {
function stop() { if (timer) { clearInterval(timer); timer = null; } }
module.exports = { instanceId, state, setEnabled, payload, report, getLastReport, start, stop };
module.exports = { instanceId, state, setEnabled, payload, report, endpoint, extraEndpoint, destinations, getLastReport, getLastError, start, stop };

25
server/routes/admin.js
View file

@ -436,17 +436,38 @@ router.get('/telemetry', requirePlatformAdmin, (req, res) => {
res.json({
state: telemetry.state(), // 'unasked' | 'on' | 'off'
payload: telemetry.payload(db), // what WOULD be sent, right now
endpoint: telemetry.endpoint(), // ours — the host an operator may need to allowlist
extra_endpoint: telemetry.extraEndpoint(),// their own collector, if configured
destinations: telemetry.destinations(), // everywhere it actually goes, right now
last_report: telemetry.getLastReport(), // what was actually sent, and when
last_error: telemetry.getLastError(), // why the last attempt failed, if it did
});
});
router.put('/telemetry', requirePlatformAdmin, (req, res) => {
router.put('/telemetry', requirePlatformAdmin, async (req, res) => {
// Both answers are recorded. Declining must persist as 'off' rather than staying 'unasked',
// or the prompt returns after every update — which is how telemetry earns its bad name.
const enabled = !!req.body.enabled;
const state = telemetry.setEnabled(enabled);
logActivity(req.user.id, 'admin_set_telemetry', `enabled: ${enabled}`, null, getClientIp(req), null);
res.json({ state, payload: telemetry.payload(db) });
// Send once, now, rather than waiting for the next daily tick. Two reasons: the operator is
// standing right here and "nothing has been sent" for the next 24h reads as broken, and an
// egress-filtered network fails HERE where we can name the host to unblock — instead of
// failing silently tonight where nobody is watching.
let first = null;
if (enabled) first = await telemetry.report(db);
res.json({
state,
payload: telemetry.payload(db),
endpoint: telemetry.endpoint(),
extra_endpoint: telemetry.extraEndpoint(),
destinations: telemetry.destinations(),
first_report: first && { sent: first.sent, reason: first.reason || null },
last_report: telemetry.getLastReport(),
last_error: telemetry.getLastError(),
});
});
// ===================== Version update indicator =====================

110
server/test/telemetry.test.js
View file

@ -32,7 +32,7 @@ test('an install that has not been asked reports nothing', async () => {
const spy = mock.method(globalThis, 'fetch', async () => { throw new Error('must not be called'); });
try {
const r = await telemetry.report(db);
assert.deepEqual(r, { sent: false, reason: 'not_enabled' });
assert.equal(r.sent, false); assert.equal(r.reason, 'not_enabled');
assert.equal(spy.mock.callCount(), 0, 'no outbound request may be made before consent');
} finally { spy.mock.restore(); }
});
@ -50,7 +50,7 @@ test('a declined install still reports nothing', async () => {
telemetry.setEnabled(false);
const spy = mock.method(globalThis, 'fetch', async () => { throw new Error('must not be called'); });
try {
assert.deepEqual(await telemetry.report(db), { sent: false, reason: 'not_enabled' });
const d = await telemetry.report(db); assert.equal(d.sent, false); assert.equal(d.reason, 'not_enabled');
assert.equal(spy.mock.callCount(), 0);
} finally { spy.mock.restore(); }
});
@ -96,7 +96,7 @@ test('when enabled it sends exactly the payload, and records what it sent', asyn
return { ok: true, status: 200 };
});
try {
const r = await telemetry.report(db, { endpoint: 'https://example.test/report' });
const r = await telemetry.report(db, { urls: [{ url: 'https://example.test/report', kind: 'screentinker' }] });
assert.equal(r.sent, true);
assert.equal(seen.method, 'POST');
assert.equal(seen.url, 'https://example.test/report');
@ -110,12 +110,112 @@ test('when enabled it sends exactly the payload, and records what it sent', asyn
} finally { spy.mock.restore(); }
});
test('a blocked outbound connection is recorded, with the address that was blocked', async () => {
// Egress filtering is the normal failure on a self-hosted box and is otherwise invisible: the
// operator sees nothing arriving and cannot tell a firewall from a broken feature. The UI can
// only name the host to allowlist if the failure is recorded here.
reset();
telemetry.setEnabled(true);
const spy = mock.method(globalThis, 'fetch', async () => { throw new Error('ECONNREFUSED'); });
try {
await telemetry.report(db, { urls: [{ url: 'https://stats.example.test/report', kind: 'screentinker' }] });
const err = telemetry.getLastError();
assert.ok(err, 'a failed attempt must be recorded, or the operator has nothing to act on');
assert.equal(err.reason, 'network');
assert.equal(err.url, 'https://stats.example.test/report', 'must record the address actually tried');
assert.equal(typeof err.at, 'number');
} finally { spy.mock.restore(); }
});
test('a later success clears the stale failure', async () => {
reset();
telemetry.setEnabled(true);
const bad = mock.method(globalThis, 'fetch', async () => { throw new Error('ECONNREFUSED'); });
try { await telemetry.report(db, { urls: [{ url: 'https://example.test/report', kind: 'screentinker' }] }); } finally { bad.mock.restore(); }
assert.ok(telemetry.getLastError(), 'precondition: a failure was recorded');
const good = mock.method(globalThis, 'fetch', async () => ({ ok: true, status: 200 }));
try {
await telemetry.report(db, { urls: [{ url: 'https://example.test/report', kind: 'screentinker' }] });
assert.equal(telemetry.getLastError(), null,
'a stale firewall warning must not outlive the problem it describes');
} finally { good.mock.restore(); }
});
test('an operator collector is ADDITIONAL — it never replaces the shared report', async () => {
// The whole point of naming it EXTRA rather than ENDPOINT: configuring your own collector must
// not silently redirect the report the operator agreed to share. If this ever becomes a
// redirect, the opt-in stops meaning what the UI says it means.
reset();
telemetry.setEnabled(true);
const original = process.env.TELEMETRY_EXTRA_ENDPOINT;
process.env.TELEMETRY_EXTRA_ENDPOINT = 'https://mine.example.test/collect';
try {
const dests = telemetry.destinations();
assert.equal(dests.length, 2, 'sharing on + own collector = both, never one');
assert.deepEqual(dests.map(d => d.kind).sort(), ['extra', 'screentinker']);
const hits = [];
const spy = mock.method(globalThis, 'fetch', async (url) => { hits.push(url); return { ok: true, status: 200 }; });
try {
await telemetry.report(db);
assert.equal(hits.length, 2, 'both destinations must receive the report');
assert.ok(hits.includes('https://mine.example.test/collect'));
assert.ok(hits.some(u => u.includes('screentinker.com')), 'the shared report must still be sent');
} finally { spy.mock.restore(); }
} finally {
if (original === undefined) delete process.env.TELEMETRY_EXTRA_ENDPOINT;
else process.env.TELEMETRY_EXTRA_ENDPOINT = original;
}
});
test('an operator can keep their own statistics while sharing nothing with us', async () => {
// Someone who wants internal fleet numbers but nothing leaving for us sets their own collector
// and leaves sharing off. Supported on purpose: it is their server posting to their host.
reset();
telemetry.setEnabled(false);
const original = process.env.TELEMETRY_EXTRA_ENDPOINT;
process.env.TELEMETRY_EXTRA_ENDPOINT = 'https://mine.example.test/collect';
try {
const hits = [];
const spy = mock.method(globalThis, 'fetch', async (url) => { hits.push(url); return { ok: true, status: 200 }; });
try {
await telemetry.report(db);
assert.deepEqual(hits, ['https://mine.example.test/collect']);
assert.ok(!hits.some(u => u.includes('screentinker.com')),
'sharing is off — nothing may reach us, whatever else is configured');
} finally { spy.mock.restore(); }
} finally {
if (original === undefined) delete process.env.TELEMETRY_EXTRA_ENDPOINT;
else process.env.TELEMETRY_EXTRA_ENDPOINT = original;
}
});
test('one unreachable destination does not stop the other', async () => {
reset();
telemetry.setEnabled(true);
const spy = mock.method(globalThis, 'fetch', async (url) => {
if (url.includes('broken')) throw new Error('ECONNREFUSED');
return { ok: true, status: 200 };
});
try {
const r = await telemetry.report(db, { urls: [
{ url: 'https://broken.example.test/a', kind: 'extra' },
{ url: 'https://working.example.test/b', kind: 'screentinker' },
] });
assert.equal(r.results.filter(x => x.sent).length, 1, 'the reachable one still receives it');
assert.equal(r.results.filter(x => !x.sent).length, 1);
assert.equal(telemetry.getLastError().url, 'https://broken.example.test/a',
'the failure names the destination that actually failed');
} finally { spy.mock.restore(); }
});
test('a failed send is quiet and local — never throws, never records a phantom report', async () => {
reset();
telemetry.setEnabled(true);
const spy = mock.method(globalThis, 'fetch', async () => { throw new Error('ECONNREFUSED'); });
try {
const r = await telemetry.report(db, { endpoint: 'https://example.test/report' });
const r = await telemetry.report(db, { urls: [{ url: 'https://example.test/report', kind: 'screentinker' }] });
assert.equal(r.sent, false);
assert.equal(r.reason, 'network');
assert.equal(telemetry.getLastReport(), null, 'a failed send must not look like a successful one');
@ -124,7 +224,7 @@ test('a failed send is quiet and local — never throws, never records a phantom
// An HTTP error is likewise not a success.
const spy2 = mock.method(globalThis, 'fetch', async () => ({ ok: false, status: 503 }));
try {
const r = await telemetry.report(db, { endpoint: 'https://example.test/report' });
const r = await telemetry.report(db, { urls: [{ url: 'https://example.test/report', kind: 'screentinker' }] });
assert.equal(r.sent, false);
assert.equal(r.reason, 'http_503');
assert.equal(telemetry.getLastReport(), null);