mirror of https://github.com/Tyrrrz/DiscordChatExporter.git synced 2026-06-10 00:02:37 -06:00

Copilot 5820f67caf feat(host): stream container scrape output during long runs

Use tee in run_subcommand_with_retry so validation logs show live export
progress instead of buffering until compose exits. Add streaming smoke
assertion and skip scrape lock in documents-scrape smoke when prove runs
against the real repo root during parallel validation.

2026-06-03 06:13:58 -05:00

1.6 KiB

Raw Blame History

title	type	status	date	origin
feat: Stream container scrape output during host runs	feat	complete	2026-06-04	/lfg — KotOR validation log frozen at ~83 lines while yes_general export ran for hours

feat: Stream container scrape output during host runs

Problem

run-discord-scrape-host.sh captures all container stdout/stderr into a temp file and only cats it after the compose run exits. Long exports (e.g. KotOR yes_general) leave operator validation logs silent for hours even though the container is actively exporting.

Requirements

ID	Requirement
R1	`run_subcommand_with_retry` streams compose output to stdout as it arrives while still capturing to the temp file for auth-failure detection
R2	Preserve exit-code semantics and auth-retry behavior (pipefail + `PIPESTATUS[0]`)
R3	Do not duplicate full output on success (tee replaces post-hoc `cat`)
R4	Host smoke adds a `streaming` fake-docker mode proving first line appears before command completes
R5	`run-all-smokes.sh` passes

Implementation

File: scripts/run-discord-scrape-host.sh — replace >"$output_file" 2>&1 + cat with "${run_args[@]}" 2>&1 | tee "$output_file" and check ${PIPESTATUS[0]} in both initial and retry paths.
File: scripts/tests/run-discord-scrape-host-smoke.sh — add streaming mode assertion.

Verification

./scripts/tests/run-discord-scrape-host-smoke.sh
DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh

Out of scope

yes_general catch-up completion
Container memory limits
Validation-level flock (host flock already exists)

1.6 KiB Raw Blame History