mirror of
https://github.com/Tyrrrz/DiscordChatExporter.git
synced 2026-06-10 00:02:37 -06:00
5820f67caf
Use tee in run_subcommand_with_retry so validation logs show live export progress instead of buffering until compose exits. Add streaming smoke assertion and skip scrape lock in documents-scrape smoke when prove runs against the real repo root during parallel validation.
42 lines
1.6 KiB
Markdown
42 lines
1.6 KiB
Markdown
---
|
|
title: "feat: Stream container scrape output during host runs"
|
|
type: feat
|
|
status: complete
|
|
date: 2026-06-04
|
|
origin: /lfg — KotOR validation log frozen at ~83 lines while yes_general export ran for hours
|
|
---
|
|
|
|
# feat: Stream container scrape output during host runs
|
|
|
|
## Problem
|
|
|
|
`run-discord-scrape-host.sh` captures all container stdout/stderr into a temp file and only `cat`s it after the compose run exits. Long exports (e.g. KotOR `yes_general`) leave operator validation logs silent for hours even though the container is actively exporting.
|
|
|
|
## Requirements
|
|
|
|
| ID | Requirement |
|
|
|----|-------------|
|
|
| R1 | `run_subcommand_with_retry` streams compose output to stdout as it arrives while still capturing to the temp file for auth-failure detection |
|
|
| R2 | Preserve exit-code semantics and auth-retry behavior (pipefail + `PIPESTATUS[0]`) |
|
|
| R3 | Do not duplicate full output on success (tee replaces post-hoc `cat`) |
|
|
| R4 | Host smoke adds a `streaming` fake-docker mode proving first line appears before command completes |
|
|
| R5 | `run-all-smokes.sh` passes |
|
|
|
|
## Implementation
|
|
|
|
- **File:** `scripts/run-discord-scrape-host.sh` — replace `>"$output_file" 2>&1` + `cat` with `"${run_args[@]}" 2>&1 | tee "$output_file"` and check `${PIPESTATUS[0]}` in both initial and retry paths.
|
|
- **File:** `scripts/tests/run-discord-scrape-host-smoke.sh` — add streaming mode assertion.
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
./scripts/tests/run-discord-scrape-host-smoke.sh
|
|
DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh
|
|
```
|
|
|
|
## Out of scope
|
|
|
|
- yes_general catch-up completion
|
|
- Container memory limits
|
|
- Validation-level flock (host flock already exists)
|