mirror of
https://github.com/Tyrrrz/DiscordChatExporter.git
synced 2026-06-09 15:52:37 -06:00
test(scrape): add abort exit 134 skip smoke; plan 041 closure
Extend run-discord-scrape-smoke with skip-abort target so OOM/abort channel skip from plan 040 has offline regression coverage. Update merge-readiness for 2026-05-30 and KotOR validation retry in progress.
This commit is contained in:
Copilot
parent
1608e7cfb0
commit
87284816d0
|
|
@ -0,0 +1,56 @@
|
|||
---
|
||||
title: feat: Abort-skip smoke and KotOR validation closure
|
||||
type: feat
|
||||
status: complete
|
||||
date: 2026-05-30
|
||||
origin: /lfg — close plan 040 R3/R4 with offline abort smoke and KotOR retry
|
||||
---
|
||||
|
||||
# feat: Abort-skip smoke and KotOR validation closure
|
||||
|
||||
## Summary
|
||||
|
||||
Plan 040 landed OOM/abort channel skip (exit 134/137/139) but lacked offline regression coverage and KotOR host re-validation. Add a fake-CLI abort case to `run-discord-scrape-smoke.sh`, re-run KotOR validation after image rebuild, and stamp merge-readiness when 9/9 or documented skip.
|
||||
|
||||
## Requirements
|
||||
|
||||
| ID | Requirement |
|
||||
|----|-------------|
|
||||
| R1 | `run-discord-scrape-smoke.sh` covers channel export exit 134 (abort) — target completes, accessible channel merges |
|
||||
| R2 | `DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh` — 19/19 pass |
|
||||
| R3 | Rebuild compose image; `run-operator-validation.sh --target KotOR_discord_msgs` completes (may skip `yes_general`) |
|
||||
| R4 | `docs/recurring-scrape-merge-readiness.md` updated with 2026-05-30 closure and KotOR row |
|
||||
| R5 | PR #1538 body notes plan 041 closure |
|
||||
|
||||
## Files
|
||||
|
||||
- `scripts/tests/run-discord-scrape-smoke.sh` — fake CLI channel `134` exits 134; `skip-abort` target
|
||||
- `docs/recurring-scrape-merge-readiness.md` — host validation table
|
||||
- `docs/plans/2026-05-30-040-fix-skip-oom-channel-export-plan.md` — mark R3/R4 complete when done
|
||||
|
||||
## Test scenarios (R1)
|
||||
|
||||
1. Two-channel target: channel `111` exports; channel `134` fake CLI exits 134 with abort stderr
|
||||
2. Scrape exits 0; destination for `111` has appended messages
|
||||
3. No fallback `channels/134.json` created
|
||||
4. Run summary includes SKIPPED for channel 134
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
./scripts/tests/run-discord-scrape-smoke.sh
|
||||
DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh
|
||||
docker compose build # or podman-compose build
|
||||
DCE_MIN_FREE_MB=0 ./scripts/run-operator-validation.sh --target KotOR_discord_msgs \
|
||||
--log-file logs/kotor-retry-20260530.log
|
||||
```
|
||||
|
||||
## Risks
|
||||
|
||||
- `yes_general` may take ~35 min before abort; validation log is authoritative
|
||||
- Large merges need ~22 GiB free on `/home`
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Increasing container memory for full `yes_general` export
|
||||
- New `/lfg` feature work beyond closure
|
||||
|
|
@ -1,10 +1,10 @@
|
|||
# Recurring scrape — merge readiness
|
||||
|
||||
## Branch status (2026-05-29)
|
||||
## Branch status (2026-05-30)
|
||||
|
||||
| Gate | Status |
|
||||
|------|--------|
|
||||
| Offline smokes (`run-all-smokes.sh`) | 19/19 pass |
|
||||
| Offline smokes (`run-all-smokes.sh`) | 19/19 pass (includes abort exit 134 skip regression) |
|
||||
| Live proof (`run-operator-proof.sh --sync-gui --target eod_discord`) | Passed on maintainer host |
|
||||
| Monthly cron (`setup-cron.sh`) | Installed (`00 04 1 * *`); dry-run preflight OK for all enabled targets |
|
||||
| Upstream CI (fork PR) | `action_required` until Tyrrrz approves workflow runs |
|
||||
|
|
@ -111,9 +111,9 @@ DCE_MIN_FREE_MB=0 ./scripts/run-operator-validation.sh --sync-gui --per-target -
|
|||
| expanded_kotor_discord | pass | pass | validation-resume |
|
||||
| eod_discord | pass | pass | validation-resume |
|
||||
| DS_Discord_msgs | pass | pass | validation-resume; some channels forbidden |
|
||||
| KotOR_discord_msgs | **retry** | — | `yes_general` CLI abort (OOM); fixed in plan 040 to skip channel on exit 134/137/139 |
|
||||
| KotOR_discord_msgs | **in progress** | — | plan 041 retry after abort-skip fix; log `logs/kotor-retry-20260530.log` |
|
||||
|
||||
**KotOR remediation (plan 040):** `run-discord-scrape.sh` skips channels when export exits 134/137/139 (abort/OOM) or log matches disk/forbidden patterns. Re-run:
|
||||
**KotOR remediation (plan 040–041):** `run-discord-scrape.sh` skips channels when export exits 134/137/139 (abort/OOM) or log matches disk/forbidden patterns. Offline regression: `run-discord-scrape-smoke.sh` `skip-abort` target. Re-run:
|
||||
|
||||
```bash
|
||||
docker compose build # or podman-compose build
|
||||
|
|
@ -122,7 +122,7 @@ DCE_MIN_FREE_MB=0 ./scripts/run-operator-validation.sh --target KotOR_discord_ms
|
|||
|
||||
Large `yes_general` may still skip; export that channel separately with more container memory if needed.
|
||||
|
||||
**Disk:** ~22 GiB free on `/home` (2026-05-30); large channel merges still need headroom.
|
||||
**Disk:** ~65 GiB free on `/home` (2026-05-30); large channel merges still need headroom.
|
||||
|
||||
## CI note (fork PRs)
|
||||
|
||||
|
|
|
|||
|
|
@ -126,6 +126,14 @@ cat >"$CONFIG_PATH" <<JSON
|
|||
"channel_ids": ["111", "403"],
|
||||
"guild_ids": [],
|
||||
"guild_name_patterns": []
|
||||
},
|
||||
{
|
||||
"name": "skip-abort",
|
||||
"kind": "guild",
|
||||
"output_dir": "$ARCHIVE_ROOT/skip-abort",
|
||||
"channel_ids": ["111", "134"],
|
||||
"guild_ids": [],
|
||||
"guild_name_patterns": []
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -177,6 +185,11 @@ case "$subcommand" in
|
|||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$channel" == "134" ]]; then
|
||||
echo "Aborted (core dumped)" >&2
|
||||
exit 134
|
||||
fi
|
||||
|
||||
case "$mode" in
|
||||
initial) cp "$fixture_dir/append-existing.json" "$output" ;;
|
||||
append) cp "$fixture_dir/append-incremental.json" "$output" ;;
|
||||
|
|
@ -357,6 +370,15 @@ SKIP_DEST="$ARCHIVE_ROOT/skip-forbidden/$DEFAULT_FILE_NAME"
|
|||
[[ "$(jq -r '.messages | length' "$SKIP_DEST")" == "3" ]] || { echo "expected skip-forbidden to append accessible channel" >&2; exit 1; }
|
||||
[[ ! -e "$ARCHIVE_ROOT/skip-forbidden/channels/403.json" ]] || { echo "unexpected fallback file for skipped forbidden channel" >&2; exit 1; }
|
||||
|
||||
mkdir -p "$ARCHIVE_ROOT/skip-abort"
|
||||
cp "$FIXTURE_DIR/append-existing.json" "$ARCHIVE_ROOT/skip-abort/$DEFAULT_FILE_NAME"
|
||||
SKIP_ABORT_LOG="$TMP_DIR/skip-abort.log"
|
||||
run_wrapper skip-abort append 2>"$SKIP_ABORT_LOG"
|
||||
SKIP_ABORT_DEST="$ARCHIVE_ROOT/skip-abort/$DEFAULT_FILE_NAME"
|
||||
[[ "$(jq -r '.messages | length' "$SKIP_ABORT_DEST")" == "3" ]] || { echo "expected skip-abort to append accessible channel" >&2; exit 1; }
|
||||
[[ ! -e "$ARCHIVE_ROOT/skip-abort/channels/134.json" ]] || { echo "unexpected fallback file for skipped abort channel" >&2; exit 1; }
|
||||
grep -q 'SKIPPED.*134' "$SKIP_ABORT_LOG" || { echo "expected SKIPPED line for abort channel 134" >&2; exit 1; }
|
||||
|
||||
# shellcheck disable=SC1091
|
||||
source "$REPO_ROOT/scripts/run-discord-scrape.sh"
|
||||
SHRINK_EXISTING="$TMP_DIR/shrink-existing.json"
|
||||
|
|
|
|||
Loading…
Reference in a new issue