fix(scrape): skip channels on OOM/abort export exit codes

Treat CLI exit 134/137/139 and abort/OOM log patterns as skippable
so KotOR yes_general core dump does not fail the entire target scrape.
This commit is contained in:
Copilot 2026-06-03 00:44:06 -05:00
parent bc1f727907
commit 1608e7cfb0
3 changed files with 46 additions and 3 deletions

View file

@ -0,0 +1,28 @@
---
title: fix: Skip OOM/aborted channel exports
type: fix
status: complete
date: 2026-05-30
origin: /lfg — KotOR yes_general CLI core dump should not fail entire target
---
# fix: Skip OOM/aborted channel exports
## Summary
`yes_general` export aborted with `Aborted (core dumped)` inside the container, failing the whole `KotOR_discord_msgs` target. Extend skippable export failures so OOM/abort/kill errors skip the channel and continue (like forbidden channels).
## Requirements
| ID | Requirement |
|----|-------------|
| R1 | `is_skippable_channel_export_failure` matches Aborted, core dumped, OOM, Killed |
| R2 | `run-discord-scrape-smoke.sh` still passes |
| R3 | Re-run `run-operator-validation.sh --target KotOR_discord_msgs` completes (may skip yes_general) |
| R4 | Update merge-readiness KotOR row; 19 smokes pass |
## Verification
- `./scripts/tests/run-discord-scrape-smoke.sh`
- `DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh`
- `logs/validation-resume-20260530.log` or new log for KotOR pass

View file

@ -111,9 +111,16 @@ DCE_MIN_FREE_MB=0 ./scripts/run-operator-validation.sh --sync-gui --per-target -
| expanded_kotor_discord | pass | pass | validation-resume | | expanded_kotor_discord | pass | pass | validation-resume |
| eod_discord | pass | pass | validation-resume | | eod_discord | pass | pass | validation-resume |
| DS_Discord_msgs | pass | pass | validation-resume; some channels forbidden | | DS_Discord_msgs | pass | pass | validation-resume; some channels forbidden |
| KotOR_discord_msgs | **fail** | — | channel `221726893064454144` (`yes_general`) failed mid-export (~11%); see log | | KotOR_discord_msgs | **retry** | — | `yes_general` CLI abort (OOM); fixed in plan 040 to skip channel on exit 134/137/139 |
**KotOR remediation:** ensure several GiB free on `/home`, run `./scripts/audit-archive-json.sh --target KotOR_discord_msgs`, salvage truncated JSON if needed, then `./scripts/run-operator-validation.sh --target KotOR_discord_msgs`. **KotOR remediation (plan 040):** `run-discord-scrape.sh` skips channels when export exits 134/137/139 (abort/OOM) or log matches disk/forbidden patterns. Re-run:
```bash
docker compose build # or podman-compose build
DCE_MIN_FREE_MB=0 ./scripts/run-operator-validation.sh --target KotOR_discord_msgs
```
Large `yes_general` may still skip; export that channel separately with more container memory if needed.
**Disk:** ~22 GiB free on `/home` (2026-05-30); large channel merges still need headroom. **Disk:** ~22 GiB free on `/home` (2026-05-30); large channel merges still need headroom.

View file

@ -552,7 +552,7 @@ message_count() {
is_skippable_channel_export_failure() { is_skippable_channel_export_failure() {
local log_file=$1 local log_file=$1
grep -qiE \ grep -qiE \
"failed: forbidden|failed: not found|Missing Access|403 Forbidden|404 Not Found|Cannot read message history|No space left on device|SQLITE_FULL|ENOSPC|disk full|not enough space" \ "failed: forbidden|failed: not found|Missing Access|403 Forbidden|404 Not Found|Cannot read message history|No space left on device|SQLITE_FULL|ENOSPC|disk full|not enough space|Aborted \\(core dumped\\)|core dumped|out of memory|OOM|Killed|SIGKILL|SIGABRT" \
"$log_file" "$log_file"
} }
@ -579,6 +579,14 @@ export_channel_incremental() {
return 0 return 0
fi fi
# SIGABRT (134), SIGKILL/OOM (137), SIGSEGV (139) — bash reports these when the CLI crashes.
if (( export_status == 134 || export_status == 137 || export_status == 139 )); then
log "Skipping channel $channel_id (export process aborted, exit $export_status)."
[[ -s "$export_log" ]] && cat "$export_log" >&2
rm -f "$export_log"
return 2
fi
if is_skippable_channel_export_failure "$export_log"; then if is_skippable_channel_export_failure "$export_log"; then
log "Skipping channel $channel_id (inaccessible or non-fatal export error)." log "Skipping channel $channel_id (inaccessible or non-fatal export error)."
cat "$export_log" >&2 cat "$export_log" >&2