feat(operators): pass --channel through handoff, prove, and proof

Complete the operator CLI chain so focused yes_general catch-up can run
end-to-end with --target and --channel on handoff dry-run, prove scrape,
and operator-proof orchestration.
This commit is contained in:
Copilot 2026-06-03 06:36:02 -05:00
parent ae120c916f
commit 8057a4443a
6 changed files with 192 additions and 8 deletions

View file

@ -0,0 +1,121 @@
---
title: "feat: Pass --channel through remaining operator wrappers"
type: feat
status: complete
date: 2026-06-04
origin: /lfg — plan 049 wired --channel on documents-scrape and operator-validation; handoff, prove, and operator-proof still reject it, blocking focused yes_general catch-up
deepened: 2026-06-04
---
# feat: Pass --channel through remaining operator wrappers
## Summary
Complete the operator CLI chain by accepting repeatable `--channel ID` in `operator-handoff.sh`, `prove-incremental-append.sh`, and `run-operator-proof.sh`, forwarding to downstream scripts the same way plan 049 did for documents-scrape and operator-validation.
## Problem Frame
Operators re-running KotOR `yes_general` (`221726893064454144`) need single-channel workflows end-to-end:
```bash
./scripts/run-operator-proof.sh --target KotOR_discord_msgs --channel 221726893064454144
```
Plan 049 fixed `run-documents-scrape.sh` and `run-operator-validation.sh`, but `operator-handoff` dry-run, `prove-incremental-append` scrape step, and `run-operator-proof` orchestration still die on `--channel`.
## Requirements
| ID | Requirement |
|----|-------------|
| R1 | `operator-handoff.sh` accepts repeatable `--channel ID` and forwards to `run-documents-scrape.sh --dry-run` |
| R2 | `prove-incremental-append.sh` accepts repeatable `--channel ID` and forwards to `run-discord-scrape-host.sh scrape` |
| R3 | `run-operator-proof.sh` accepts repeatable `--channel ID` and forwards to handoff, documents scrape, and prove |
| R4 | Usage text on all three scripts documents `--channel` requires exactly one `--target` (enforced downstream) |
| R5 | Smokes cover `--channel` acceptance on handoff dry-run path; prove smoke documents passthrough via arg capture or dry-run stub where feasible |
| R6 | `DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh` passes |
## Key Technical Decisions
- **Mirror plan 049 passthrough pattern**: collect `--channel` into an array during option parsing; append to downstream invocations unchanged.
- **Snapshot scope unchanged**: `prove-incremental-append` still snapshots all archives under the target; single-channel scrape must not shrink any existing archive (existing compare logic).
- **Handoff only dry-runs scrape**: `--channel` affects the dry-run invocation only; verify-operator-ready remains target-wide.
## Implementation Units
### U1. operator-handoff --channel passthrough
**Goal:** Handoff dry-run accepts and forwards `--channel`.
**Requirements:** R1, R4
**Files:**
- `scripts/operator-handoff.sh`
- `scripts/tests/operator-handoff-smoke.sh`
**Approach:** Parse `--channel` into `CHANNEL_ARGS`; pass `"${CHANNEL_ARGS[@]}"` to `"$DOCUMENTS_SCRAPE" --dry-run`.
**Test scenarios:**
- Handoff with `--target demo --channel 111...` on temp config completes with "Handoff complete" (dry-run path; no live Discord).
**Verification:** `operator-handoff-smoke.sh` passes.
### U2. prove-incremental-append --channel passthrough
**Goal:** Prove script forwards `--channel` to host scrape runner.
**Requirements:** R2, R4
**Files:**
- `scripts/prove-incremental-append.sh`
- `scripts/tests/prove-incremental-append-smoke.sh` (optional arg-capture if host stubbed; at minimum usage/help acceptance)
**Approach:** Parse `--channel` into array; append to `"$HOST_RUNNER" scrape ...` invocation.
**Test scenarios:**
- `--help` or dry-run parse path accepts `--channel` without "Unknown option".
- When fake host runner captures argv, `--channel` appears on scrape command (if smoke infrastructure allows; otherwise extend handoff-style config-only test).
**Verification:** `prove-incremental-append-smoke.sh` passes.
### U3. run-operator-proof --channel passthrough
**Goal:** End-to-end operator proof orchestrator forwards `--channel` to all three steps.
**Requirements:** R3, R4
**Files:**
- `scripts/run-operator-proof.sh`
**Approach:** Parse `--channel` once; pass to `"$HANDOFF"`, `"$DOCUMENTS"`, and `"$PROVE"` when `--target` is set.
**Test scenarios:**
- `--dry-run` with `--target demo --channel 111...` completes without unknown-option error.
**Verification:** Manual or documents-scrape-smoke pattern; full proof requires auth (out of scope for smoke).
### U4. Full smoke gate
**Goal:** Confirm no regressions across operator chain.
**Requirements:** R6
**Files:** (none — verification only)
**Verification:** `DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh` → 20/20 (or current count) pass.
## Scope Boundaries
### In scope
- CLI passthrough on three remaining operator wrappers
- Targeted smoke updates
### Deferred to Follow-Up Work
- Stopping stale full-target validation processes
- Running live yes_general catch-up inside LFG
- Container memory tuning for large channels
## Assumptions
- Downstream `--channel` + single `--target` validation in `run-discord-scrape.sh` remains authoritative; wrappers do not re-validate channel membership.

View file

@ -10,17 +10,22 @@ CONFIG_PATH="${DCE_CONFIG_FILE:-$REPO_ROOT/config/scrape-targets.json}"
VERIFY_READY="$REPO_ROOT/scripts/verify-operator-ready.sh" VERIFY_READY="$REPO_ROOT/scripts/verify-operator-ready.sh"
DOCUMENTS_SCRAPE="$REPO_ROOT/scripts/run-documents-scrape.sh" DOCUMENTS_SCRAPE="$REPO_ROOT/scripts/run-documents-scrape.sh"
SKIP_DF=0 SKIP_DF=0
TARGET=""
CHANNEL_ARGS=()
usage() { usage() {
cat <<EOF cat <<EOF
Usage: Usage:
$(basename "$0") [--config PATH] [--skip-df] $(basename "$0") [--config PATH] [--skip-df] [--target NAME] [--channel ID]
Run operator handoff checks before cron install or a full scrape: Run operator handoff checks before cron install or a full scrape:
1. Free-space summary (archive_root + repo) 1. Free-space summary (archive_root + repo)
2. verify-operator-ready (jq, compose, auth, archives) 2. verify-operator-ready (jq, compose, auth, archives)
3. run-documents-scrape --dry-run (archive paths only) 3. run-documents-scrape --dry-run (archive paths only)
--target NAME Limit dry-run scrape plan to one configured target
--channel ID With exactly one --target, limit dry-run to channel ID (repeatable)
Environment: Environment:
DCE_MIN_FREE_MB Minimum MiB free (default 1024 in verify-operator-ready) DCE_MIN_FREE_MB Minimum MiB free (default 1024 in verify-operator-ready)
EOF EOF
@ -63,6 +68,16 @@ main() {
SKIP_DF=1 SKIP_DF=1
shift shift
;; ;;
--target)
[[ $# -ge 2 ]] || die "Missing value for --target."
TARGET=$2
shift 2
;;
--channel)
[[ $# -ge 2 ]] || die "Missing value for --channel."
CHANNEL_ARGS+=(--channel "$2")
shift 2
;;
--help|-h) --help|-h)
usage usage
exit 0 exit 0
@ -86,10 +101,16 @@ main() {
fi fi
"$VERIFY_READY" --config "$CONFIG_PATH" "$VERIFY_READY" --config "$CONFIG_PATH"
"$DOCUMENTS_SCRAPE" --dry-run --config "$CONFIG_PATH" local -a dry_run_args=(--dry-run --config "$CONFIG_PATH")
[[ -n "$TARGET" ]] && dry_run_args+=(--target "$TARGET")
dry_run_args+=("${CHANNEL_ARGS[@]}")
"$DOCUMENTS_SCRAPE" "${dry_run_args[@]}"
printf '\nHandoff complete. Safe to run:\n' printf '\nHandoff complete. Safe to run:\n'
printf ' ./scripts/run-documents-scrape.sh\n' printf ' ./scripts/run-documents-scrape.sh'
[[ -n "$TARGET" ]] && printf ' --target %s' "$TARGET"
((${#CHANNEL_ARGS[@]})) && printf ' %s' "${CHANNEL_ARGS[*]}"
printf '\n'
printf ' ./scripts/setup-cron.sh --dry-run\n' printf ' ./scripts/setup-cron.sh --dry-run\n'
} }

View file

@ -12,7 +12,7 @@ SNAPSHOT_DIR=""
usage() { usage() {
cat <<EOF cat <<EOF
Usage: Usage:
$(basename "$0") --target NAME [--config PATH] $(basename "$0") --target NAME [--config PATH] [--channel ID]
$(basename "$0") --target NAME --snapshot-only --snapshot-file PATH [--config PATH] $(basename "$0") --target NAME --snapshot-only --snapshot-file PATH [--config PATH]
$(basename "$0") --compare-snapshots BEFORE.tsv AFTER.tsv $(basename "$0") --compare-snapshots BEFORE.tsv AFTER.tsv
@ -21,6 +21,8 @@ run one incremental scrape, then assert:
- archive file paths are unchanged (no parallel channels/ fallbacks) - archive file paths are unchanged (no parallel channels/ fallbacks)
- message counts never shrink - message counts never shrink
--channel ID Limit incremental scrape to channel ID (repeatable; requires --target)
Requires valid Discord auth (scrape.env, exported DISCORD_TOKEN, or token file). Requires valid Discord auth (scrape.env, exported DISCORD_TOKEN, or token file).
EOF EOF
} }
@ -119,6 +121,7 @@ main() {
local snapshot_file="" local snapshot_file=""
local compare_before="" local compare_before=""
local compare_after="" local compare_after=""
local -a channel_args=()
trap cleanup EXIT trap cleanup EXIT
@ -149,6 +152,11 @@ main() {
compare_after=$3 compare_after=$3
shift 3 shift 3
;; ;;
--channel)
[[ $# -ge 2 ]] || die "Missing value for --channel."
channel_args+=(--channel "$2")
shift 2
;;
--help|-h) --help|-h)
usage usage
exit 0 exit 0
@ -198,7 +206,7 @@ main() {
"$REPO_ROOT/config/scrape-targets.json"|config/scrape-targets.json|./config/scrape-targets.json) ;; "$REPO_ROOT/config/scrape-targets.json"|config/scrape-targets.json|./config/scrape-targets.json) ;;
*) container_config="$CONFIG_PATH" ;; *) container_config="$CONFIG_PATH" ;;
esac esac
"$HOST_RUNNER" scrape --config "$container_config" --target "$target" "$HOST_RUNNER" scrape --config "$container_config" --target "$target" "${channel_args[@]}"
snapshot_archives "$output_dir" "$after_file" snapshot_archives "$output_dir" "$after_file"
compare_snapshots "$before_file" "$after_file" compare_snapshots "$before_file" "$after_file"

View file

@ -16,17 +16,20 @@ source "$SCRIPT_DIR/lib/scrape-run-plan.sh"
TARGET="" TARGET=""
SYNC_GUI_FLAG=0 SYNC_GUI_FLAG=0
DRY_RUN=0 DRY_RUN=0
CHANNEL_ARGS=()
usage() { usage() {
cat <<EOF cat <<EOF
Usage: Usage:
$(basename "$0") [--target NAME] [--config PATH] [--sync-gui] [--dry-run] $(basename "$0") [--target NAME] [--channel ID] [--config PATH] [--sync-gui] [--dry-run]
End-to-end operator proof: End-to-end operator proof:
operator-handoff → incremental scrape → prove-incremental-append operator-handoff → incremental scrape → prove-incremental-append
When --target is omitted, all enabled targets in the config are processed. When --target is omitted, all enabled targets in the config are processed.
--channel ID With exactly one --target, limit scrape/prove to channel ID (repeatable)
Logs append to logs/operator-proof-<timestamp>.log Logs append to logs/operator-proof-<timestamp>.log
EOF EOF
} }
@ -57,6 +60,11 @@ main() {
DRY_RUN=1 DRY_RUN=1
shift shift
;; ;;
--channel)
[[ $# -ge 2 ]] || die "Missing value for --channel."
CHANNEL_ARGS+=(--channel "$2")
shift 2
;;
--help|-h) --help|-h)
usage usage
exit 0 exit 0
@ -98,7 +106,10 @@ main() {
"$SYNC_GUI" --force "$SYNC_GUI" --force
fi fi
"$HANDOFF" --config "$CONFIG_PATH" local -a handoff_args=(--config "$CONFIG_PATH")
[[ -n "$TARGET" ]] && handoff_args+=(--target "$TARGET")
handoff_args+=("${CHANNEL_ARGS[@]}")
"$HANDOFF" "${handoff_args[@]}"
if (( DRY_RUN == 1 )); then if (( DRY_RUN == 1 )); then
printf '\nDry run complete (no Discord scrape).\n' printf '\nDry run complete (no Discord scrape).\n'
@ -107,7 +118,9 @@ main() {
for name in "${targets[@]}"; do for name in "${targets[@]}"; do
printf '\n--- Target: %s ---\n' "$name" printf '\n--- Target: %s ---\n' "$name"
if "$DOCUMENTS" --config "$CONFIG_PATH" --target "$name" && "$PROVE" --config "$CONFIG_PATH" --target "$name"; then local -a scrape_args=(--config "$CONFIG_PATH" --target "$name")
scrape_args+=("${CHANNEL_ARGS[@]}")
if "$DOCUMENTS" "${scrape_args[@]}" && "$PROVE" "${scrape_args[@]}"; then
succeeded=$((succeeded + 1)) succeeded=$((succeeded + 1))
printf 'Operator proof passed for %s\n' "$name" printf 'Operator proof passed for %s\n' "$name"
else else

View file

@ -50,4 +50,20 @@ if [[ "$handoff_status" -ne 0 ]] || ! grep -q 'Handoff complete' <<<"$handoff_ou
exit 1 exit 1
fi fi
set +e
channel_output=$(
DCE_MIN_FREE_MB=0 \
DCE_CONFIG_FILE="$CONFIG_PATH" \
DCE_ENV_FILE="$ENV_PATH" \
"$HANDOFF" --config "$CONFIG_PATH" --skip-df --target demo --channel 111111111111111111 2>&1
)
channel_status=$?
set -e
if [[ "$channel_status" -ne 0 ]] || ! grep -q 'Handoff complete' <<<"$channel_output"; then
printf 'operator-handoff --channel failed (status=%s)\n' "$channel_status" >&2
printf '%s\n' "$channel_output" >&2
exit 1
fi
printf 'operator-handoff-smoke: ok\n' printf 'operator-handoff-smoke: ok\n'

View file

@ -75,4 +75,9 @@ if "$PROVE" --compare-snapshots "$AFTER" "$BEFORE" 2>/dev/null; then
exit 1 exit 1
fi fi
if ! "$PROVE" --help 2>&1 | grep -q -- '--channel'; then
printf 'ERROR: prove --help should document --channel\n' >&2
exit 1
fi
printf 'prove-incremental-append-smoke: ok\n' printf 'prove-incremental-append-smoke: ok\n'