diff --git a/.docs/Recurring-Scrape-Setup.md b/.docs/Recurring-Scrape-Setup.md index 59efff07..ac7f7705 100644 --- a/.docs/Recurring-Scrape-Setup.md +++ b/.docs/Recurring-Scrape-Setup.md @@ -321,14 +321,24 @@ Space requirements: ## Smoke test validation -Run the full local suite from the repo root (requires `jq`; `container-smoke.sh` also needs Docker/Podman and a writable `archive_root` from `config/scrape-targets.json`): +Run the full offline suite from the repo root (requires `jq`): ```bash -chmod +x scripts/*.sh scripts/tests/*.sh -for script in scripts/tests/*.sh; do - echo "==> $script" - "$script" -done +./scripts/run-all-smokes.sh +``` + +With Docker/Podman, include the container smoke: + +```bash +./scripts/run-all-smokes.sh --include-container +``` + +**Archive integrity helpers** (not smokes; run against live `~/Documents` trees): + +```bash +./scripts/audit-archive-json.sh --target KotOR_discord_msgs +./scripts/salvage-truncated-export.sh path/to/export.json # truncated JSON only +./scripts/prove-incremental-append.sh --target NAME # live grow-only proof (needs token) ``` | Script | CI (`recurring-scrape-smoke`) | Notes | @@ -342,9 +352,13 @@ done | `gh-approve-pr-runs-smoke.sh` | yes | Fork PR workflow helper | | `documents-scrape-smoke.sh` | yes | Unified Documents workflow | | `verify-documents-auth-smoke.sh` | yes | Archive verify + auth bootstrap | -| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets` | +| `scrape-here-smoke.sh` | yes | Workspace bridge launcher | +| `bootstrap-recurring-scrape-smoke.sh` | yes | Bootstrap dry-run | +| `audit-archive-json-smoke.sh` | yes | Invalid JSON detection | +| `prove-incremental-append-smoke.sh` | yes | Offline prove snapshot/compare | +| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets`; use `--include-container` | -GitHub Actions runs the CI-marked scripts on every push/PR via `.github/workflows/main.yml` job `recurring-scrape-smoke`. +GitHub Actions runs `./scripts/run-all-smokes.sh` via `.github/workflows/main.yml` job `recurring-scrape-smoke`. ## Next Steps diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 690ed0d5..0d978a58 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -65,21 +65,7 @@ jobs: run: sudo apt-get update && sudo apt-get install -y jq - name: Run recurring scrape smoke tests - run: | - chmod +x scripts/*.sh scripts/tests/*.sh - ./scripts/tests/run-discord-scrape-smoke.sh - ./scripts/tests/error-path-smoke.sh - ./scripts/tests/cron-idempotency-smoke.sh - ./scripts/tests/end-to-end-preflight-smoke.sh - ./scripts/tests/setup-cron-smoke.sh - ./scripts/tests/run-discord-scrape-host-smoke.sh - ./scripts/tests/gh-approve-pr-runs-smoke.sh - ./scripts/tests/documents-scrape-smoke.sh - ./scripts/tests/verify-documents-auth-smoke.sh - ./scripts/tests/scrape-here-smoke.sh - ./scripts/tests/bootstrap-recurring-scrape-smoke.sh - ./scripts/tests/audit-archive-json-smoke.sh - ./scripts/tests/prove-incremental-append-smoke.sh + run: ./scripts/run-all-smokes.sh test: # Tests need access to secrets, so we can't run them against PRs because of limited trust diff --git a/docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md b/docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md new file mode 100644 index 00000000..2963fc8e --- /dev/null +++ b/docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md @@ -0,0 +1,28 @@ +--- +title: feat: Unified smoke runner and validation docs +type: feat +status: complete +date: 2026-05-29 +origin: Repeated /lfg — consolidate test entrypoint and align docs with CI +--- + +# feat: Unified smoke runner and validation docs + +## Summary + +Recurring scrape is feature-complete. Add one authoritative smoke runner script, wire CI to it, and update setup docs for audit/salvage/prove offline modes. + +## Requirements + +| ID | Requirement | +|----|-------------| +| R1 | `scripts/run-all-smokes.sh` runs all `scripts/tests/*.sh` except `container-smoke.sh` by default | +| R2 | `--include-container` runs container smoke for local Docker/Podman validation | +| R3 | CI `recurring-scrape-smoke` job uses `run-all-smokes.sh` | +| R4 | `.docs/Recurring-Scrape-Setup.md` smoke table lists all 12 smokes + audit/salvage/prove notes | +| R5 | Operator checklist references `run-all-smokes.sh` | + +## Verification + +- `./scripts/run-all-smokes.sh` +- `./scripts/run-all-smokes.sh --include-container` when Docker available (optional local) diff --git a/docs/recurring-scrape-operator-checklist.md b/docs/recurring-scrape-operator-checklist.md index 5edccd80..907ed8a7 100644 --- a/docs/recurring-scrape-operator-checklist.md +++ b/docs/recurring-scrape-operator-checklist.md @@ -33,4 +33,10 @@ Installed jobs are marked `# BEGIN discord-scrape` in `crontab -l`. Logs append See `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`. +Validate scripts after changes: + +```bash +./scripts/run-all-smokes.sh +``` + Full detail: [.docs/Recurring-Scrape-Setup.md](../.docs/Recurring-Scrape-Setup.md) diff --git a/scripts/run-all-smokes.sh b/scripts/run-all-smokes.sh new file mode 100755 index 00000000..5fe62999 --- /dev/null +++ b/scripts/run-all-smokes.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash + +set -Eeuo pipefail + +SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P) +REPO_ROOT="${DCE_REPO_ROOT:-$(cd "$SCRIPT_DIR/.." && pwd -P)}" +INCLUDE_CONTAINER=0 + +usage() { + cat <&2 + exit 1 +} + +main() { + while (($#)); do + case "$1" in + --include-container) + INCLUDE_CONTAINER=1 + shift + ;; + --help|-h) + usage + exit 0 + ;; + *) + die "Unknown option: $1" + ;; + esac + done + + local tests_dir="$REPO_ROOT/scripts/tests" + [[ -d "$tests_dir" ]] || die "Missing tests directory: $tests_dir" + + chmod +x "$REPO_ROOT"/scripts/*.sh "$tests_dir"/*.sh 2>/dev/null || true + + local script_path failures=0 ran=0 + for script_path in "$tests_dir"/*.sh; do + [[ -f "$script_path" ]] || continue + local base + base=$(basename "$script_path") + if [[ "$base" == "container-smoke.sh" && "$INCLUDE_CONTAINER" -eq 0 ]]; then + printf 'SKIP: %s (pass --include-container to run)\n' "$base" + continue + fi + printf '==> %s\n' "$base" + if ! "$script_path"; then + failures=$((failures + 1)) + fi + ran=$((ran + 1)) + done + + if (( ran == 0 )); then + die "No smoke scripts found under $tests_dir" + fi + + if (( failures > 0 )); then + die "$failures smoke script(s) failed." + fi + + printf 'All %d smoke script(s) passed.\n' "$ran" +} + +main "$@"