mirror of
https://github.com/Tyrrrz/DiscordChatExporter.git
synced 2026-06-09 15:52:37 -06:00
feat(scrape): add run-all-smokes.sh and align CI docs
Single entrypoint for offline smokes; CI uses it instead of listing each script. Update setup guide and operator checklist with full smoke table.
This commit is contained in:
parent
10cd2a534d
commit
89091d76ef
|
|
@ -321,14 +321,24 @@ Space requirements:
|
||||||
|
|
||||||
## Smoke test validation
|
## Smoke test validation
|
||||||
|
|
||||||
Run the full local suite from the repo root (requires `jq`; `container-smoke.sh` also needs Docker/Podman and a writable `archive_root` from `config/scrape-targets.json`):
|
Run the full offline suite from the repo root (requires `jq`):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
chmod +x scripts/*.sh scripts/tests/*.sh
|
./scripts/run-all-smokes.sh
|
||||||
for script in scripts/tests/*.sh; do
|
```
|
||||||
echo "==> $script"
|
|
||||||
"$script"
|
With Docker/Podman, include the container smoke:
|
||||||
done
|
|
||||||
|
```bash
|
||||||
|
./scripts/run-all-smokes.sh --include-container
|
||||||
|
```
|
||||||
|
|
||||||
|
**Archive integrity helpers** (not smokes; run against live `~/Documents` trees):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/audit-archive-json.sh --target KotOR_discord_msgs
|
||||||
|
./scripts/salvage-truncated-export.sh path/to/export.json # truncated JSON only
|
||||||
|
./scripts/prove-incremental-append.sh --target NAME # live grow-only proof (needs token)
|
||||||
```
|
```
|
||||||
|
|
||||||
| Script | CI (`recurring-scrape-smoke`) | Notes |
|
| Script | CI (`recurring-scrape-smoke`) | Notes |
|
||||||
|
|
@ -342,9 +352,13 @@ done
|
||||||
| `gh-approve-pr-runs-smoke.sh` | yes | Fork PR workflow helper |
|
| `gh-approve-pr-runs-smoke.sh` | yes | Fork PR workflow helper |
|
||||||
| `documents-scrape-smoke.sh` | yes | Unified Documents workflow |
|
| `documents-scrape-smoke.sh` | yes | Unified Documents workflow |
|
||||||
| `verify-documents-auth-smoke.sh` | yes | Archive verify + auth bootstrap |
|
| `verify-documents-auth-smoke.sh` | yes | Archive verify + auth bootstrap |
|
||||||
| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets` |
|
| `scrape-here-smoke.sh` | yes | Workspace bridge launcher |
|
||||||
|
| `bootstrap-recurring-scrape-smoke.sh` | yes | Bootstrap dry-run |
|
||||||
|
| `audit-archive-json-smoke.sh` | yes | Invalid JSON detection |
|
||||||
|
| `prove-incremental-append-smoke.sh` | yes | Offline prove snapshot/compare |
|
||||||
|
| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets`; use `--include-container` |
|
||||||
|
|
||||||
GitHub Actions runs the CI-marked scripts on every push/PR via `.github/workflows/main.yml` job `recurring-scrape-smoke`.
|
GitHub Actions runs `./scripts/run-all-smokes.sh` via `.github/workflows/main.yml` job `recurring-scrape-smoke`.
|
||||||
|
|
||||||
## Next Steps
|
## Next Steps
|
||||||
|
|
||||||
|
|
|
||||||
16
.github/workflows/main.yml
vendored
16
.github/workflows/main.yml
vendored
|
|
@ -65,21 +65,7 @@ jobs:
|
||||||
run: sudo apt-get update && sudo apt-get install -y jq
|
run: sudo apt-get update && sudo apt-get install -y jq
|
||||||
|
|
||||||
- name: Run recurring scrape smoke tests
|
- name: Run recurring scrape smoke tests
|
||||||
run: |
|
run: ./scripts/run-all-smokes.sh
|
||||||
chmod +x scripts/*.sh scripts/tests/*.sh
|
|
||||||
./scripts/tests/run-discord-scrape-smoke.sh
|
|
||||||
./scripts/tests/error-path-smoke.sh
|
|
||||||
./scripts/tests/cron-idempotency-smoke.sh
|
|
||||||
./scripts/tests/end-to-end-preflight-smoke.sh
|
|
||||||
./scripts/tests/setup-cron-smoke.sh
|
|
||||||
./scripts/tests/run-discord-scrape-host-smoke.sh
|
|
||||||
./scripts/tests/gh-approve-pr-runs-smoke.sh
|
|
||||||
./scripts/tests/documents-scrape-smoke.sh
|
|
||||||
./scripts/tests/verify-documents-auth-smoke.sh
|
|
||||||
./scripts/tests/scrape-here-smoke.sh
|
|
||||||
./scripts/tests/bootstrap-recurring-scrape-smoke.sh
|
|
||||||
./scripts/tests/audit-archive-json-smoke.sh
|
|
||||||
./scripts/tests/prove-incremental-append-smoke.sh
|
|
||||||
|
|
||||||
test:
|
test:
|
||||||
# Tests need access to secrets, so we can't run them against PRs because of limited trust
|
# Tests need access to secrets, so we can't run them against PRs because of limited trust
|
||||||
|
|
|
||||||
28
docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md
Normal file
28
docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
---
|
||||||
|
title: feat: Unified smoke runner and validation docs
|
||||||
|
type: feat
|
||||||
|
status: complete
|
||||||
|
date: 2026-05-29
|
||||||
|
origin: Repeated /lfg — consolidate test entrypoint and align docs with CI
|
||||||
|
---
|
||||||
|
|
||||||
|
# feat: Unified smoke runner and validation docs
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Recurring scrape is feature-complete. Add one authoritative smoke runner script, wire CI to it, and update setup docs for audit/salvage/prove offline modes.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
| ID | Requirement |
|
||||||
|
|----|-------------|
|
||||||
|
| R1 | `scripts/run-all-smokes.sh` runs all `scripts/tests/*.sh` except `container-smoke.sh` by default |
|
||||||
|
| R2 | `--include-container` runs container smoke for local Docker/Podman validation |
|
||||||
|
| R3 | CI `recurring-scrape-smoke` job uses `run-all-smokes.sh` |
|
||||||
|
| R4 | `.docs/Recurring-Scrape-Setup.md` smoke table lists all 12 smokes + audit/salvage/prove notes |
|
||||||
|
| R5 | Operator checklist references `run-all-smokes.sh` |
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- `./scripts/run-all-smokes.sh`
|
||||||
|
- `./scripts/run-all-smokes.sh --include-container` when Docker available (optional local)
|
||||||
|
|
@ -33,4 +33,10 @@ Installed jobs are marked `# BEGIN discord-scrape` in `crontab -l`. Logs append
|
||||||
|
|
||||||
See `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`.
|
See `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`.
|
||||||
|
|
||||||
|
Validate scripts after changes:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/run-all-smokes.sh
|
||||||
|
```
|
||||||
|
|
||||||
Full detail: [.docs/Recurring-Scrape-Setup.md](../.docs/Recurring-Scrape-Setup.md)
|
Full detail: [.docs/Recurring-Scrape-Setup.md](../.docs/Recurring-Scrape-Setup.md)
|
||||||
|
|
|
||||||
73
scripts/run-all-smokes.sh
Executable file
73
scripts/run-all-smokes.sh
Executable file
|
|
@ -0,0 +1,73 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -Eeuo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)
|
||||||
|
REPO_ROOT="${DCE_REPO_ROOT:-$(cd "$SCRIPT_DIR/.." && pwd -P)}"
|
||||||
|
INCLUDE_CONTAINER=0
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<EOF
|
||||||
|
Usage:
|
||||||
|
$(basename "$0") [--include-container]
|
||||||
|
|
||||||
|
Run every script in scripts/tests/ (offline smokes by default).
|
||||||
|
Use --include-container to also run container-smoke.sh (needs Docker/Podman).
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf 'ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
while (($#)); do
|
||||||
|
case "$1" in
|
||||||
|
--include-container)
|
||||||
|
INCLUDE_CONTAINER=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--help|-h)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "Unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
local tests_dir="$REPO_ROOT/scripts/tests"
|
||||||
|
[[ -d "$tests_dir" ]] || die "Missing tests directory: $tests_dir"
|
||||||
|
|
||||||
|
chmod +x "$REPO_ROOT"/scripts/*.sh "$tests_dir"/*.sh 2>/dev/null || true
|
||||||
|
|
||||||
|
local script_path failures=0 ran=0
|
||||||
|
for script_path in "$tests_dir"/*.sh; do
|
||||||
|
[[ -f "$script_path" ]] || continue
|
||||||
|
local base
|
||||||
|
base=$(basename "$script_path")
|
||||||
|
if [[ "$base" == "container-smoke.sh" && "$INCLUDE_CONTAINER" -eq 0 ]]; then
|
||||||
|
printf 'SKIP: %s (pass --include-container to run)\n' "$base"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
printf '==> %s\n' "$base"
|
||||||
|
if ! "$script_path"; then
|
||||||
|
failures=$((failures + 1))
|
||||||
|
fi
|
||||||
|
ran=$((ran + 1))
|
||||||
|
done
|
||||||
|
|
||||||
|
if (( ran == 0 )); then
|
||||||
|
die "No smoke scripts found under $tests_dir"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if (( failures > 0 )); then
|
||||||
|
die "$failures smoke script(s) failed."
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf 'All %d smoke script(s) passed.\n' "$ran"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Loading…
Reference in a new issue