mirror of
https://github.com/Tyrrrz/DiscordChatExporter.git
synced 2026-06-09 15:52:37 -06:00
feat(scrape): add run-all-smokes.sh and align CI docs
Single entrypoint for offline smokes; CI uses it instead of listing each script. Update setup guide and operator checklist with full smoke table.
This commit is contained in:
parent
10cd2a534d
commit
89091d76ef
|
|
@ -321,14 +321,24 @@ Space requirements:
|
|||
|
||||
## Smoke test validation
|
||||
|
||||
Run the full local suite from the repo root (requires `jq`; `container-smoke.sh` also needs Docker/Podman and a writable `archive_root` from `config/scrape-targets.json`):
|
||||
Run the full offline suite from the repo root (requires `jq`):
|
||||
|
||||
```bash
|
||||
chmod +x scripts/*.sh scripts/tests/*.sh
|
||||
for script in scripts/tests/*.sh; do
|
||||
echo "==> $script"
|
||||
"$script"
|
||||
done
|
||||
./scripts/run-all-smokes.sh
|
||||
```
|
||||
|
||||
With Docker/Podman, include the container smoke:
|
||||
|
||||
```bash
|
||||
./scripts/run-all-smokes.sh --include-container
|
||||
```
|
||||
|
||||
**Archive integrity helpers** (not smokes; run against live `~/Documents` trees):
|
||||
|
||||
```bash
|
||||
./scripts/audit-archive-json.sh --target KotOR_discord_msgs
|
||||
./scripts/salvage-truncated-export.sh path/to/export.json # truncated JSON only
|
||||
./scripts/prove-incremental-append.sh --target NAME # live grow-only proof (needs token)
|
||||
```
|
||||
|
||||
| Script | CI (`recurring-scrape-smoke`) | Notes |
|
||||
|
|
@ -342,9 +352,13 @@ done
|
|||
| `gh-approve-pr-runs-smoke.sh` | yes | Fork PR workflow helper |
|
||||
| `documents-scrape-smoke.sh` | yes | Unified Documents workflow |
|
||||
| `verify-documents-auth-smoke.sh` | yes | Archive verify + auth bootstrap |
|
||||
| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets` |
|
||||
| `scrape-here-smoke.sh` | yes | Workspace bridge launcher |
|
||||
| `bootstrap-recurring-scrape-smoke.sh` | yes | Bootstrap dry-run |
|
||||
| `audit-archive-json-smoke.sh` | yes | Invalid JSON detection |
|
||||
| `prove-incremental-append-smoke.sh` | yes | Offline prove snapshot/compare |
|
||||
| `container-smoke.sh` | no (local) | Docker build + `help` / `list-targets`; use `--include-container` |
|
||||
|
||||
GitHub Actions runs the CI-marked scripts on every push/PR via `.github/workflows/main.yml` job `recurring-scrape-smoke`.
|
||||
GitHub Actions runs `./scripts/run-all-smokes.sh` via `.github/workflows/main.yml` job `recurring-scrape-smoke`.
|
||||
|
||||
## Next Steps
|
||||
|
||||
|
|
|
|||
16
.github/workflows/main.yml
vendored
16
.github/workflows/main.yml
vendored
|
|
@ -65,21 +65,7 @@ jobs:
|
|||
run: sudo apt-get update && sudo apt-get install -y jq
|
||||
|
||||
- name: Run recurring scrape smoke tests
|
||||
run: |
|
||||
chmod +x scripts/*.sh scripts/tests/*.sh
|
||||
./scripts/tests/run-discord-scrape-smoke.sh
|
||||
./scripts/tests/error-path-smoke.sh
|
||||
./scripts/tests/cron-idempotency-smoke.sh
|
||||
./scripts/tests/end-to-end-preflight-smoke.sh
|
||||
./scripts/tests/setup-cron-smoke.sh
|
||||
./scripts/tests/run-discord-scrape-host-smoke.sh
|
||||
./scripts/tests/gh-approve-pr-runs-smoke.sh
|
||||
./scripts/tests/documents-scrape-smoke.sh
|
||||
./scripts/tests/verify-documents-auth-smoke.sh
|
||||
./scripts/tests/scrape-here-smoke.sh
|
||||
./scripts/tests/bootstrap-recurring-scrape-smoke.sh
|
||||
./scripts/tests/audit-archive-json-smoke.sh
|
||||
./scripts/tests/prove-incremental-append-smoke.sh
|
||||
run: ./scripts/run-all-smokes.sh
|
||||
|
||||
test:
|
||||
# Tests need access to secrets, so we can't run them against PRs because of limited trust
|
||||
|
|
|
|||
28
docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md
Normal file
28
docs/plans/2026-05-29-020-feat-smoke-runner-docs-plan.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
title: feat: Unified smoke runner and validation docs
|
||||
type: feat
|
||||
status: complete
|
||||
date: 2026-05-29
|
||||
origin: Repeated /lfg — consolidate test entrypoint and align docs with CI
|
||||
---
|
||||
|
||||
# feat: Unified smoke runner and validation docs
|
||||
|
||||
## Summary
|
||||
|
||||
Recurring scrape is feature-complete. Add one authoritative smoke runner script, wire CI to it, and update setup docs for audit/salvage/prove offline modes.
|
||||
|
||||
## Requirements
|
||||
|
||||
| ID | Requirement |
|
||||
|----|-------------|
|
||||
| R1 | `scripts/run-all-smokes.sh` runs all `scripts/tests/*.sh` except `container-smoke.sh` by default |
|
||||
| R2 | `--include-container` runs container smoke for local Docker/Podman validation |
|
||||
| R3 | CI `recurring-scrape-smoke` job uses `run-all-smokes.sh` |
|
||||
| R4 | `.docs/Recurring-Scrape-Setup.md` smoke table lists all 12 smokes + audit/salvage/prove notes |
|
||||
| R5 | Operator checklist references `run-all-smokes.sh` |
|
||||
|
||||
## Verification
|
||||
|
||||
- `./scripts/run-all-smokes.sh`
|
||||
- `./scripts/run-all-smokes.sh --include-container` when Docker available (optional local)
|
||||
|
|
@ -33,4 +33,10 @@ Installed jobs are marked `# BEGIN discord-scrape` in `crontab -l`. Logs append
|
|||
|
||||
See `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`.
|
||||
|
||||
Validate scripts after changes:
|
||||
|
||||
```bash
|
||||
./scripts/run-all-smokes.sh
|
||||
```
|
||||
|
||||
Full detail: [.docs/Recurring-Scrape-Setup.md](../.docs/Recurring-Scrape-Setup.md)
|
||||
|
|
|
|||
73
scripts/run-all-smokes.sh
Executable file
73
scripts/run-all-smokes.sh
Executable file
|
|
@ -0,0 +1,73 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
set -Eeuo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)
|
||||
REPO_ROOT="${DCE_REPO_ROOT:-$(cd "$SCRIPT_DIR/.." && pwd -P)}"
|
||||
INCLUDE_CONTAINER=0
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage:
|
||||
$(basename "$0") [--include-container]
|
||||
|
||||
Run every script in scripts/tests/ (offline smokes by default).
|
||||
Use --include-container to also run container-smoke.sh (needs Docker/Podman).
|
||||
EOF
|
||||
}
|
||||
|
||||
die() {
|
||||
printf 'ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
main() {
|
||||
while (($#)); do
|
||||
case "$1" in
|
||||
--include-container)
|
||||
INCLUDE_CONTAINER=1
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "Unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
local tests_dir="$REPO_ROOT/scripts/tests"
|
||||
[[ -d "$tests_dir" ]] || die "Missing tests directory: $tests_dir"
|
||||
|
||||
chmod +x "$REPO_ROOT"/scripts/*.sh "$tests_dir"/*.sh 2>/dev/null || true
|
||||
|
||||
local script_path failures=0 ran=0
|
||||
for script_path in "$tests_dir"/*.sh; do
|
||||
[[ -f "$script_path" ]] || continue
|
||||
local base
|
||||
base=$(basename "$script_path")
|
||||
if [[ "$base" == "container-smoke.sh" && "$INCLUDE_CONTAINER" -eq 0 ]]; then
|
||||
printf 'SKIP: %s (pass --include-container to run)\n' "$base"
|
||||
continue
|
||||
fi
|
||||
printf '==> %s\n' "$base"
|
||||
if ! "$script_path"; then
|
||||
failures=$((failures + 1))
|
||||
fi
|
||||
ran=$((ran + 1))
|
||||
done
|
||||
|
||||
if (( ran == 0 )); then
|
||||
die "No smoke scripts found under $tests_dir"
|
||||
fi
|
||||
|
||||
if (( failures > 0 )); then
|
||||
die "$failures smoke script(s) failed."
|
||||
fi
|
||||
|
||||
printf 'All %d smoke script(s) passed.\n' "$ran"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Loading…
Reference in a new issue