From 9c22a3efeed346a071af7631decefce8d5578f89 Mon Sep 17 00:00:00 2001 From: Boden Date: Fri, 29 May 2026 16:08:36 -0500 Subject: [PATCH] docs(scrape): track GUI zip bridge doc in source repo Add docs/gui-zip-recurring-scrape-bridge.md and cross-links so GUI-only users have a versioned quick-start beside the linux-x64 zip folder. --- Readme.md | 2 +- docs/gui-zip-recurring-scrape-bridge.md | 52 +++++++++++++++++++ ...30-docs-version-gui-bridge-in-repo-plan.md | 26 ++++++++++ docs/recurring-scrape-merge-readiness.md | 2 + docs/recurring-scrape-operator-checklist.md | 2 +- 5 files changed, 82 insertions(+), 2 deletions(-) create mode 100644 docs/gui-zip-recurring-scrape-bridge.md create mode 100644 docs/plans/2026-05-29-030-docs-version-gui-bridge-in-repo-plan.md diff --git a/Readme.md b/Readme.md index 8268bb36..8c715fd8 100644 --- a/Readme.md +++ b/Readme.md @@ -81,7 +81,7 @@ To learn more about the war and how you can help, [click here](https://tyrrrz.me ## See also -- [**Recurring Exports**](.docs/Recurring-Scrape-Setup.md) — append-only incremental JSON exports via Docker/cron (Linux/macOS). From the source repo: `./scripts/operator-handoff.sh`, then `./scripts/run-documents-scrape.sh`; validate with `./scripts/run-all-smokes.sh`. GUI zip users: `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md`. Maintainer summary: [docs/recurring-scrape-merge-readiness.md](docs/recurring-scrape-merge-readiness.md). +- [**Recurring Exports**](.docs/Recurring-Scrape-Setup.md) — append-only incremental JSON exports via Docker/cron (Linux/macOS). From the source repo: `./scripts/operator-handoff.sh`, then `./scripts/run-documents-scrape.sh`; validate with `./scripts/run-all-smokes.sh`. GUI zip users: [docs/gui-zip-recurring-scrape-bridge.md](docs/gui-zip-recurring-scrape-bridge.md) (copy to `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` optional). Maintainer summary: [docs/recurring-scrape-merge-readiness.md](docs/recurring-scrape-merge-readiness.md). - [**Documented solutions**](docs/solutions/) — searchable learnings (append-only scrape, Docker/cron workflow); YAML frontmatter: `module`, `tags`, `problem_type` - [**Chat Analytics**](https://github.com/mlomb/chat-analytics) — solution for analyzing chat patterns of Discord users, using exports produced by **DiscordChatExporter**. - [**DiscordChatExporter-frontend**](https://github.com/slatinsky/DiscordChatExporter-frontend) — convenient viewer for exports produced by **DiscordChatExporter**. diff --git a/docs/gui-zip-recurring-scrape-bridge.md b/docs/gui-zip-recurring-scrape-bridge.md new file mode 100644 index 00000000..daa1ccab --- /dev/null +++ b/docs/gui-zip-recurring-scrape-bridge.md @@ -0,0 +1,52 @@ +# Recurring scrape — GUI zip users + +The **DiscordChatExporter.linux-x64** (or similar) GUI zip is only the desktop app. Docker, cron, and append-only JSON merges live in the **source repo** checked out beside the zip: + +```text +DiscordChatExporter.linux-x64/ ← GUI (this folder) +DiscordChatExporter/ ← source repo (scripts, Docker, config) +``` + +Copy this file to `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` if you want the quick reference next to the GUI binary. + +## Quick start (run from source repo) + +```bash +cd ../DiscordChatExporter # or your clone path +./scripts/operator-handoff.sh # disk + verify + archive dry-run +./scripts/sync-token-from-gui.sh --force # token from GUI Settings.dat +# or: cp scrape.env.example scrape.env and set DISCORD_TOKEN (never commit) +./scripts/bootstrap-recurring-scrape.sh # verify, build image, preflight +./scripts/run-documents-scrape.sh # incremental append-only scrape +./scripts/setup-cron.sh --dry-run # preview monthly job (default 1st @ 04:00) +./scripts/setup-cron.sh # install cron after handoff passes +``` + +Optional integrity tools: + +```bash +./scripts/audit-archive-json.sh +# ./scripts/salvage-truncated-export.sh path/to/export.json +``` + +Archives: `config/scrape-targets.json` (typically `~/Documents/*` per target `output_dir`). + +**Disk:** Free several GiB on `/home` and archive roots before large scrapes (`DCE_MIN_FREE_MB`, default 1024). + +**Validate scripts:** `./scripts/run-all-smokes.sh` + +## Wrappers in the GUI zip folder + +From `DiscordChatExporter.linux-x64/`: + +```bash +./bootstrap-recurring-scrape.sh --dry-run +``` + +That script delegates to `../DiscordChatExporter/scripts/bootstrap-recurring-scrape.sh`. + +## More detail + +- [Recurring-Scrape-Setup.md](../.docs/Recurring-Scrape-Setup.md) +- [recurring-scrape-operator-checklist.md](recurring-scrape-operator-checklist.md) +- [recurring-scrape-merge-readiness.md](recurring-scrape-merge-readiness.md) diff --git a/docs/plans/2026-05-29-030-docs-version-gui-bridge-in-repo-plan.md b/docs/plans/2026-05-29-030-docs-version-gui-bridge-in-repo-plan.md new file mode 100644 index 00000000..7004e577 --- /dev/null +++ b/docs/plans/2026-05-29-030-docs-version-gui-bridge-in-repo-plan.md @@ -0,0 +1,26 @@ +--- +title: docs: Version GUI zip bridge doc in source repo +type: docs +status: complete +date: 2026-05-29 +origin: /lfg — RECURRING-SCRAPE.md lived only in sibling GUI zip; track canonical copy in git +--- + +# docs: Version GUI zip bridge doc in source repo + +## Summary + +Add `docs/gui-zip-recurring-scrape-bridge.md` as the git-tracked canonical bridge for GUI zip users. Cross-link from README, operator checklist, and merge-readiness. + +## Requirements + +| ID | Requirement | +|----|-------------| +| R1 | `docs/gui-zip-recurring-scrape-bridge.md` with operator-handoff quick path (repo-relative paths) | +| R2 | `Readme.md` and operator checklist link to bridge doc | +| R3 | `docs/recurring-scrape-merge-readiness.md` notes GUI zip users read bridge doc | +| R4 | `run-all-smokes.sh` still passes | + +## Verification + +- `DCE_MIN_FREE_MB=0 ./scripts/run-all-smokes.sh` diff --git a/docs/recurring-scrape-merge-readiness.md b/docs/recurring-scrape-merge-readiness.md index 6350781b..a58e7af1 100644 --- a/docs/recurring-scrape-merge-readiness.md +++ b/docs/recurring-scrape-merge-readiness.md @@ -2,6 +2,8 @@ Fork branch `feat/recurring-cli-scrape` adds append-only, Docker-based incremental exports with optional monthly cron. Intended for personal archive trees under a configurable `archive_root` (for example `~/Documents/*`). +GUI zip users: [docs/gui-zip-recurring-scrape-bridge.md](gui-zip-recurring-scrape-bridge.md). + ## What ships - **Config:** `config/scrape-targets.json` — per-server `output_dir`, optional `channel_ids`, `enabled` flags diff --git a/docs/recurring-scrape-operator-checklist.md b/docs/recurring-scrape-operator-checklist.md index 654ab8ac..46230ce8 100644 --- a/docs/recurring-scrape-operator-checklist.md +++ b/docs/recurring-scrape-operator-checklist.md @@ -33,7 +33,7 @@ Installed jobs are marked `# BEGIN discord-scrape` in `crontab -l`. Logs append ## GUI zip only -See `../DiscordChatExporter.linux-x64/RECURRING-SCRAPE.md` or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`. +See [gui-zip-recurring-scrape-bridge.md](gui-zip-recurring-scrape-bridge.md) or run `../DiscordChatExporter.linux-x64/bootstrap-recurring-scrape.sh`. Validate scripts after changes: