mirrors/copyparty
1
0
Fork 0
mirror of https://github.com/9001/copyparty.git synced 2025-08-17 09:02:15 -06:00
Code Issues Actions Packages Projects Releases Wiki Activity

readme/doc cleanup

Browse source
This commit is contained in:
ed 2021-09-04 21:07:45 +02:00
parent b332e1992b
commit 1fbb4c09cc
2 changed files with 84 additions and 62 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

142
README.md
View file

@ -8,21 +8,21 @@
turn your phone or raspi into a portable file server with resumable uploads/downloads using *any* web browser
* server runs on anything with `py2.7` or `py3.3+`
* server only needs `py2.7` or `py3.3+`, all dependencies optional
* browse/upload with IE4 / netscape4.0 on win3.11 (heh)
* *resumable* uploads need `firefox 34+` / `chrome 41+` / `safari 7+` for full speed
* code standard: `black`
📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer) // [ie4](#browser-support)
## readme toc
* top
* [quickstart](#quickstart) - download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
* **[quickstart](#quickstart)** - download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
* [on debian](#on-debian) - recommended additional steps on debian
* [notes](#notes) - general notes
* [status](#status) - summary: all planned features work! now please enjoy the bloatening
* [status](#status) - feature summary
* [testimonials](#testimonials) - small collection of user feedback
* [bugs](#bugs)
* [general bugs](#general-bugs)
@ -32,7 +32,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
* [tabs](#tabs) - the main tabs in the ui
* [hotkeys](#hotkeys) - the browser has the following hotkeys (always qwerty)
* [navpane](#navpane) - switching between breadcrumbs or navpane
* [thumbnails](#thumbnails) - press `g` to toggle image/video thumbnails instead of the file listing
* [thumbnails](#thumbnails) - press `g` to toggle grid-view instead of the file listing, and `t` to toggle icons/thumbnails
* [zip downloads](#zip-downloads) - download folders (or file selections) as `zip` or `tar` files
* [uploading](#uploading) - web-browsers can upload using `bup` and `up2k`
* [file-search](#file-search) - drop files/folders into up2k to see if they exist on the server
@ -71,12 +71,12 @@ turn your phone or raspi into a portable file server with resumable uploads/down
## quickstart
download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
running the sfx without arguments (for example doubleclicking it on Windows) will give everyone full access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
running the sfx without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
some recommended options:
* `-e2dsa` enables general file indexing, see [search configuration](#search-configuration)
* `-e2dsa` enables general [file indexing](#file-indexing)
* `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen), see [optional dependencies](#optional-dependencies)
* `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
* replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
@ -119,7 +119,7 @@ browser-specific:
## status
summary: all planned features work! now please enjoy the bloatening
feature summary
* backend stuff
* ☑ sanic multipart parser
@ -137,7 +137,7 @@ summary: all planned features work! now please enjoy the bloatening
* ☑ [folders as zip / tar files](#zip-downloads)
* ☑ FUSE client (read-only)
* browser
* ☑ navpane (directory tree sidebar)
* ☑ [navpane](#navpane) (directory tree sidebar)
* ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
* ☑ audio player (with OS media controls)
* ☑ image gallery with webm player
@ -168,12 +168,11 @@ small collection of user feedback
* Windows: python 3.7 and older cannot read tags with FFprobe, so use Mutagen or upgrade
* Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
* Windows: python 2.7 cannot handle filenames with mojibake
* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions
* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
## general bugs
* all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
* cannot mount something at `/d1/d2/d3` unless `d2` exists inside `d1`
* probably more, pls let me know
## not my bugs
@ -194,15 +193,16 @@ per-folder, per-user permissions
* `-a usr:pwd` adds account `usr` with password `pwd`
* `-v .::r` adds current-folder `.` as the webroot, `r`eadable by anyone
* the syntax is `-v src:dst:perm:perm:...` so local-path, url-path, and one or more permissions to set
* when granting permissions to an account, the names are comma-separated: `-v .::r,usr1,usr2:rw,usr3,usr4`
* granting the same permissions to multiple accounts:
`-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
permissions:
* `r` (read): browse folder contents, download files, download as zip/tar
* `w` (write): upload files, move files *into* folder
* `m` (move): move files/folders *from* folder
* `w` (write): upload files, move files *into* this folder
* `m` (move): move files/folders *from* this folder
* `d` (delete): delete files/folders
example:
examples:
* add accounts named u1, u2, u3 with passwords p1, p2, p3: `-a u1:p1 -a u2:p2 -a u3:p3`
* make folder `/srv` the root of the filesystem, read-only by anyone: `-v /srv::r`
* make folder `/mnt/music` available at `/music`, read-only for u1 and u2, read-write for u3: `-v /mnt/music:music:r,u1,u2:rw,u3`
@ -236,14 +236,14 @@ the main tabs in the ui
## hotkeys
the browser has the following hotkeys (always qwerty)
* `B` toggle breadcrumbs / navpane
* `B` toggle breadcrumbs / [navpane](#navpane)
* `I/K` prev/next folder
* `M` parent folder (or unexpand current)
* `G` toggle list / grid view
* `G` toggle list / [grid view](#thumbnails)
* `T` toggle thumbnails / icons
* `ctrl-X` cut selected files/folders
* `ctrl-V` paste
* `F2` rename selected file/folder
* `F2` [rename](#batch-rename) selected file/folder
* when a file/folder is selected (in not-grid-view):
* `Up/Down` move cursor
* shift+`Up/Down` select and move cursor
@ -270,7 +270,7 @@ the browser has the following hotkeys (always qwerty)
* `M` mute
* when the navpane is open:
* `A/D` adjust tree width
* in the grid view:
* in the [grid view](#thumbnails):
* `S` toggle multiselect
* shift+`A/D` zoom
* in the markdown editor:
@ -288,12 +288,14 @@ switching between breadcrumbs or navpane
click the `🌲` or pressing the `B` hotkey to toggle between breadcrumbs path (default), or a navpane (tree-browser sidebar thing)
click `[-]` and `[+]` (or hotkeys `A`/`D`) to adjust the size, and the `[a]` toggles if the tree should widen dynamically as you go deeper or stay fixed-size
* `[-]` and `[+]` (or hotkeys `A`/`D`) adjust the size
* `[v]` jumps to the currently open folder
* `[a]` toggles automatic widening as you go deeper
## thumbnails
press `g` to toggle image/video thumbnails instead of the file listing
press `g` to toggle grid-view instead of the file listing, and `t` to toggle icons/thumbnails
![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
@ -308,7 +310,7 @@ in the grid/thumbnail view, if the audio player panel is open, songs will start
download folders (or file selections) as `zip` or `tar` files
select which type of archive you want in the browser settings tab:
select which type of archive you want in the `[⚙️] config` tab:
| name | url-suffix | description |
|--|--|--|
@ -355,18 +357,22 @@ see [up2k](#up2k) for details on how it works
the up2k UI is the epitome of polished inutitive experiences:
* "parallel uploads" specifies how many chunks to upload at the same time
* `[🏃]` analysis of other files should continue while one is uploading
* `[💭]` ask for confirmation before files are added to the list
* `[💭]` ask for confirmation before files are added to the queue
* `[💤]` sync uploading between other copyparty browser-tabs so only one is active
* `[🔎]` switch between upload and file-search mode
* `[🔎]` switch between upload and [file-search](#file-search) mode
and then theres the tabs below it,
* `[ok]` is uploads which completed successfully
* `[ng]` is the uploads which failed / got rejected (already exists, ...)
* `[ok]` is the files which completed successfully
* `[ng]` is the ones that failed / got rejected (already exists, ...)
* `[done]` shows a combined list of `[ok]` and `[ng]`, chronological order
* `[busy]` files which are currently hashing, pending-upload, or uploading
* plus up to 3 entries each from `[done]` and `[que]` for context
* `[que]` is all the files that are still queued
note that since up2k has to read each file twice, `[🎈 bup]` can be up to 2x faster in some extreme cases (huge files combined with internet connection faster than the read-speed of your HDD)
if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
### file-search
@ -381,10 +387,6 @@ files go into `[ok]` if they exist (and you get a link to where it is), otherwis
adding the same file multiple times is blocked, so if you first search for a file and then decide to upload it, you have to click the `[cleanup]` button to discard `[done]` files (or just refresh the page)
note that since up2k has to read the file twice, `[🎈 bup]` can be up to 2x faster in extreme cases (if your internet connection is faster than the read-speed of your HDD)
up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
### unpost
@ -399,12 +401,22 @@ you can unpost even if you don't have regular move/delete access, however only f
cut/paste, rename, and delete files/folders (if you have permission)
file selection: click somewhere on the line (not the link itsef), then:
* `space` to toggle
* `up/down` to move
* `shift-up/down` to move-and-select
* `ctrl-shift-up/down` to also scroll
* cut: select some files and `ctrl-x`
* paste: `ctrl-v` in another folder
* rename: `F2`
you can move files across browser tabs (cut in one tab, paste in another)
## batch rename
select some files and press F2 to bring up the rename UI
select some files and press `F2` to bring up the rename UI
![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
@ -486,15 +498,15 @@ add the argument `-e2ts` to also scan/index tags from music files, which brings
## file indexing
file indexing relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volume flags, or a mix of both.
through arguments:
* `-e2d` enables file indexing on upload
* `-e2ds` scans writable folders for new files on startup
* `-e2dsa` scans all mounted volumes (including readonly ones)
* `-e2ds` also scans writable folders for new files on startup
* `-e2dsa` also scans all mounted volumes (including readonly ones)
* `-e2t` enables metadata indexing on upload
* `-e2ts` scans for tags in all files that don't have tags yet
* `-e2tsr` deletes all existing tags, does a full reindex
* `-e2ts` also scans for tags in all files that don't have tags yet
* `-e2tsr` also deletes all existing tags, doing a full reindex
the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
* `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
@ -502,10 +514,11 @@ the same arguments can be set as volume flags, in addition to `d2d` and `d2t` fo
* `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
note:
* the parser currently can't handle `c,e2dsa,e2tsr` so you have to `c,e2dsa:c,e2tsr`
* `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
* the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
to save some time, you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
* initial indexing is way faster, especially when the volume is on a network disk
* makes it impossible to [file-search](#file-search)
* if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
@ -557,7 +570,7 @@ some examples,
## database location
can be stored in-volume (default) or elsewhere
in-volume (`.hist/up2k.db`, default) or somewhere else
copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
@ -579,7 +592,7 @@ set `-e2t` to index tags on upload
if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the `[⚙️] config` pane
`-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
@ -647,10 +660,10 @@ TLDR: yes
| **= feature =** | ie6 | ie9 | ie10 | ie11 | ff 52 | c 49 | iOS | Andr |
* internet explorer 6 to 8 behave the same
* firefox 52 and chrome 49 are the last winxp versions
* `*1` yes, but extremely slow (ie10: 1 MiB/s, ie11: 270 KiB/s)
* firefox 52 and chrome 49 are the final winxp versions
* `*1` yes, but extremely slow (ie10: `1 MiB/s`, ie11: `270 KiB/s`)
* `*2` causes a full-page refresh on each navigation
* `*3` using a wasm decoder which can sometimes get stuck and consumes a bit more power
* `*3` using a wasm decoder which consumes a bit more power
quick summary of more eccentric web-browsers trying to view a directory index:
@ -710,12 +723,25 @@ quick outline of the up2k protocol, see [uploading](#uploading) for the web-clie
* server writes chunks into place based on the hash
* client does another handshake with the hashlist; server replies with OK or a list of chunks to reupload
up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
## why chunk-hashes
a single sha512 would be better right?
this is due to `crypto.subtle` not providing a streaming api (or the option to seed the sha512 hasher with a starting hash)
as a result, the hashes are much less useful than they could have been (search the server by sha512, provide the sha512 in the response http headers, ...)
hashwasm would solve the streaming issue but reduces hashing speed for sha512 (xxh128 does 6 GiB/s), and it would make old browsers and [iphones](https://bugs.webkit.org/show_bug.cgi?id=228552) unsupported
# performance
defaults are good for most cases
defaults are good for most cases, expected speeds are `8 GiB/s` download and `1 GiB/s` up2k-upload with firefox
you can ignore the `cannot efficiently use multiple CPU cores` message, it's very unlikely to be a problem
you can ignore the `cannot efficiently use multiple CPU cores` message, very unlikely to be a problem
below are some tweaks roughly ordered by usefulness:
@ -744,17 +770,11 @@ enable music tags:
* either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
* or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
enable thumbnails of images:
* `Pillow` (requires py2.7 or py3.5+)
enable thumbnails of videos:
* `ffmpeg` and `ffprobe` somewhere in `$PATH`
enable thumbnails of HEIF pictures:
* `pyheif-pillow-opener` (requires Linux or a C compiler)
enable thumbnails of AVIF pictures:
* `pillow-avif-plugin`
enable [thumbnails](#thumbnails) of...
* **images:** `Pillow` (requires py2.7 or py3.5+)
* **videos:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
* **HEIF pictures:** `pyheif-pillow-opener` (requires Linux or a C compiler)
* **AVIF pictures:** `pillow-avif-plugin`
## install recommended deps
@ -791,8 +811,10 @@ if you don't need all the features, you can repack the sfx and save a bunch of s
* `223k` after `./scripts/make-sfx.sh re no-ogv no-cm`
the features you can opt to drop are
* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files
* `cm`/easymde, the "fancy" markdown editor
* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files, saves ~192k
* `cm`/easymde, the "fancy" markdown editor, saves ~92k
* `no-fnt`, source-code-pro, the monospace font, saves ~9k
* `no-dd`, the custom mouse cursor for the media player tray tab, saves ~2k
for the `re`pack to work, first run one of the sfx'es once to unpack it
@ -828,7 +850,7 @@ pip install black bandit pylint flake8 # vscode tooling
## just the sfx
grab the web-dependencies from a previous sfx (unless you need to modify something in those):
first grab the web-dependencies from a previous sfx (unless you need to modify something in those):
```sh
rm -rf copyparty/web/deps
@ -854,8 +876,8 @@ in the `scripts` folder:
* run `make -C deps-docker` to build all dependencies
* `git tag v1.2.3 && git push origin --tags`
* create github release with `make-tgz-release.sh`
* upload to pypi with `make-pypi-release.(sh|bat)`
* create github release with `make-tgz-release.sh`
* create sfx with `make-sfx.sh`

4
scripts/make-sfx.sh
View file

@ -16,10 +16,10 @@ help() { exec cat <<'EOF'
#
# `no-sh` makes just the python sfx, skips the sh/unix sfx
#
# `no-ogv` saves ~500k by removing the opus/vorbis audio codecs
# `no-ogv` saves ~192k by removing the opus/vorbis audio codecs
# (only affects apple devices; everything else has native support)
#
# `no-cm` saves ~90k by removing easymde/codemirror
# `no-cm` saves ~92k by removing easymde/codemirror
# (the fancy markdown editor)
#
# `no-fnt` saves ~9k by removing the source-code-pro font