-1. classic light | 
+
+0. classic dark | 
+2. flat pm-monokai | 
+4. vice | |
+1. classic light |
3. flat light
- | 
+ |
5. hotdog stand |
the classname of the HTML tag is set according to the selected theme, which is used to set colors as css variables ++
@@ -1987,7 +1993,6 @@ see the top of [./copyparty/web/browser.css](./copyparty/web/browser.css) where
if you want to change the fonts, see [./docs/rice/](./docs/rice/)
-
## complete examples
* see [running on windows](./docs/examples/windows.md) for a fancy windows setup
@@ -2028,7 +2033,6 @@ if you want to change the fonts, see [./docs/rice/](./docs/rice/)
* ...with logging to disk
`-lo log/cpp-%Y-%m%d-%H%M%S.txt.xz`
-
## listen on port 80 and 443
become a *real* webserver which people can access by just going to your IP or domain without specifying a port
@@ -2042,25 +2046,27 @@ become a *real* webserver which people can access by just going to your IP or d
* **option 1:** set up a [reverse-proxy](#reverse-proxy) -- this one makes a lot of sense if you're running on a proper headless server, because that way you get real HTTPS too
* **option 2:** NAT to port 3923 -- this is cumbersome since you'll need to do it every time you reboot, and the exact command may depend on your linux distribution:
+
```bash
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 3923
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 3923
```
* **option 3:** disable the [security policy](https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html) which prevents the use of 80 and 443; this is *probably* fine:
- ```
+
+ ```sh
setcap CAP_NET_BIND_SERVICE=+eip $(realpath $(which python))
python copyparty-sfx.py -p 80,443
```
* **option 4:** run copyparty as root (please don't)
-
## reverse-proxy
running copyparty next to other websites hosted on an existing webserver such as nginx, caddy, or apache
you can either:
+
* give copyparty its own domain or subdomain (recommended)
* or do location-based proxying, using `--rp-loc=/stuff` to tell copyparty where it is mounted -- has a slight performance cost and higher chance of bugs
* if copyparty says `incorrect --rp-loc or webserver config; expected vpath starting with [...]` it's likely because the webserver is stripping away the proxy location from the request URLs -- see the `ProxyPass` in the apache example below
@@ -2072,6 +2078,7 @@ when running behind a reverse-proxy (this includes services like cloudflare), it
Note that `--rp-loc` in particular will not work at all unless you configure the above correctly
some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which *could* be a nice speed boost, depending on a lot of factors
+
* **warning:** nginx-QUIC (HTTP/3) is still experimental and can make uploads much slower, so HTTP/1.1 is recommended for now
* depending on server/client, HTTP/1.1 can also be 5x faster than HTTP/2
@@ -2088,7 +2095,6 @@ example webserver / reverse-proxy configs:
* [nginx config](contrib/nginx/copyparty.conf) -- recommended
* [traefik config](contrib/traefik/copyparty.yaml)
-
### real-ip
teaching copyparty how to see client IPs when running behind a reverse-proxy, or a WAF, or another protection service such as cloudflare
@@ -2097,7 +2103,6 @@ if you (and maybe everybody else) keep getting a message that says `thank you fo
for most common setups, there should be a helpful message in the server-log explaining what to do, but see [docs/xff.md](docs/xff.md) if you want to learn more, including a quick hack to **just make it work** (which is **not** recommended, but hey...)
-
### reverse-proxy performance
most reverse-proxies support connecting to copyparty either using uds/unix-sockets (`/dev/shm/party.sock`, faster/recommended) or using tcp (`127.0.0.1`)
@@ -2129,7 +2134,6 @@ in summary, `haproxy > caddy > traefik > nginx > apache > lighttpd`, and use uds
* if these results are bullshit because my config exampels are bad, please submit corrections!
-
## permanent cloudflare tunnel
if you have a domain and want to get your copyparty online real quick, either from your home-PC behind a CGNAT or from a server without an existing [reverse-proxy](#reverse-proxy) setup, one approach is to create a [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/) (formerly "Argo Tunnel")
@@ -2149,7 +2153,6 @@ config file example:
xff-hdr: cf-connecting-ip
```
-
## prometheus
metrics/stats can be enabled at URL `/.cpr/metrics` for grafana / prometheus / etc (openmetrics 1.0.0)
@@ -2171,6 +2174,7 @@ scrape_configs:
```
currently the following metrics are available,
+
* `cpp_uptime_seconds` time since last copyparty restart
* `cpp_boot_unixtime_seconds` same but as an absolute timestamp
* `cpp_active_dl` number of active downloads
@@ -2181,6 +2185,7 @@ currently the following metrics are available,
* `cpp_total_bans` number of IPs banned since last restart
these are available unless `--nos-vst` is specified:
+
* `cpp_db_idle_seconds` time since last database activity (upload/rename/delete)
* `cpp_db_act_seconds` same but as an absolute timestamp
* `cpp_idle_vols` number of volumes which are idle / ready
@@ -2191,10 +2196,12 @@ these are available unless `--nos-vst` is specified:
* `cpp_mtpq_files` number of files queued for plugin-based analysis
and these are available per-volume only:
+
* `cpp_disk_size_bytes` total HDD size
* `cpp_disk_free_bytes` free HDD space
and these are per-volume and `total`:
+
* `cpp_vol_bytes` size of all files in volume
* `cpp_vol_files` number of files
* `cpp_dupe_bytes` disk space presumably saved by deduplication
@@ -2202,9 +2209,11 @@ and these are per-volume and `total`:
* `cpp_unf_bytes` currently unfinished / incoming uploads
some of the metrics have additional requirements to function correctly,
+
* `cpp_vol_*` requires either the `e2ds` volflag or `-e2dsa` global-option
the following options are available to disable some of the metrics:
+
* `--nos-hdd` disables `cpp_disk_*` which can prevent spinning up HDDs
* `--nos-vol` disables `cpp_vol_*` which reduces server startup time
* `--nos-vst` disables volume state, reducing the worst-case prometheus query time by 0.5 sec
@@ -2213,12 +2222,10 @@ the following options are available to disable some of the metrics:
note: the following metrics are counted incorrectly if multiprocessing is enabled with `-j`: `cpp_http_conns`, `cpp_http_reqs`, `cpp_sus_reqs`, `cpp_active_bans`, `cpp_total_bans`
-
## other extremely specific features
you'll never find a use for these:
-
### custom mimetypes
change the association of a file extension
@@ -2235,7 +2242,6 @@ in a config file, this is the same as:
run copyparty with `--mimes` to list all the default mappings
-
### GDPR compliance
imagine using copyparty professionally... **TINLA/IANAL; EU laws are hella confusing**
@@ -2247,7 +2253,6 @@ imagine using copyparty professionally... **TINLA/IANAL; EU laws are hella conf
* if you actually *are* a lawyer then I'm open for feedback, would be fun
-
### feature chickenbits
buggy feature? rip it out by setting any of the following environment variables to disable its associated bell or whistle,
@@ -2266,7 +2271,6 @@ buggy feature? rip it out by setting any of the following environment variables
example: `PRTY_NO_IFADDR=1 python3 copyparty-sfx.py`
-
### feature beefybits
force-enable features with known issues on your OS/env by setting any of the following environment variables, also affectionately known as `fuckitbits` or `hail-mary-bits`
@@ -2276,28 +2280,24 @@ force-enable features with known issues on your OS/env by setting any of the fo
| `PRTY_FORCE_MP` | force-enable multiprocessing (real multithreading) on MacOS and other broken platforms |
| `PRTY_FORCE_MAGIC` | use [magic](https://pypi.org/project/python-magic/) on Windows (you will segfault) |
-
# packages
the party might be closer than you think
if your distro/OS is not mentioned below, there might be some hints in the [«on servers»](#on-servers) section
-
## arch package
`pacman -S copyparty` (in [arch linux extra](https://archlinux.org/packages/extra/any/copyparty/))
it comes with a [systemd service](./contrib/systemd/copyparty@.service) as well as a [user service](./contrib/systemd/copyparty-user.service), and expects to find a [config file](./contrib/systemd/copyparty.example.conf) in `/etc/copyparty/copyparty.conf` or `~/.config/copyparty/copyparty.conf`
-after installing, start either the system service or the user service and navigate to http://127.0.0.1:3923 for further instructions (unless you already edited the config files, in which case you are good to go, probably)
-
+after installing, start either the system service or the user service and navigate to for further instructions (unless you already edited the config files, in which case you are good to go, probably)
## fedora package
does not exist yet; there are rumours that it is being packaged! keep an eye on this space...
-
## nix package
`nix profile install github:9001/copyparty`
@@ -2308,7 +2308,6 @@ some recommended dependencies are enabled by default; [override the package](htt
`ffmpeg-full` was chosen over `ffmpeg-headless` mainly because we need `withWebp` (and `withOpenmpt` is also nice) and being able to use a cached build felt more important than optimizing for size at the time -- PRs welcome if you disagree 👍
-
## nixos module
for this setup, you will need a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of NixOS.
@@ -2339,6 +2338,7 @@ for this setup, you will need a [flake-enabled](https://nixos.wiki/wiki/Flakes)
```
copyparty on NixOS is configured via `services.copyparty` options, for example:
+
```nix
services.copyparty = {
enable = true;
@@ -2401,7 +2401,6 @@ services.copyparty = {
the passwordFile at /run/keys/copyparty/ could for example be generated by [agenix](https://github.com/ryantm/agenix), or you could just dump it in the nix store instead if that's acceptable
-
# browser support
TLDR: yes
@@ -2446,7 +2445,7 @@ quick summary of more eccentric web-browsers trying to view a directory index:
| **links** (2.21/macports) | can browse, login, upload/mkdir/msg |
| **lynx** (2.8.9/macports) | can browse, login, upload/mkdir/msg |
| **w3m** (0.5.3/macports) | can browse, login, upload at 100kB/s, mkdir/msg |
-| **netsurf** (3.10/arch) | is basically ie6 with much better css (javascript has almost no effect) |
+| **netsurf** (3.10/arch) | is basically ie6 with much better css (javascript has almost no effect) |
| **opera** (11.60/winxp) | OK: thumbnails, image-viewer, zip-selection, rename/cut/paste. NG: up2k, navpane, markdown, audio |
| **ie4** and **netscape** 4.0 | can browse, upload with `?b=u`, auth with `&pw=wark` |
| **ncsa mosaic** 2.7 | does not get a pass, [pic1](https://user-images.githubusercontent.com/241032/174189227-ae816026-cf6f-4be5-a26e-1b3b072c1b2f.png) - [pic2](https://user-images.githubusercontent.com/241032/174189225-5651c059-5152-46e9-ac26-7e98e497901b.png) |
@@ -2455,8 +2454,7 @@ quick summary of more eccentric web-browsers trying to view a directory index:
| **nintendo 3ds** | can browse, upload, view thumbnails (thx bnjmn) |
| **Nintendo Wii (Opera 9.0 "Internet Channel")** | can browse, can't upload or download (no local storage), can view images - works best with `?b=u`, default view broken |
-
-
+
# client examples
@@ -2506,8 +2504,10 @@ interact with copyparty using non-browser clients
copyparty returns a truncated sha512sum of your PUT/POST as base64; you can generate the same checksum locally to verify uploads:
- b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
- b512 `s; attempts are made to prevent scripts from executing (unless `-emp` is specified) but this is not 100% bulletproof, so setting the `nohtml` volflag is still the safest choice
* or eliminate the problem entirely by only giving write-access to trustworthy people :^)
-
## cors
cross-site request config
by default, except for `GET` and `HEAD` operations, all requests must either:
+
* not contain an `Origin` header at all
* or have an `Origin` matching the server domain
* or the header `PW` with your password as value
cors can be configured with `--acao` and `--acam`, or the protections entirely disabled with `--allow-csrf`
-
## filekeys
prevent filename bruteforcing
@@ -2711,11 +2702,11 @@ volflag `dks` lets people enter subfolders as well, and also enables download-as
if you enable dirkeys, it is probably a good idea to enable filekeys too, otherwise it will be impossible to hotlink files from a folder which was accessed using a dirkey
dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and have a few limitations:
+
* the key does not change if the contents of the folder is modified
* if you need a new dirkey, either change the salt or rename the folder
* linking to a textfile (so it opens in the textfile viewer) is not possible if recipient doesn't have read-access
-
## password hashing
you can hash passwords before putting them into config files / providing them as arguments; see `--help-pwhash` for all the details
@@ -2727,12 +2718,12 @@ optionally also specify `--ah-cli` to enter an interactive mode where it will ha
the default configs take about 0.4 sec and 256 MiB RAM to process a new password on a decent laptop
when generating hashes using `--ah-cli` for docker or systemd services, make sure it is using the same `--ah-salt` by:
+
* inspecting the generated salt using `--show-ah-salt` in copyparty service configuration
* setting the same `--ah-salt` in both environments
> ⚠️ if you have enabled `--usernames` then provide the password as `username:password` when hashing it, for example `ed:hunter2`
-
## https
both HTTP and HTTPS are accepted by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
@@ -2740,11 +2731,11 @@ both HTTP and HTTPS are accepted by default, but letting a [reverse proxy](#rev
copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3
if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, copyparty will automatically create a CA and server-cert on startup
+
* the certs are written to `--crt-dir` for distribution, see `--help` for the other `--crt` options
* this will be a self-signed certificate so you must install your `ca.pem` into all your browsers/devices
* if you want to avoid the hassle of distributing certs manually, please consider using a reverse proxy
-
# recovering from crashes
## client crashes
@@ -2764,17 +2755,15 @@ however you can hit `F12` in the up2k tab and use the devtools to see how far yo
* send the list of filenames to copyparty for safekeeping:
`await fetch('/inc', {method:'PUT', body:JSON.stringify(ng,null,1)})`
-
# HTTP API
see [devnotes](./docs/devnotes.md#http-api)
-
# dependencies
mandatory deps:
-* `jinja2` (is built into the SFX)
+* `jinja2` (is built into the SFX)
## optional dependencies
@@ -2783,14 +2772,17 @@ install these to enable bonus features
enable [hashed passwords](#password-hashing) in config: `argon2-cffi`
enable [ftp-server](#ftp-server):
+
* for just plaintext FTP, `pyftpdlib` (is built into the SFX)
* with TLS encryption, `pyftpdlib pyopenssl`
enable [music tags](#metadata-from-audio-files):
+
* either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
* or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
enable [thumbnails](#thumbnails) of...
+
* **images:** `Pillow` and/or `pyvips` and/or `ffmpeg` (requires py2.7 or py3.5+)
* **videos/audio:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
* **HEIF pictures:** `pyvips` or `ffmpeg` or `pyheif-pillow-opener` (requires Linux or a C compiler)
@@ -2806,7 +2798,6 @@ enable [smb](#smb-server) support (**not** recommended): `impacket==0.12.0`
to install FFmpeg on Windows, grab [a recent build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) -- you need `ffmpeg.exe` and `ffprobe.exe` from inside the `bin` folder; copy them into `C:\Windows\System32` or any other folder that's in your `%PATH%`
-
### dependency chickenbits
prevent loading an optional dependency , for example if:
@@ -2839,14 +2830,12 @@ example: `PRTY_NO_PIL=1 python3 copyparty-sfx.py`
* `PRTY_NO_VIPS` saves ram and startup time
* python2.7 on windows: `PRTY_NO_FFMPEG` + `PRTY_NO_FFPROBE` saves startup time
-
## optional gpl stuff
some bundled tools have copyleft dependencies, see [./bin/#mtag](bin/#mtag)
these are standalone programs and will never be imported / evaluated by copyparty, and must be enabled through `-mtp` configs
-
# sfx
the self-contained "binary" (recommended!) [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
@@ -2855,7 +2844,6 @@ if you only need english, [copyparty-en.py](https://github.com/9001/copyparty/re
you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
-
## copyparty.exe
download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
@@ -2877,7 +2865,6 @@ meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/d
then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) every once ina while if you can afford the size
-
## zipapp
another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) has less features, is slow, requires python 3.7 or newer, worse compression, and more importantly is unable to benefit from more recent versions of jinja2 and such (which makes it less secure)... lots of drawbacks with this one really -- but, unlike the sfx, it is a completely normal zipfile which does not unpack any temporary files to disk, so it *may* just work if the regular sfx fails to start because the computer is messed up in certain funky ways, so it's worth a shot if all else fails
@@ -2885,12 +2872,13 @@ another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty
run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails
it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
-* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
+* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
# install on android
install [Termux](https://termux.com/) + its companion app `Termux:API` (see [ocv.me/termux](https://ocv.me/termux/)) and then copy-paste this into Termux (long-tap) all at once:
+
```sh
yes | pkg upgrade && termux-setup-storage && yes | pkg install python termux-api && python -m ensurepip && python -m pip install --user -U copyparty && { grep -qE 'PATH=.*\.local/bin' ~/.bashrc 2>/dev/null || { echo 'PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc && . ~/.bashrc; }; }
echo $?
@@ -2902,25 +2890,25 @@ if you want thumbnails (photos+videos) and you're okay with spending another 132
* or if you want to use `vips` for photo-thumbs instead, `pkg install libvips && python -m pip install --user -U wheel && python -m pip install --user -U pyvips && (cd /data/data/com.termux/files/usr/lib/; ln -s libgobject-2.0.so{,.0}; ln -s libvips.so{,.42})`
-
# reporting bugs
ideas for context to include, and where to submit them
please get in touch using any of the following URLs:
-* https://github.com/9001/copyparty/ **(primary)**
-* https://gitlab.com/9001/copyparty/ *(mirror)*
-* https://codeberg.org/9001/copyparty *(mirror)*
+
+* **(primary)**
+* *(mirror)*
+* *(mirror)*
in general, commandline arguments (and config file if any)
if something broke during an upload (replacing FILENAME with a part of the filename that broke):
-```
+
+```sh
journalctl -aS '48 hour ago' -u copyparty | grep -C10 FILENAME | tee bug.log
```
-if there's a wall of base64 in the log (thread stacks) then please include that, especially if you run into something freezing up or getting stuck, for example `OperationalError('database is locked')` -- alternatively you can visit `/?stack` to see the stacks live, so http://127.0.0.1:3923/?stack for example
-
+if there's a wall of base64 in the log (thread stacks) then please include that, especially if you run into something freezing up or getting stuck, for example `OperationalError('database is locked')` -- alternatively you can visit `/?stack` to see the stacks live, so for example
# devnotes
@@ -2929,4 +2917,3 @@ for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
specifically you may want to [build the sfx](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#just-the-sfx) or [build from scratch](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#build-from-scratch)
see [./docs/TODO.md](./docs/TODO.md) for planned features / fixes / changes
-
|
-0. classic dark
-2. flat pm-monokai
-4. vice
-1. classic light
+
+