Compare commits

..

No commits in common. "hovudstraum" and "v1.20.18" have entirely different histories.

109 changed files with 328 additions and 1283 deletions

1
.gitattributes vendored
View file

@ -4,5 +4,4 @@
*.png binary
*.gif binary
*.ico binary
*.gz binary

View file

@ -76,7 +76,6 @@ built in Norway 🇳🇴 with contributions from [not-norway](https://github.com
* [connecting to webdav from windows](#connecting-to-webdav-from-windows) - using the GUI
* [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
* [smb server](#smb-server) - unsafe, slow, not recommended for wan
* [wopi server](#wopi-server) - edit office documents in the web-ui
* [browser ux](#browser-ux) - tweaking the ui
* [opengraph](#opengraph) - discord and social-media embeds
* [file deduplication](#file-deduplication) - enable symlink-based upload deduplication
@ -84,7 +83,6 @@ built in Norway 🇳🇴 with contributions from [not-norway](https://github.com
* [exclude-patterns](#exclude-patterns) - to save some time
* [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
* [periodic rescan](#periodic-rescan) - filesystem monitoring
* [u2sz](#u2sz) - configure upload-chunking
* [upload rules](#upload-rules) - set upload rules using volflags
* [compress uploads](#compress-uploads) - files can be autocompressed on upload
* [chmod and chown](#chmod-and-chown) - per-volume filesystem-permissions and ownership
@ -138,7 +136,6 @@ built in Norway 🇳🇴 with contributions from [not-norway](https://github.com
* [client-side](#client-side) - when uploading files
* [security](#security) - there is a [discord server](https://discord.gg/25J8CdTT6G) with announcements
* [gotchas](#gotchas) - behavior that might be unexpected
* [csp nonce](#csp-nonce) - unauthorized javascript is blocked
* [cors](#cors) - cross-site request config
* [filekeys](#filekeys) - prevent filename bruteforcing
* [dirkeys](#dirkeys) - share specific folders in a volume
@ -776,7 +773,7 @@ press `g` or `田` to toggle grid-view instead of the file listing and `t` togg
![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
* Pillow is 3x faster (and safer) than FFmpeg
* pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
* disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
* for installing FFmpeg on windows, see [optional dependencies](#optional-dependencies)
@ -785,7 +782,6 @@ audio files are converted into spectrograms using FFmpeg unless you `--no-athumb
images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
* the order is significant, so if both `cover.png` and `folder.jpg` exist in a folder, it will pick the first matching `--th-covers` entry (`folder.jpg`)
* and, if you enable [file indexing](#file-indexing), it will also try those names as dotfiles (`.folder.jpg` and so), and then fallback on the first picture in the folder (if it has any pictures at all)
* disable folderthumbs with `--th-covers no`
enabling `multiselect` lets you click files to select them, and then shift-click another file for range-select
* `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
@ -923,7 +919,7 @@ note that since up2k has to read each file twice, `[🎈] bup` can *theoreticall
if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option [--u2sz](#u2sz)
if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option `--u2sz` (the default is 96 MiB to support Cloudflare)
if you want to replace existing files on the server with new uploads by default, run with `--u2ow 2` (only works if users have the delete-permission, and can still be disabled with `🛡️` in the UI)
@ -1341,8 +1337,7 @@ for the above example to work, add the commandline argument `-e2ts` to also scan
# server config
using arguments or config files, or a mix of both:
* using a config-file (`-c some.conf`) is best, easier to read/maintain; see [./docs/example.conf](docs/example.conf)
* for complex and intricate setups, see how [./docs/example2.conf](docs/example2.conf) includes additional config-files/folders with `% copyparty.d` which loads [./docs/copyparty.d/some.conf](docs/copyparty.d/some.conf)
* config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf) and [./docs/example2.conf](docs/example2.conf)
* `kill -s USR1` (same as `systemctl reload copyparty`) to reload accounts and volumes from config files without restarting
* or click the `[reload cfg]` button in the control-panel if the user has `a`/admin in any volume
* changes to the `[global]` config section requires a restart to take effect
@ -1634,15 +1629,6 @@ authenticate with one of the following:
* username `$password`, password `k`
## wopi server
edit office documents in the web-ui
needs a wopi client to work, and the only supported wopi client is a selfhosted collabora online server; eventually onlyoffice will probably also be supported: [issue #1574](https://github.com/9001/copyparty/issues/1574)
no further documentation or examples yet, [issue #1575](https://github.com/9001/copyparty/issues/1575)
## browser ux
tweaking the ui
@ -1827,23 +1813,6 @@ config file example:
```
## u2sz
configure upload-chunking; the default `64,96` is optimal for nearly every scenario, however:
* if you are seeing the message `bad server config; see "u2sz" in readme` then you have a proxy/CDN/whatever with a `max-request-body-size` which is too small, and you must either increase the limit in the reverseproxy config to at least 96 MiB (recommended) or reduce the target chunksize in copyparty (continue reading)
the global-option `--u2sz` is two numbers, `target,max` MiB of each chunk to send, aiming for `target` MiB and allowing up-to-and-including `max` MiB [depending on filesize](./docs/devnotes.md#list-of-chunk-sizes), and if you only specify one number then it sets that for both
cloudflare max is 96 MiB, but if you have a reverseproxy (or CDN/WAF/whatever) which is more restrictive, then try some of the following:
* `u2sz: 32` is still acceptable performance in most cases
* `u2sz: 8` is the lowest recommended value; anything lower will reduce performance over long physical distances and/or shitty routes
* `u2sz: 1` is the absolute minimum, and will be slow in many cases
the setting can also be changed in the web-UI; the textbox next to `date-chk` in the settings tab
## upload rules
set upload rules using volflags, some examples:
@ -2606,7 +2575,6 @@ buggy feature? rip it out by setting any of the following environment variables
| `PRTY_NO_IPV6` | disable some ipv6 support (should not be necessary since windows 2000) |
| `PRTY_NO_LZMA` | disable streaming xz compression of incoming uploads |
| `PRTY_NO_MP` | disable all use of the python `multiprocessing` module (actual multithreading, cpu-count for parsers/thumbnailers) |
| `PRTY_NO_UNIDATA` | disable loading the unicodedata c-extension; saves 400k ram, breaks `--srch-nfkc` |
| `PRTY_NO_SQLITE` | disable all database-related functionality (file indexing, metadata indexing, most file deduplication logic) |
| `PRTY_NO_TLS` | disable native HTTPS support; if you still want to accept HTTPS connections then TLS must now be terminated by a reverse-proxy |
| `PRTY_NO_TPOKE` | disable systemd-tmpfilesd avoider |
@ -2682,6 +2650,8 @@ requires a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of nix
some recommended dependencies are enabled by default; [override the package](https://github.com/9001/copyparty/blob/hovudstraum/contrib/package/nix/copyparty/default.nix#L3-L22) if you want to add/remove some features/deps
`ffmpeg-full` was chosen over `ffmpeg-headless` mainly because we need `withWebp` (and `withOpenmpt` is also nice) and being able to use a cached build felt more important than optimizing for size at the time -- PRs welcome if you disagree 👍
## nixos module
@ -3126,23 +3096,6 @@ behavior that might be unexpected
* or eliminate the problem entirely by only giving write-access to trustworthy people :^)
## csp nonce
unauthorized javascript is blocked by means of the default [content-security-policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP) which enforces "script nonces", to help prevent XSS and similar
usually this Just Works and is correctly configured by default; you can stop reading this section and move on, nothing to see here
but if you are the unlucky 0.1% with a hosting-setup funky enough for this to not work as intended, then you will also be seeing this message: `Looks like javascript is broken; see "csp nonce" in the readme`
there is two possible reasons why this might happen:
1. your browser is simply too old to understand the javascript, which is not likely, because just about every browser is supported, even internet explorer 11
2. something in your hosting-setup is tampering with the responses from copyparty and destroying this security feature; the culprit is probably a misconfigured reverseproxy, a CDN, a WAF, or some other mitm, but a popular offender is **"cloudflare rocket loader"**
you can try to **temporarily** set global-options `--csp-ui no` and `--csp-dl no` and restart copyparty to check whether it is due to reason #2. Remember to refresh the webbrowser with CTRL-SHIFT-R to check properly. If that makes the problem go away, then turn them back off (**never** keep them enabled), restart copyparty, open the `Network` tab in the webbrowser, and refresh the website; you should see the `content-security-policy` header including a nonce, for example `nonce-BxcJbvdu1z3S/ycPBjVWuA==` which should also appear inside the HTML of that response like this: `<script nonce="BxcJbvdu1z3S/ycPBjVWuA==">` -- if it doesn't then that's the problem; something in your hosting stack is dangerously misconfigured
## cors
cross-site request config
@ -3286,7 +3239,7 @@ enable sending [zeromq messages](#zeromq) from event-hooks: `pyzmq`
enable [smb](#smb-server) support (**not** recommended): `impacket==0.13.0`
`pyvips` adds support for some additional image formats, but can use extreme amounts of RAM
`pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram
* to install `pyvips` on Linux: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
* to install `pyvips` on windows: `pip install --user -U "pyvips[binary]"`

View file

@ -626,7 +626,7 @@ class Gateway(object):
"st_atime": ts,
"st_mtime": ts,
"st_ctime": ts,
"st_blocks": -int(-sz // 512),
"st_blocks": int((sz + 511) / 512),
}
def stat_file(self, ts, sz):
@ -638,7 +638,7 @@ class Gateway(object):
"st_atime": ts,
"st_mtime": ts,
"st_ctime": ts,
"st_blocks": -int(-sz // 512),
"st_blocks": int((sz + 511) / 512),
}

View file

@ -1,54 +0,0 @@
# thumbnail extractors
extract (or generate) thumbnails for custom file formats or override default thumbnail extraction for standard media files
## usage
* load plugins with `--th-extract baz=~/dev/copyparty/bin/thumbs/randomcolor.py`
* `baz` is the file extension (may be comma-separated list of extensions)
* add multiple different extractor plugins by repeating the `--th-extract` argument
## api
### in
each plugin must define a function `main(abspath, **kwargs)`:
* `abspath` is the path to a file to extract thumbnail for
* `kwargs` currently receives these keyword arguments:
* `vn` is the VFS which contains the requested file
* `th_srv` is an instance of [copyparty/th_srv](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/th_srv.py)
### out
the `main()` function must
* return `None` or raise an `Exception` in case it is unable to extract thumbnail
* return tuple `fmt, stream, offset, whence, size` otherwise:
* `fmt: str` format (filename extension) of the thumbnail; should be one of the image formats supported by copyparty (see `--th-r-*` options)
* `stream: IO[bytes]` binary file-like object supporting `seek()`, `read()`, `close()` methods; should contain extracted thumbnail image
* `offset: int, whence: int` arguments to `stream.seek()`; copyparty will call `stream.seek(offset, whence)` once before starting reading the stream contents
* `size: int | None` number of bytes to read from the `stream`; if `size` is negative or `None`, copyparty will read until the end of the stream
### notes
it is possible to extract or generate custom thumbnails for standard media files too,
e.g. run copyparty with `--th-extract mp3,m4a,aac,flac,opus=~/dev/copyparty/bin/thumbs/my_custom_mp3_th_extractor.py` and do your magic with audio file thumbnails
in case copyparty doesn't like the `fmt` option, it will `close()` the `stream` without ever calling `seek()` or `read()`
if your scenario involves heavy computation or i/o, it is better to defer the heavy parts until the first `seek()` or `read()` to avoid redundant work in case the result gets rejected based on `fmt` value
if you provide custom `stream` object:
copyparty performs buffered reading, so expect multiple `read()` calls, respect `size` argument in `read(size)` calls, and maintain correct seek position
## examples
* [randomcolor](https://github.com/9001/copyparty/blob/hovudstraum/bin/thumbs/randomcolor.py) generates a random .gif image
## some other known plugins seen on the internets
* [fpkg_thumb](https://github.com/kamaeff/copyparty-dumb-fpkgi-handler/blob/master/fpkg_thumb.py) extracts cover images from playstation4 software installation packages ("pkg" and "fpkg" files)

View file

@ -1,29 +0,0 @@
import io
import os
import struct
from base64 import b64decode
def create_gif() -> bytes:
head = r"R0lGODdhIAAgAIAB"
tail = r"AAD/LAAAAAAgACAAAAIehI+py+0Po5y02ouz3rz7D4biSJbmiabqyrbuC5sFADs="
#color = struct.pack(">L", random.randrange(0xffffff)) # wow, way too stable
color = b"\x00" + os.urandom(3)
return b64decode(head) + color + b64decode(tail)
def main(abspath, **ka) -> Optional[tuple[str, IO[bytes], int, int, Optional[int]]]:
print("hello from randomcolor.py pretending to extract a thumb from " + abspath)
# need to return a file-like object
gif_bytes = create_gif()
file_obj = io.BytesIO(gif_bytes)
# where copyparty should seek to in the "file" to start reading from
seek_offset = 0
seek_whence = 0
filesize = None # auto; read until end
# "gif" is the file-extension to decode the file as
return "gif", file_obj, seek_offset, seek_whence, filesize

View file

@ -1,5 +1,5 @@
#!/usr/bin/env python3
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
S_VERSION = "2.22"
S_BUILD_DT = "2026-06-13"
@ -398,7 +398,7 @@ class MTHash(object):
self.csz = chunksz
chunks = {}
nchunks = -int(-fsz // chunksz)
nchunks = int(math.ceil(fsz / chunksz))
for nch in range(nchunks):
self.work_q.put(nch)
@ -725,7 +725,7 @@ def up2k_chunksize(filesize):
stepsize = 512 * 1024
while True:
for mul in [1, 2]:
nchunks = -int(-filesize // chunksize)
nchunks = math.ceil(filesize * 1.0 / chunksize)
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
return chunksize
@ -1576,7 +1576,7 @@ NOTE: if server has --usernames enabled, then password is "username:password"
ap.add_argument("url", type=unicode, help="server url, including destination folder")
ap.add_argument("files", type=files_decoder, nargs="+", help="files and/or folders to process")
ap.add_argument("-v", action="store_true", help="verbose")
ap.add_argument("-a", metavar="PASSWD", default="", help="password (or $filepath) for copyparty (is sent in header 'PW'), default is env.var U2C_PW")
ap.add_argument("-a", metavar="PASSWD", default="", help="password (or $filepath) for copyparty (is sent in header 'PW')")
ap.add_argument("--ba", metavar="PASSWD", default="", help="password (or $filepath) for basic-auth (usually not necessary)")
ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
ap.add_argument("-x", type=unicode, metavar="REGEX", action="append", help="skip file if filesystem-abspath matches REGEX (option can be repeated), example: '.*/\\.hist/.*'")
@ -1729,9 +1729,6 @@ NOTE: if server has --usernames enabled, then password is "username:password"
with open(zs[1:], "rb") as f:
setattr(ar, k, f.read().decode("utf-8").strip())
if not ar.a:
ar.a = os.environ.get("U2C_PW", "")
if ar.ba:
ar.ba = "Basic " + base64.b64encode(ar.ba.encode("utf-8")).decode("utf-8")

View file

@ -17,12 +17,8 @@ defaults
listen foo1
bind *:8081
http-request add-header X-Forwarded-Proto https if { ssl_fc }
http-request add-header X-Forwarded-Proto http unless { ssl_fc }
server srv1 127.0.0.1:3923 maxconn 512
listen foo2
bind *:8082
http-request add-header X-Forwarded-Proto https if { ssl_fc }
http-request add-header X-Forwarded-Proto http unless { ssl_fc }
server srv1 /dev/shm/party.sock maxconn 512

View file

@ -285,7 +285,6 @@ in
environment = {
PYTHONUNBUFFERED = "true";
NIX_SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
XDG_CONFIG_HOME = externalStateDir;
};

View file

@ -3,7 +3,7 @@
# NOTE: You generally shouldn't use this PKGBUILD on Arch, as it is mainly for testing purposes. Install copyparty using pacman instead.
pkgname=copyparty
pkgver="1.20.20"
pkgver="1.20.17"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@ -16,7 +16,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
"python-mutagen: music tags (alternative)"
"python-paramiko: sftp server",
"python-pillow: thumbnails for images"
"python-pyvips: thumbnails for images (eats ram, prefer pillow)"
"python-pyvips: thumbnails for images (higher quality, faster, uses more ram)"
"libkeyfinder: detection of musical keys"
"python-pyopenssl: ftps functionality"
"python-pyzmq: send zeromq messages from event-hooks"
@ -24,7 +24,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}/copyparty.conf" )
sha256sums=("a31f9e4a43e56608765e108353140784d50181c37e705e9ce8ed9f3899f57dfa")
sha256sums=("949553aeffea942a0ddabd3e12d28a643fea862d78930bb36e0accc66c4c9475")
build() {
cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"

View file

@ -2,7 +2,7 @@
pkgname=copyparty
pkgver=1.20.20
pkgver=1.20.17
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("/etc/${pkgname}.d/init" )
sha256sums=("a31f9e4a43e56608765e108353140784d50181c37e705e9ce8ed9f3899f57dfa")
sha256sums=("949553aeffea942a0ddabd3e12d28a643fea862d78930bb36e0accc66c4c9475")
build() {
cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"

View file

@ -30,7 +30,7 @@
# create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
withThumbnails ? true,
# create thumbnails with PyVIPS; uses more memory, pillow is preferred
# create thumbnails with PyVIPS; even faster, uses more memory
# -- can be combined with Pillow to support more filetypes
withFastThumbnails ? false,

View file

@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.20.20/copyparty-1.20.20.tar.gz",
"version": "1.20.20",
"hash": "sha256-ox+eSkPlZgh2XhCDUxQHhNUBgcN+cF6c6O2fOJn1ffo="
"url": "https://github.com/9001/copyparty/releases/download/v1.20.17/copyparty-1.20.17.tar.gz",
"version": "1.20.17",
"hash": "sha256-lJVTrv/qlCoN2r0+EtKKZD/qhi14kwuzbgrMxmxMlHU="
}

View file

@ -16,7 +16,7 @@ let
withMagic = true;
};
call = attrs: final.python3.pkgs.callPackage ./copyparty ({ ffmpeg = final.ffmpeg-headless; } // attrs);
call = attrs: final.python3.pkgs.callPackage ./copyparty ({ ffmpeg = final.ffmpeg-full; } // attrs);
in
{
copyparty = call { stable = true; };

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import platform
@ -40,9 +40,6 @@ VT100 = "--ansi" in sys.argv or (
)
# introduced in anniversary update
zs = os.environ.get("PRTY_CONFIG", "")
CFG_DEF = [zs] if zs else []
ANYWIN = WINDOWS or sys.platform in ["msys", "cygwin"]
MACOS = platform.system() == "Darwin"
@ -138,7 +135,6 @@ web/tl/vie.js
web/ui.css
web/up2k.js
web/util.js
web/wopi.html
"""
RES = set(zs.strip().split("\n"))
RESM = {

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
"""copyparty: http file sharing hub (py2/py3)"""
__author__ = "ed <copyparty@ocv.me>"
@ -23,7 +23,6 @@ import uuid
from .__init__ import (
ANYWIN,
CFG_DEF,
CORES,
EXE,
MACOS,
@ -64,7 +63,6 @@ from .util import (
URL_BUG,
URL_PRJ,
Daemon,
absreal,
align_tab,
b64enc,
ctypes,
@ -107,6 +105,10 @@ except:
u = unicode
zsid = uuid.uuid4().urn[4:]
CFG_DEF = [os.environ.get("PRTY_CONFIG", "")]
if not CFG_DEF[0]:
CFG_DEF.pop()
FULL_HELP = os.environ.get("PRTY_FULL_HELP", "")
@ -777,54 +779,6 @@ def get_sects():
).rstrip()
+ build_flags_desc(),
],
[
"thumb-ex",
"extract thumbnails from custom file formats with python scripts",
dedent(
"""
copyparty is able to extract thumbnails from various media formats:
images, audio, video, ebooks
if you want to extract thumbnails from unsupported file formats,
you can load a plugin, which extracts image data from custom formats
load the plugin using --args; for example \033[36m
--tx-extract iso,mdf,dmg=~/party-thumb-extractors/th_diskimg.py
\033[0m
the file must define the function \033[35mmain(abspath, **kwargs)\033[0m:
\033[35mabspath\033[0m: path to the file to extract thumbnail from
\033[35mkwargs\033[0m currently have:
\033[35mvn\033[0m: the VFS which contains the requested file
\033[35mrth_srv\033[0m: the copyparty ThumbSrv instance
if `main` couldn't extract a thumbnail, it must return None;
otherwise the return value must be a tuple:
\033[35mreturn fmt, stream, offset, whence, size\033[0m
> \033[32mfmt\033[0m: thumbnail format, should be one of the formats
supported by copyparty; e.g. "png", "jpg"
> \033[32mstream\033[0m: binary file-like object containing the thumbnail,
supporting seek(), read(), and close() methods
> \033[32moffset, whence\033[0m: arguments to seek() method;
stream.seek(offset, whence) will be called once before copyparty
starts reading from the stream
> \033[32msize\033[0m: number of bytes to read ftom the stream;
if negative, copyparty will read through the end of the stream;
you can also supply extractors for standard formats to override default
thumbnail extraction behaviour; for example \033[36m
--tx-extract mp3=~/party-thumb-extractors/th_mp3.py
\033[0mwill let you extract or generate custom thumbnails for mp3 files
\033[1;35mPS!\033[0m the folder that contains the python file should ideally
not contain many other python files, and especially nothing
with filenames that overlap with modules used by copyparty
"""
),
],
[
"handlers",
"use plugins to handle certain events",
@ -1088,12 +1042,7 @@ def get_sects():
.1 = when necessary, append a dot followed by a single digit
.1! = counter is always added, even when not necessary
-3 = a hyphen followed by three-digit counter
no = disable counter; append to existing file
del = delete existing logfile and create new
over = overwrite existing logfile
append is not possible for .xz-compressed logfiles;
if logfile is *.xz and rlo is no then rlo will be .1
no = disable counter; overwrite existing logfile
"""
),
],
@ -1380,11 +1329,11 @@ def add_upload(ap):
ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
ap2.add_argument("--df", metavar="GiB", type=u, default="0", help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests; assumes gigabytes unless a unit suffix is given: [\033[32m256m\033[0m], [\033[32m4\033[0m], [\033[32m2T\033[0m] (volflag=df)")
ap2.add_argument("--sparse", metavar="MiB", type=int, default=1, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
ap2.add_argument("--nosubtle", metavar="N", type=int, default=0, help="when to use a wasm-hasher instead of the browser's builtin; faster on chrome, but buggy in older chrome versions. [\033[32m0\033[0m] = only when necessary (non-https), [\033[32m1\033[0m] = always (all browsers), [\033[32m2\033[0m] = always on chrome/firefox, [\033[32m3\033[0m] = always on chrome, [\033[32mN\033[0m] = chrome-version N and newer (recommendation: 137)")
ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good when latency is low (same-country), 2~4 for android-clients, 2~6 for cross-atlantic. Max is 6 in most browsers. Big values increase network-speed but may reduce HDD-speed")
ap2.add_argument("--u2sz", metavar="N,N", type=u, default="64,96", help="web-client: default upload chunksize in MiB, either \033[33mtarget,max\033[0m or just \033[33mtarget\033[0m (sets both); this is \033[33mtarget\033[0m in the web-UI. Each HTTP POST will aim for \033[33mtarget\033[0m and never exceed \033[33mmax\033[0m. Cloudflare max is 96. Minimum is [\033[32m1\033[0m] and will be slow. Lowest recommended is [\033[32m8\033[0m]. Big values are good for cross-atlantic but may increase HDD fragmentation on some FS. Disable this optimization with [\033[32mno\033[0m]")
ap2.add_argument("--u2sz", metavar="N,N,N", type=u, default="1,64,96", help="web-client: default upload chunksize (MiB); sets \033[33mmin,default,max\033[0m in the settings gui. Each HTTP POST will aim for \033[33mdefault\033[0m, and never exceed \033[33mmax\033[0m. Cloudflare max is 96. Big values are good for cross-atlantic but may increase HDD fragmentation on some FS. Disable this optimization with [\033[32m1,1,1\033[0m]")
ap2.add_argument("--u2ow", metavar="NUM", type=int, default=0, help="web-client: default setting for when to replace/overwrite existing files; [\033[32m0\033[0m]=never, [\033[32m1\033[0m]=if-client-newer, [\033[32m2\033[0m]=always (volflag=u2ow)")
ap2.add_argument("--u2sort", metavar="TXT", type=u, default="s", help="upload order; [\033[32ms\033[0m]=smallest-first, [\033[32mn\033[0m]=alphabetical, [\033[32mfs\033[0m]=force-s, [\033[32mfn\033[0m]=force-n -- alphabetical is a bit slower on fiber/LAN but makes it easier to eyeball if everything went fine")
ap2.add_argument("--write-uplog", action="store_true", help="write POST reports to textfiles in working-directory")
@ -1478,7 +1427,7 @@ def add_auth(ap):
ap2.add_argument("--no-bauth", action="store_true", help="disable basic-authentication support; do not accept passwords from the 'Authenticate' header at all. NOTE: This breaks support for the android app")
ap2.add_argument("--bauth-last", action="store_true", help="keeps basic-authentication enabled, but only as a last-resort; if a cookie is also provided then the cookie wins")
ap2.add_argument("--ses-db", metavar="PATH", type=u, default=ses_db, help="where to store the sessions database (if you run multiple copyparty instances, make sure they use different DBs)")
ap2.add_argument("--ses-len", metavar="CHARS", type=int, default=24, help="session key length; default is 144 bits ((24//4)*4*6)")
ap2.add_argument("--ses-len", metavar="CHARS", type=int, default=20, help="session key length; default is 120 bits ((20//4)*4*6)")
ap2.add_argument("--no-ses", action="store_true", help="disable sessions; use plaintext passwords in cookies")
ap2.add_argument("--grp-all", metavar="NAME", type=u, default="acct", help="the name of the auto-generated group which contains every username which is known")
ap2.add_argument("--ipu", metavar="CIDR=USR", type=u, action="append", help="\033[34mREPEATABLE:\033[0m users with IP matching \033[33mCIDR\033[0m are auto-authenticated as username \033[33mUSR\033[0m; example: [\033[32m172.16.24.0/24=dave]")
@ -1628,18 +1577,6 @@ def add_opds(ap):
ap2.add_argument("--opds-exts", metavar="T,T", type=u, default="epub,cbz,pdf", help="file formats to list in OPDS feeds; leave empty to show everything (volflag=opds_exts)")
def add_wopi(ap):
ap2 = ap.add_argument_group("WOPI options")
ap2.add_argument("--wopi", action="store_true", help="enable integration with office suites using WOPI")
ap2.add_argument("--wopi-api", metavar="URL", type=u, default="", help="URL that the WOPI-client should use to communicate with copyparty; default is same as user's webbrowser. Example: [\033[32mhttps://party.example.com/\033[0m]")
ap2.add_argument("--wopi-url", metavar="URL", type=u, default="", help="URL to your WOPI client; the host of e.g. Collabora Online. Example: [\033[32mhttps://code.example.com/\033[0m]")
ap2.add_argument("--wopi-urls", metavar="H=U", type=u, action="append", help="\033[34mREPEATABLE:\033[0m maps http \033[33mH\033[0mOST copyparty being accessed by to specific WOPI client instance \033[33mU\033[0mRL; falls back to \033[33m--wopi-url\033[0m; examples: [\033[32mparty.public.com=https://office.public.com/\033[0m], [\033[32mparty.internal.net:8443=https://office.internal.net:8443/\033[0m]")
ap2.add_argument("--wopi-crt", metavar="TXT", type=u, default="", help="if \033[33m--wopi-url\033[0m is selfsigned: path to ca.pem or cert.pem to expect/verify (can be [\033[32mno\033[0m] for full-yolo)")
ap2.add_argument("--wopi-crt-icn", action="store_true", help="if \033[33m--wopi-url\033[0m is selfsigned: ignore the CN (server ip/name) in cert")
ap2.add_argument("--wopi-ttl", metavar="SEC", type=int, default=1800, help="session lifetime; allow editing for this many seconds (default is 30 min)")
ap2.add_argument("--wopi-wdel", action="store_true", help="require permissions read+write+delete for writing to file; default is just read+write")
def add_handlers(ap):
ap2 = ap.add_argument_group("handlers (see --help-handlers)")
ap2.add_argument("--on404", metavar="PY", type=u, action="append", help="\033[34mREPEATABLE:\033[0m handle 404s by executing \033[33mPY\033[0m file")
@ -1774,8 +1711,6 @@ def add_safety(ap):
ap2.add_argument("--loris", metavar="B", type=int, default=60, help="if a client maxes out the server connection limit without sending headers, ban it for \033[33mB\033[0m minutes; disable with [\033[32m0\033[0m]")
ap2.add_argument("--acao", metavar="V[,V]", type=u, default="*", help="Access-Control-Allow-Origin; list of origins (domains/IPs without port) to accept requests from; [\033[32mhttps://1.2.3.4\033[0m]. Default [\033[32m*\033[0m] allows requests from all sites but removes cookies and http-auth; only ?pw=hunter2 survives")
ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like \033[33m--acao\033[0m's description)")
if FULL_HELP or ANYWIN:
ap2.add_argument("--unsafe-tools", action="store_true", help="windows-only: allow running programs (ffmpeg, dcraw_emu, ...) when their location is inside a folder that can be uploaded to (dangerous due to DLL-hijacking)")
if FULL_HELP or (not ANYWIN and not UNIX):
ap2.add_argument("--th-bwrap", metavar="CMD", type=u, default=th_bwrap, help="optional bwrap sandbox command for FFmpeg and dcraw (Linux-only)")
ap2.add_argument("--use-bwrap", metavar="TXT", type=u, default="n", help="a/n/f; [\033[32ma\033[0m]=auto (yes if the program 'bwrap' exists (assumes it works)), [\033[32mn\033[0m]=no (assumes bwrap is broken), [\033[32mf\033[0m]=force (disables FFmpeg if bwrap unavailable)")
@ -1783,7 +1718,7 @@ def add_safety(ap):
ap2.add_argument("--use-bwrap", metavar="TXT", type=u, default="n", help=argparse.SUPPRESS)
def add_salt(ap, fk_salt, dk_salt, ah_salt, wopi_salt):
def add_salt(ap, fk_salt, dk_salt, ah_salt):
ap2 = ap.add_argument_group("salting options")
ap2.add_argument("--ah-alg", metavar="ALG", type=u, default="none", help="account-pw hashing algorithm; one of these, best to worst: \033[32margon2 scrypt sha2 none\033[0m (each optionally followed by alg-specific comma-sep. config)")
ap2.add_argument("--ah-salt", metavar="SALT", type=u, default=ah_salt, help="account-pw salt; ignored if \033[33m--ah-alg\033[0m is none (default)")
@ -1792,7 +1727,6 @@ def add_salt(ap, fk_salt, dk_salt, ah_salt, wopi_salt):
ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files")
ap2.add_argument("--dk-salt", metavar="SALT", type=u, default=dk_salt, help="per-directory accesskey salt; used to generate unpredictable URLs to share folders with users who only have the 'get' permission")
ap2.add_argument("--warksalt", metavar="SALT", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
ap2.add_argument("--wopi-salt", metavar="TXT", type=u, default=wopi_salt, help="WOPI file ID salt; used by \033[33m--wopi\033[0m for persistent file IDs")
ap2.add_argument("--show-ah-salt", action="store_true", help="on startup, print the effective value of \033[33m--ah-salt\033[0m (the autogenerated value in $XDG_CONFIG_HOME unless otherwise specified)")
ap2.add_argument("--show-fk-salt", action="store_true", help="on startup, print the effective value of \033[33m--fk-salt\033[0m (the autogenerated value in $XDG_CONFIG_HOME unless otherwise specified)")
ap2.add_argument("--show-dk-salt", action="store_true", help="on startup, print the effective value of \033[33m--dk-salt\033[0m (the autogenerated value in $XDG_CONFIG_HOME unless otherwise specified)")
@ -1810,7 +1744,7 @@ def add_logging(ap):
ap2.add_argument("-q", action="store_true", help="quiet; disable most STDOUT messages")
ap2.add_argument("-lo", metavar="PATH", type=u, default="", help="logfile; use .txt for plaintext or .xz for compressed. Example: \033[32mcpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz\033[0m (NB: some errors may appear on STDOUT only)")
ap2.add_argument("--flo", metavar="N", type=int, default=1, help="log format for \033[33m-lo\033[0m; [\033[32m1\033[0m]=classic/colors, [\033[32m2\033[0m]=no-color")
ap2.add_argument("--rlo", metavar="TXT", type=u, default="no", help="logrotate counter format; see \033[33m--help-rlo\033[0m")
ap2.add_argument("--rlo", metavar="TXT", type=u, default=".1", help="logrotate counter format; see \033[33m--help-rlo\033[0m")
ap2.add_argument("--logrot-sig", metavar="S", type=u, default="", help="immediately logrotate when unix-signal \033[33mS\033[0m is received; examples: [\033[32mSIGHUP\033[0m], [\033[32mHUP\033[0m], [\033[32m1\033[0m]")
ap2.add_argument("--no-ansi", action="store_true", default=not VT100, help="disable colors; same as environment-variable NO_COLOR")
ap2.add_argument("--ansi", action="store_true", help="force colors; overrides environment-variable NO_COLOR")
@ -1861,7 +1795,7 @@ def add_thumbnail(ap):
ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32my\033[0m]=yes, [\033[32mn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
ap2.add_argument("--th-qv", metavar="N", type=int, default=40, help="webp/jpg thumbnail quality (10~90); higher is larger filesize and better quality (volflag=th_qv)")
ap2.add_argument("--th-qvx", metavar="N", type=int, default=64, help="jxl thumbnail quality (10~90); higher is larger filesize and better quality (volflag=th_qvx)")
ap2.add_argument("--th-dec", metavar="LIBS", default="raw,pil,ff,vips", help="image decoders, in order of preference")
ap2.add_argument("--th-dec", metavar="LIBS", default="vips,raw,pil,ff", help="image decoders, in order of preference")
ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
ap2.add_argument("--th-no-jxl", action="store_true", help="disable jpeg-xl output")
@ -1873,32 +1807,21 @@ def add_thumbnail(ap):
ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age -- folders which haven't been poked for longer than \033[33m--th-poke\033[0m seconds will get deleted every \033[33m--th-clean\033[0m seconds")
ap2.add_argument("--th-pregen", metavar="F,F", type=u, default="", help="pregenerate thumbnails on startup; \033[33mF,F\033[0m is comma-separated list of formats; example: [\033[32mj,jf,w,w3,wf,wf3,x,xf\033[0m] NOTE: remember to set \033[33m--th-maxage 123456789\033[0m (volflag=th_pregen)")
ap2.add_argument("--th-pre-rl", metavar="SEC", type=int, default=30, help="while pregen is running, ratelimit the thumbnailer logger to one message every \033[33mSEC\033[0m seconds (only works with \033[33m-j1\033[0m); set 0 to disable ratelimit")
ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for; enabling \033[33m-e2d\033[0m will make these case-insensitive, and try them as dotfiles (.folder.jpg), and also automatically select thumbnails for all folders that contain pics, even if none match this pattern (volflag=th_covers)")
ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for; enabling \033[33m-e2d\033[0m will make these case-insensitive, and try them as dotfiles (.folder.jpg), and also automatically select thumbnails for all folders that contain pics, even if none match this pattern")
ap2.add_argument("--th-spec-p", metavar="N", type=u, default=1, help="for music, do spectrograms or embedded coverart? [\033[32m0\033[0m]=only-art, [\033[32m1\033[0m]=prefer-art, [\033[32m2\033[0m]=only-spec")
ap2.add_argument("--th-spec-fl", action="store_true", help="generate spectrograms with logarithmic frequency scale instead of linear")
# https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
# https://github.com/libvips/libvips
# https://stackoverflow.com/a/47612661
# ffmpeg -hide_banner -demuxers | awk '/^ D /{print$2}' | while IFS= read -r x; do ffmpeg -hide_banner -h demuxer=$x; done | grep -E '^Demuxer |extensions:'
ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,cbz,dcx,dds,dib,emf,eps,epub,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,jxl,kra,ora,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,cbz,dcx,dds,dib,emf,eps,epub,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="3fr,arw,avif,cr2,cr3,crw,dcr,dng,erf,exr,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,jp2,jpeg,jpg,jpx,jxl,k25,kdc,mdc,mef,mrw,nef,nii,nrw,orf,pfm,pgm,png,ppm,raf,raw,rw2,sr2,srf,srw,svg,tif,tiff,webp,x3f", help="image formats to decode using pyvips")
ap2.add_argument("--th-r-raw", metavar="T,T", type=u, default="3fr,arw,cr2,cr3,crw,dcr,dng,erf,k25,kdc,mdc,mef,mos,mrw,nef,nrw,orf,pef,raf,raw,rw2,sr2,srf,srw,x3f", help="image formats to decode using rawpy (if available) or libraw's dcraw_emu")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,cbz,dds,dib,epub,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,kra,ora,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,cbz,dds,dib,epub,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bcstm,bfstm,brstm,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,m4b,m4r,mdgz,mdxz,mdz,mka,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,oga,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
ap2.add_argument("--th-pil-add", metavar="T", type=u, default="", help="add or remove image formats to decode using pillow listed in \033[33m--th-r-pil\033[0m; example: [\033[32mfb2,pdf,-psd,-bmp\033[0m] (add fb2 and pdf, remove psd and bmp)")
ap2.add_argument("--th-vips-add", metavar="T", type=u, default="", help="add or remove formats; same as \033[33m--th-pil-add\033[0m but for pyvips")
ap2.add_argument("--th-raw-add", metavar="T", type=u, default="", help="add or remove formats; same as \033[33m--th-pil-add\033[0m but for rawpy/libraw")
ap2.add_argument("--th-ffi-add", metavar="T", type=u, default="", help="add or remove formats; same as \033[33m--th-pil-add\033[0m but for ffmpeg images")
ap2.add_argument("--th-ffv-add", metavar="T", type=u, default="", help="add or remove formats; same as \033[33m--th-pil-add\033[0m but for ffmpeg videos")
ap2.add_argument("--th-ffa-add", metavar="T", type=u, default="", help="add or remove formats; same as \033[33m--th-pil-add\033[0m but for ffmpeg audio")
ap2.add_argument("--th-spec-cnv", metavar="T", type=u, default="it,itgz,itxz,itz,mdgz,mdxz,mdz,mo3,mod,s3m,s3gz,s3xz,s3z,xm,xmgz,xmxz,xmz,xpk", help="audio formats which provoke https://trac.ffmpeg.org/ticket/10797 (huge ram usage for s3xmodit spectrograms)")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz, cbz=jpg.cbz, epub=jpg.epub, kra=png.kra, ora=png.ora", help="audio/image formats to decompress before passing to ffmpeg")
ap2.add_argument("--th-extract", metavar="T", type=u, action="append", help="\033[34mREPEATABLE:\033[0m list of file extensions to thumbnail using a custom plugin (a python script); example: [\033[32mmdf,iso,dmg=/thumbs/diskimg.py\033[0m], see --help-thumb-ex")
ap2.add_argument("--th-extr-sz", metavar="M", type=int, default=16, help="max num megabytes to allow \033[33m--th-extract\033[0m plugins to extract from each file")
ap2.add_argument("--hot-th-extr", action="store_true", help="recompile extractors on each thumbnail extraction -- expensive but convenient when hacking on stuff")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz, cbz=jpg.cbz, epub=jpg.epub", help="audio/image formats to decompress before passing to ffmpeg")
def add_transcoding(ap):
@ -1959,8 +1882,7 @@ def add_db_general(ap, hcores):
ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="rescan filesystem for changes every \033[33mSEC\033[0m seconds; 0=off (volflag=scan)")
ap2.add_argument("--db-act", metavar="SEC", type=float, default=10.0, help="defer any scheduled volume reindexing until \033[33mSEC\033[0m seconds after last db write (uploads, renames, ...)")
ap2.add_argument("--srch-icase", action="store_true", help="case-insensitive search for all unicode characters (the default is icase for just ascii). NOTE: will make searches much slower (around 3x), and NOTE: only applies to filenames/paths, not tags")
ap2.add_argument("--srch-nfkc", action="store_true", help="case-insensitive and unicode-normalization-insensitive search (NFC/NFD) for filenames/paths; slightly slower than \033[33m--srch-icase\033[0m (about 15%% slower / 87%% as fast)")
ap2.add_argument("--srch-icase", action="store_true", help="case-insensitive search for all unicode characters (the default is icase for just ascii). NOTE: will make searches much slower (around 4x), and NOTE: only applies to filenames/paths, not tags")
ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than \033[33mSEC\033[0m seconds")
ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
ap2.add_argument("--srch-excl", metavar="PTN", type=u, default="", help="regex: exclude files from search results if the file-URL matches \033[33mPTN\033[0m (case-sensitive). Example: [\033[32mpassword|logs/[0-9]\033[0m] any URL containing 'password' or 'logs/DIGIT' (volflag=srch_excl)")
@ -2136,7 +2058,6 @@ def run_argparse(
fk_salt = get_salt("fk", 18)
dk_salt = get_salt("dk", 30)
ah_salt = get_salt("ah", 18)
wopi_salt = get_salt("wopi", 18)
# alpine peaks at 5 threads for some reason,
# all others scale past that (but try to avoid SMT),
@ -2171,9 +2092,8 @@ def run_argparse(
add_tftp(ap)
add_smb(ap)
add_opds(ap)
add_wopi(ap)
add_safety(ap)
add_salt(ap, fk_salt, dk_salt, ah_salt, wopi_salt)
add_salt(ap, fk_salt, dk_salt, ah_salt)
add_optouts(ap)
add_shutdown(ap)
add_yolo(ap)
@ -2276,9 +2196,6 @@ def main(argv: Optional[list[str]] = None) -> None:
argv.pop(n)
break
if CFG_DEF:
CFG_DEF[0] = absreal(CFG_DEF[0])
ensure_locale()
ensure_webdeps()

View file

@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 20, 20)
VERSION = (1, 20, 18)
CODENAME = "sftp is fine too"
BUILD_DT = (2026, 8, 4)
BUILD_DT = (2026, 7, 9)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import base64
@ -13,7 +13,7 @@ import threading
import time
from datetime import datetime
from .__init__ import ANYWIN, CFG_DEF, MACOS, PY2, TYPE_CHECKING, WINDOWS, E
from .__init__ import ANYWIN, MACOS, PY2, TYPE_CHECKING, WINDOWS, E
from .bos import bos
from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
from .pwhash import PWHash
@ -25,7 +25,6 @@ from .util import (
FN_EMB,
HAVE_SQLITE3,
IMPLICATIONS,
LOG,
META_NOBOTS,
MIMES,
SQLITE_VER,
@ -2379,7 +2378,6 @@ class AuthSrv(object):
vol.flags["dvthumb"] = True
vol.flags["dathumb"] = True
vol.flags["dithumb"] = True
vol.flags["dethumb"] = True
have_fk = False
for vol in vfs.all_nodes.values():
@ -2614,13 +2612,6 @@ class AuthSrv(object):
t = "WARNING: volume [/%s]: invalid value specified for ext-th: %s"
self.log(t % (vol.vpath, etv), 3)
zsl = [x.strip() for x in vol.flags["th_covers"].split(",")]
zsl = [x for x in zsl if x not in ("", "no")]
vol.flags["th_coversl"] = zsl
vol.flags["th_coversd"] = zsl + ["." + x for x in zsl]
vol.flags["th_covers_set"] = set(vol.flags["th_coversl"])
vol.flags["th_coversd_set"] = set(vol.flags["th_coversd"])
zsl = [x.strip() for x in vol.flags["rw_edit"].split(",")]
zsl = [x for x in zsl if x]
vol.flags["rw_edit"] = ",".join(zsl)
@ -2662,11 +2653,10 @@ class AuthSrv(object):
if head_s and not head_s.endswith("\n"):
head_s += "\n"
zs2 = "Content-Security-Policy: %s\r\n"
zs = vol.flags.get("csp_ui", "")
csp_ui = zs2 % (zs,) if zs.lower() not in ("", "no") else ""
csp_ui = "Content-Security-Policy: %s\r\n" % (zs,) if zs else ""
zs = vol.flags.get("csp_dl", "")
csp_dl = zs2 % (zs,) if zs.lower() not in ("", "no") else ""
csp_dl = "Content-Security-Policy: %s\r\n" % (zs,) if zs else ""
zs = "X-Content-Type-Options: nosniff\r\n"
if "norobots" in vol.flags:
@ -2764,8 +2754,7 @@ class AuthSrv(object):
for k in drop:
t = 'cannot enable [%s] for volume "/%s" because this requires one of the following: e2d / e2ds / e2dsa (either as volflag or global-option)'
if not (enshare and vp.startswith(shrs)):
self.log(t % (k, vol.vpath), 1)
self.log(t % (k, vol.vpath), 1)
vol.flags.pop(k)
zi = vol.flags.get("lifetime") or 0
@ -2874,13 +2863,12 @@ class AuthSrv(object):
continue
try:
bos.makedirs(vol.realpath, vf=vol.flags)
for fn, _ in statdir(
self.log_func, not self.args.no_scandir, False, vol.realpath, True
):
files = os.listdir(vol.realpath)
for fn in files:
fn2 = fn.lower()
if fn == fn2:
fn2 = fn.upper()
if fn == fn2:
if fn == fn2 or fn2 in files:
continue
is_ci = os.path.exists(os.path.join(vol.realpath, fn2))
ccs = "y" if is_ci else "n"
@ -3049,10 +3037,6 @@ class AuthSrv(object):
t = "hint: enable upload deduplication with --dedup (but see readme for consequences)"
self.log(t, 6)
if MACOS and not self.args.srch_nfkc:
t = "hint: enable --srch-nfkc to improve unicode filename/path search on MacOS"
self.log(t, 6)
zv, _ = vfs.get("/", "*", False, False)
zs = zv.realpath.lower()
if zs in ("/", "c:\\") or zs.startswith(r"c:\windows"):
@ -3063,8 +3047,6 @@ class AuthSrv(object):
zv, _ = vfs.get("", "*", False, True, err=999)
if self.warn_anonwrite and verbosity > 4 and os.getcwd() == zv.realpath:
t = "anyone can write to the current directory: {}\n"
if ANYWIN:
t += "/!\\ NOTE: because you are using Windows, this is kinda dangerous (DLL-hijacking); you should configure accounts and volumes if this is accessible from an untrusted network\n"
self.log(t.format(zv.realpath), c=1)
self.warn_anonwrite = False
@ -3258,7 +3240,6 @@ class AuthSrv(object):
self.js_ls = {}
self.js_htm = ""
self.md_htm = ""
for vp, vn in self.vfs.all_nodes.items():
if enshare and vp.startswith(shrs):
continue # propagates later in this func
@ -3350,8 +3331,6 @@ class AuthSrv(object):
zs2 = getattr(self.args, zs, "")
if zs2:
js_htm[zs] = zs2
if self.args.wopi:
js_htm["have_wopi"] = True
zs = "have_emp md_no_br"
md_htm = {x: js_htm[x] for x in zs.split(" ")}
@ -3377,7 +3356,6 @@ class AuthSrv(object):
dbv = vol.get_dbv("")[0]
vol.js_ls = vol.js_ls or dbv.js_ls or {}
vol.js_htm = vol.js_htm or dbv.js_htm or "{}"
vol.md_htm = vol.md_htm or dbv.md_htm or "{}"
zs = str(vol.flags.get("tcolor") or self.args.tcolor)
vol.flags["tcolor"] = zs.lstrip("#")
@ -3805,12 +3783,12 @@ class AuthSrv(object):
"",
]
csv = set("i p zm_on zm_off zs_on zs_off".split())
csv = set("i p th_covers zm_on zm_off zs_on zs_off".split())
zs = "c ihead ohead mtm mtp on403 on404 xac xad xar xau xiu xban xbc xbd xbr xbu xm"
lst = set(zs.split())
askip = set("a v c vc cgen exp_lg exp_md theme".split())
t = "emb_all emb_lgs emb_mds exp_lg exp_md ext_th_d ls_q_m mv_re_r mv_re_t rm_re_r rm_re_t oh_f oh_g rw_edit_set srch_re_dots srch_re_nodot th_coversd th_coversl th_covers_set th_coversd_set"
t = "exp_lg exp_md ext_th_d mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot"
fskip = set(t.split())
# keymap from argv to vflag
@ -3937,9 +3915,6 @@ class AuthSrv(object):
except:
pass
if k in ("chmod_d", "chmod_f"):
v = "%o" % (v,)
try:
ak = vmap[k]
v2 = getattr(self.args, ak)
@ -3949,7 +3924,7 @@ class AuthSrv(object):
except:
pass
if v2 == v:
if v2 is v:
continue
except:
pass
@ -3993,7 +3968,7 @@ class AuthSrv(object):
def derive_args(args: argparse.Namespace) -> None:
args.have_idp_hdrs = bool(args.idp_h_usr or args.idp_hm_usr)
args.have_ipu_or_ipr = bool(args.ipu or args.ipr or args.wopi)
args.have_ipu_or_ipr = bool(args.ipu or args.ipr)
def n_du_who(s: str) -> int:
@ -4081,11 +4056,6 @@ def expand_config_file(
if fp2 in ipath:
continue
if CFG_DEF and CFG_DEF[0] == fp2:
t = "ignoring PRTY_CONFIG because it was also autodiscovered by %s -> %s"
LOG[0]("root", t % (ipath, fp))
continue
expand_config_file(log, shenvexp, ret, fp2, ipath)
return

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import time

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import threading
import time

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import os

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import threading

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
# awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
@ -157,7 +157,6 @@ def vf_vmap() -> dict[str, str]:
"tail_tmax",
"tail_who",
"tcolor",
"th_covers",
"th_pregen",
"th_qv",
"th_qvx",
@ -316,14 +315,12 @@ flagcats = {
"dvthumb": "disables video thumbnails",
"dathumb": "disables audio thumbnails (spectrograms)",
"dithumb": "disables image thumbnails",
"dethumb": "disables custom thumbnails (--th-extract)",
"pngquant": "compress audio waveforms 33% better",
"thsize": "thumbnail res; WxH",
"crop": "center-cropping (y/n/fy/fn)",
"th3x": "3x resolution (y/n/fy/fn)",
"th_qv=40": "webp/jpg thumbnail quality (10~90)",
"th_qvx=40": "jxl thumbnail quality (10~90)",
"th_covers=a.jpg,a.png": "filenames to prefer as folder-thumbs",
"convt": "convert-to-image timeout in seconds",
"aconvt": "convert-to-audio timeout in seconds",
"th_spec_p=1": "make spectrograms? 0=never 1=fallback 2=always",

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import importlib
import sys

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import json
@ -248,7 +248,7 @@ def ramdisk_chk(asrv: AuthSrv) -> None:
t2 = ["\n volume=[/%s], abspath=%r, type=%s, root=%r" % x for x in mods]
log("vfs", t + "".join(t2) + "\n", 1)
assume = "mac" if MACOS else "win" if ANYWIN else "lin"
assume = "mac" if MACOS else "lin"
for vol in asrv.vfs.all_nodes.values():
if not vol.realpath or vol.flags.get("is_file"):
continue
@ -263,4 +263,3 @@ def ramdisk_chk(asrv: AuthSrv) -> None:
htm = json.loads(vol.js_htm)
vol.flags["fsnt"] = vol.js_ls["fsnt"] = htm["fsnt"] = fs
vol.js_htm = json_hesc(json.dumps(htm))
# md_htm is dontcare; only relevant for up2k request body

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import errno
@ -149,7 +149,6 @@ class FtpFs(AbstractedFS):
self.hub: "SvcHub" = cmd_channel.hub
self.args = cmd_channel.args
self.uname = cmd_channel.uname
self.log_src = "%s:FTP" % (cmd_channel.cli_ip,)
self.cwd = "/" # pyftpdlib convention of leading slash
self.root = "/var/lib/empty"
@ -157,8 +156,8 @@ class FtpFs(AbstractedFS):
self.listdirinfo = self.listdir
self.chdir(".")
def clog(self, msg: str, c: Union[int, str] = 0) -> None:
self.hub.log(self.log_src, msg, c)
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.hub.log("ftpd", msg, c)
def v2a(
self,
@ -239,8 +238,6 @@ class FtpFs(AbstractedFS):
return True
def open(self, filename: str, mode: str) -> typing.IO[Any]:
self.clog("open(%r,%r) @%s" % (filename, mode, self.uname))
r = "r" in mode
w = "w" in mode or "a" in mode or "+" in mode
@ -258,7 +255,7 @@ class FtpFs(AbstractedFS):
xbu = vfs.flags.get("xbu")
if xbu:
hr = runhook(
self.clog,
self.log,
None,
self.hub.up2k,
"xbu.ftp",
@ -278,7 +275,7 @@ class FtpFs(AbstractedFS):
if t or hr.get("rc") != 0:
if not t:
t = "upload blocked by xbu server config: %r" % (filename,)
self.clog(t, 3)
self.log(t, 3)
raise FSE(t)
if w and need_unlink: # type: ignore # !rm
@ -303,13 +300,12 @@ class FtpFs(AbstractedFS):
# Don't unlink file for append mode
elif "a" not in mode:
wunlink(self.clog, ap, VF_CAREFUL)
wunlink(self.log, ap, VF_CAREFUL)
ret = open(fsenc(ap), mode, self.args.iobuf)
if w and "fperms" in vfs.flags:
set_fperms(ret, vfs.flags)
self.clog(" `-- %r" % (ap,))
return ret
def chdir(self, path: str) -> None:
@ -335,12 +331,10 @@ class FtpFs(AbstractedFS):
self.cwd = nwd
def mkdir(self, path: str) -> None:
self.clog("mkdir(%r) @%s" % (path, self.uname))
ap, vfs, _ = self.rv2a(path, w=True)
bos.makedirs(ap, vf=vfs.flags) # filezilla expects this
def listdir(self, path: str) -> list[str]:
self.clog("ls(%r) @%s" % (path, self.uname))
vpath = join(self.cwd, path)
try:
ap, vfs, rem = self.v2a(vpath, True, False)
@ -390,7 +384,6 @@ class FtpFs(AbstractedFS):
return ret
def rmdir(self, path: str) -> None:
self.clog("rmdir(%r) @%s" % (path, self.uname))
ap = self.rv2a(path, d=True)[0]
try:
bos.rmdir(ap)
@ -399,7 +392,6 @@ class FtpFs(AbstractedFS):
raise
def remove(self, path: str) -> None:
self.clog("rm(%r) @%s" % (path, self.uname))
if self.args.no_del:
raise FSE("The delete feature is disabled in server config")
@ -410,7 +402,6 @@ class FtpFs(AbstractedFS):
raise FSE(str(ex))
def rename(self, src: str, dst: str) -> None:
self.clog("mv(%r,%r) @%s" % (src, dst, self.uname))
if self.args.no_mv:
raise FSE("The rename/move feature is disabled in server config")
@ -440,7 +431,6 @@ class FtpFs(AbstractedFS):
return st
def utime(self, path: str, timeval: float) -> None:
self.clog("utime(%r,%r) @%s" % (path, timeval, self.uname))
ap = self.rv2a(path, w=True)[0]
bos.utime_c(logging.warning, ap, int(timeval), False)
@ -448,9 +438,6 @@ class FtpFs(AbstractedFS):
ap = self.rv2a(path)[0]
return bos.stat(ap)
def readlink(self, *a) -> None:
raise OSError(errno.EINVAL, "nope")
def isfile(self, path: str) -> bool:
try:
st = self.stat(path)
@ -493,13 +480,9 @@ class FtpFs(AbstractedFS):
def get_user_by_uid(self, uid: int) -> str:
return "root"
def get_group_by_gid(self, gid: int) -> str:
def get_group_by_uid(self, gid: int) -> str:
return "root"
def mkstemp(self, *a, **ka):
self.clog("mkstemp%r @%s" % (a, self.uname))
raise FSE("nope")
class FtpHandler(FTPHandler):
abstracted_fs = FtpFs
@ -528,7 +511,6 @@ class FtpHandler(FTPHandler):
return
self.cli_ip = cip
self.log_src = "%s:FTP" % (cip,)
# abspath->vpath mapping to resolve log_transfer paths
self.vfs_map: dict[str, str] = {}
@ -536,15 +518,6 @@ class FtpHandler(FTPHandler):
# reduce non-debug logging
self.log_cmds_list = [x for x in self.log_cmds_list if x not in ("CWD", "XCWD")]
def clog(self, msg: str, c: Union[int, str] = 0) -> None:
self.hub.log(self.log_src, msg, c)
def ftp_NOOP(self, *a) -> None:
self.respond("200 pong", logging.debug)
def ftp_STOU(self, *a) -> None:
self.respond("550 nope", logging.warning)
def ftp_STOR(self, file: str, mode: str = "w") -> Any:
# Optional[str]
vp = join(self.fs.cwd, file).lstrip("/")
@ -557,7 +530,7 @@ class FtpHandler(FTPHandler):
xbu = vfs.flags.get("xbu")
if xbu:
hr = runhook(
self.clog,
None,
None,
self.hub.up2k,
"xbu.ftpd",

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse # typechk
import copy
@ -30,7 +30,7 @@ try:
except:
pass
from .__init__ import ANYWIN, PY2, RES, RESM, TYPE_CHECKING, EnvParams, unicode
from .__init__ import ANYWIN, RES, RESM, TYPE_CHECKING, EnvParams, unicode
from .__version__ import S_VERSION
from .authsrv import LEELOO_DALLAS, VFS # typechk
from .bos import bos
@ -149,11 +149,6 @@ if True: # pylint: disable=using-constant-test
if TYPE_CHECKING:
from .httpconn import HttpConn
if PY2:
from urllib2 import urlopen
else:
from urllib.request import urlopen
if not hasattr(socket, "AF_UNIX"):
setattr(socket, "AF_UNIX", -9001)
@ -176,27 +171,6 @@ RSS_SORT = {"m": "mt", "u": "at", "n": "fn", "s": "sz"}
ACODE2_FMT = set(["opus", "owa", "caf", "mp3", "flac", "wav"])
IDX_HTML = set(["index.htm", "index.html"])
# coolwsd-26.04.2.3 discovery editnew; browser.js wopi_set is superset of this
WOPI_NEW = (
".odt",
".fodt",
".ott",
".docx",
".dotx",
".rtf",
".odm",
".ods",
".fods",
".ots",
".xlsx",
".odp",
".fodp",
".otp",
".pptx",
".fodg",
".otg",
)
A_FILE = os.stat_result(
(0o644, -1, -1, 1, 1000, 1000, 8, 0x39230101, 0x39230101, 0x39230101)
)
@ -839,22 +813,6 @@ class HttpCli(object):
if not ipr[self.uname].map(self.ip):
self.log("username [%s] rejected by --ipr" % (self.uname,), 3)
self.uname = "*"
if self.args.wopi and "access_token" in self.uparam:
wopi_a = self.uparam["access_token"]
try:
wopi_f = self.conn.hsrv.wopi_files[wopi_a]
if wopi_f["expires"] < time.time():
raise Exception("expired")
uname = wopi_f["uname"]
self.asrv.vfs.get(
wopi_f["vp"], uname, True, True, False, self.args.wopi_wdel
)
self.uname = uname
except Exception as ex:
self.conn.hsrv.wopi_files.pop(wopi_a, None)
self.cbonk(self.conn.hsrv.gpwd, wopi_a, "wopi", "bad wopi tokens")
self.loud_reply("bad wopi token %s (%s)" % (wopi_a, ex), status=400)
return False
self.rvol = self.asrv.vfs.aread[self.uname]
self.wvol = self.asrv.vfs.awrite[self.uname]
@ -1574,143 +1532,8 @@ class HttpCli(object):
if "rss" in self.uparam:
return self.tx_rss()
if self.args.wopi:
if "wopi" in self.uparam:
return self.tx_wopi()
if self.vpath.startswith("wopi"):
return self.tx_wopi_api()
return self.tx_browser()
def tx_wopi_api(self) -> bool:
atoken = self.uparam["access_token"]
session = self.conn.hsrv.wopi_files[atoken]
if self.do_log:
self.log(" `-- wopi: %r" % (session["vp"],))
zs = "wopi/files/%s" % (session["file_id"],)
if not self.vpath.startswith(zs):
return self.tx_404()
query = self.vpath[len(zs) :]
vfs, rem = self.asrv.vfs.get(session["vp"], self.uname, True, True)
vpath = vjoin(vfs.vpath, rem)
ap = vfs.canonical(rem)
if query.startswith("/contents"):
return self.tx_file("oh_f", ap)
else:
st = bos.stat(ap)
file_info = {
"BaseFileName": vpath.split("/")[-1],
"Size": st.st_size,
"OwnerId": self.uname,
"UserId": self.uname,
"UserFriendlyName": self.uname,
"UserCanWrite": True,
"UserCanNotWriteRelative": True,
"LastModifiedTime": time.strftime(
"%Y-%m-%dT%H:%M:%SZ", time.gmtime(st.st_mtime)
),
}
ret = json.dumps(file_info).encode("utf-8", "replace")
self.reply(ret, 200, "application/json; charset=utf-8")
return True
return self.tx_404()
def tx_wopi(self) -> bool:
vpath = vjoin(self.vpath, self.uparam["wopi"])
vfs, rem = self.asrv.vfs.get(
vpath, self.uname, True, True, False, self.args.wopi_wdel
)
if not bos.path.isfile(vfs.canonical(rem)):
return self.tx_404()
wopi_files = self.conn.hsrv.wopi_files
found = None
rm = []
with self.conn.hsrv.mutex:
now = time.time()
for atoken, session in wopi_files.items():
if session["expires"] < now:
rm.append(atoken)
continue
if session["vp"] != vpath or session["uname"] != self.uname:
continue
if session["expires"] - now < self.args.wopi_ttl * 0.9:
rm.append(atoken)
continue
found = session
break
for zs in rm:
del wopi_files[zs]
if len(wopi_files) > 9000: # about 6 MiB
raise Pebkac(500, "too many wopi sessions")
if not found:
atoken = ub64enc(os.urandom(18)).decode("ascii") # 18 = 144b = 24c
zb = (vpath + self.args.wopi_salt).encode("utf-8", "replace")
file_id = ub64enc(hashlib.sha512(zb).digest()[:15]).decode("ascii")
wopi_files[atoken] = session = {
"vp": vpath,
"uname": self.uname,
"file_id": file_id,
"expires": time.time() + self.args.wopi_ttl,
}
xml = url = "?"
try:
from .dxml import parse_xml
uo_kw = {}
if self.args.wopi_crt:
import ssl
if self.args.wopi_crt == "no":
ctx = ssl._create_unverified_context()
else:
ctx = ssl.create_default_context(cafile=self.args.wopi_crt)
ctx.check_hostname = not self.args.wopi_crt_icn
uo_kw["context"] = ctx
wopi_urls = dict(x.lower().split("=", 1) for x in self.args.wopi_urls or [])
url = wopi_urls.get(self.host.lower(), self.args.wopi_url).rstrip("/")
url += "/hosting/discovery"
buf = urlopen(url, **uo_kw).read()
xml = buf.decode("ascii", "replace").lower()
enc = self.get_xml_enc(xml)
xml = buf.decode(enc, "replace")
xroot = parse_xml(xml)
ext = vpath.split(".")[-1]
url = xroot.find(
".//action[@ext='%s'][@name='edit'][@urlsrc]" % (ext,)
).get("urlsrc")
if not url.endswith(("?", "&")):
url += "&" if "?" in url else "?"
url += "WOPISrc="
if self.args.wopi_api:
zs = self.args.wopi_api.rstrip("/")
else:
zs = ("https://" if self.is_https else "http://") + self.host
url += quotep(zs + "/wopi/files/" + session["file_id"])
except:
del wopi_files[atoken] # dont reuse an atoken wopi-client doesnt like
t = "reading WOPI-client response from %s failed; %s\n%s"
self.log(t % (url, min_ex(), xml), 3)
raise Pebkac(500, "wopi error (see fileserver log)")
html = self.j2s(
"wopi",
title=self.uparam["wopi"],
url=url,
atoken=atoken,
ttl=int(session["expires"] * 1000),
).encode("utf-8", "replace")
self.reply(html, 200, "text/html; charset=utf-8")
return True
def tx_rss(self) -> bool:
if self.do_log:
self.log("RSS %s @%s" % (self.req, self.uname))
@ -2564,9 +2387,6 @@ class HttpCli(object):
raise Pebkac(405, "POST(%r) is disabled in server config" % (ctype,))
if self.args.wopi and self.vpath.startswith("wopi"):
return self.handle_post_binary()
raise Pebkac(405, "don't know how to handle POST(%r)" % (ctype,))
def handle_smsg(self) -> bool:
@ -3336,9 +3156,6 @@ class HttpCli(object):
except:
raise Pebkac(400, "you must supply a content-length for binary POST")
if self.args.wopi and self.vpath.startswith("wopi"):
return self.rx_wopi(postsize)
try:
chashes = self.headers["x-up2k-hash"].split(",")
wark = self.headers["x-up2k-wark"]
@ -3550,47 +3367,6 @@ class HttpCli(object):
self.reply(b"thank")
return True
def rx_wopi(self, postsize: int) -> bool:
atoken = self.uparam["access_token"]
session = self.conn.hsrv.wopi_files[atoken]
if self.do_log:
self.log(" `-- wopi: %r" % (session["vp"],))
zs = "wopi/files/%s/contents" % (session["file_id"],)
if not self.vpath.startswith(zs):
return self.tx_404()
vpath = self.conn.hsrv.wopi_files[self.uparam["access_token"]]["vp"]
vfs, rem = self.asrv.vfs.get(vpath, self.uname, False, True)
vpath = vjoin(vfs.vpath, rem)
ap = vfs.canonical(rem)
st = bos.stat(ap)
if "x-cool-wopi-timestamp" in self.headers:
zs = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(st.st_mtime))
if self.headers["x-cool-wopi-timestamp"] != zs:
self.reply(json.dumps({"COOLStatusCode": 1010}).encode("utf-8"), 409)
return True
buf = b""
for rbuf in self.get_body_reader()[0]:
buf += rbuf
if not rbuf:
break
if len(buf) != postsize:
t = "wopi post with incorrect length; expected %d, got %d"
raise Pebkac(400, t % (postsize, len(buf)))
with open(ap, "wb") as file:
file.write(buf)
st = bos.stat(ap)
zs = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(st.st_mtime))
ret = json.dumps({"LastModifiedTime": zs}).encode("utf-8", "replace")
self.reply(ret, 200, "application/json; charset=utf-8")
return True
def handle_chpw(self) -> bool:
assert self.parser # !rm
if self.args.usernames:
@ -3745,7 +3521,7 @@ class HttpCli(object):
def _mkdir(self, vpath: str, dav: bool = False) -> bool:
nullwrite = self.args.nw
self.gctx = vpath
vpath = sanitize_vpath(undot(vpath))
vpath = undot(vpath)
vfs, rem = self.asrv.vfs.get(vpath, self.uname, False, True)
if "nosub" in vfs.flags:
raise Pebkac(403, "mkdir is forbidden below this folder")
@ -3851,9 +3627,6 @@ class HttpCli(object):
True,
)
if self.args.wopi and sanitized.endswith(WOPI_NEW):
return self.redirect(vjoin(vfs.vpath, rem), "?wopi=" + quotep(sanitized))
vpath = "{}/{}".format(self.vpath, sanitized).lstrip("/")
self.redirect(vpath, "?edit")
return True
@ -7083,8 +6856,7 @@ class HttpCli(object):
nothumb = "dthumb" in dbv.flags
if is_dir:
vrem = vrem.rstrip("/")
cvs = dbv.flags["th_coversl"]
if nothumb or not cvs:
if nothumb:
pass
elif icur and vrem:
q = "select fn from cv where rd=? and dn=?"
@ -7101,7 +6873,7 @@ class HttpCli(object):
except:
pass
else:
for fn in cvs:
for fn in self.args.th_covers:
fp = os.path.join(abspath, fn)
try:
st = bos.stat(fp)
@ -7447,18 +7219,20 @@ class HttpCli(object):
dirs = []
files = []
ptn_hr = RE_HR
base = ""
if is_opds or (vpath and not (is_ls or is_js or self.trailing_slash)):
if is_opds:
base = self.args.SRS
if vpath:
base += vpath + "/"
else:
base = "/" + vpath + "/"
use_abs_url = is_opds or (
vpath and not is_ls and not is_js and not self.trailing_slash
)
for fn in ls_names:
href = base + fn
base = ""
href = fn
if use_abs_url:
if is_opds:
base = self.args.SRS
if vpath:
base += vpath + "/"
else:
base = "/" + vpath + "/"
href = base + fn
if fn in vfs_virt:
fspath = vfs_virt[fn].realpath
@ -7628,26 +7402,6 @@ class HttpCli(object):
(fe["sz"], fe["tags"][".files"]) = hit
except:
pass # 404 or mojibake
if vfs_virt:
q = "select sz, nf from ds where rd='' limit 1"
try:
for fe in [x for x in dirs if x["name"] in vfs_virt]:
if ".files" not in fe["tags"]:
fe["tags"][".files"] = 0
vols = [vn.nodes[fe["name"]]]
while vols:
vn2 = vols.pop()
if self.uname not in vn2.axs.uread:
continue
vols += list(vn2.nodes.values())
if vn2.dbv not in (vn2, None):
continue
hit = idx.get_cur(vn2).execute(q).fetchone()
if hit:
fe["sz"] += hit[0]
fe["tags"][".files"] += hit[1]
except:
pass
taglist = [k for k in lmte if k in tagset]
else:
@ -7882,7 +7636,7 @@ class HttpCli(object):
self.conn.hsrv.j2[tpl] = j2env.get_template(tname)
thumb = ""
is_pic = is_vid = is_au = False
for fn in vn.flags["th_coversd"]:
for fn in self.args.th_coversd:
if fn in lnames:
thumb = lnames[fn]
break
@ -7893,10 +7647,6 @@ class HttpCli(object):
ext in self.thumbcli.fmt_pil
or ext in self.thumbcli.fmt_vips
or ext in self.thumbcli.fmt_ffi
or (
ext in self.thumbcli.thumbable
and ext not in self.thumbcli.thumbable_native
)
)
is_vid = ext in self.thumbcli.fmt_ffv
is_au = ext in self.thumbcli.fmt_ffa

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse # typechk
import os

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import hashlib
import math
@ -180,8 +180,6 @@ class HttpSrv(object):
self.u2idx_free: dict[str, U2idx] = {}
self.u2idx_n = 0
self.wopi_files: dict[str, dict[str, str]] = {}
assert jinja2 # type: ignore # !rm
env = jinja2.Environment()
env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
@ -197,7 +195,6 @@ class HttpSrv(object):
"shares",
"splash",
"svcs",
"wopi",
]
self.j2 = {x: env.get_template(x + ".html") for x in jn}
self.j2["opds"] = env.get_template("opds.xml")
@ -318,7 +315,7 @@ class HttpSrv(object):
self.srvs.append(sck)
self.bound.add((ip, port))
self.nclimax = -int(-self.args.nc // nlisteners)
self.nclimax = math.ceil(self.args.nc * 1.0 / nlisteners)
Daemon(
self.thr_listen,
"httpsrv-n{}-listen-{}-{}".format(self.nid or "0", ip, port),

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse # typechk
import colorsys

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import errno
import os

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import json
import time

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import json
@ -17,9 +17,7 @@ from .util import (
FFMPEG_URL,
LOG,
REKOBO_LKEY,
SCWD,
VF_CAREFUL,
absreal,
expand_osenv_c,
fsenc,
gzip,
@ -49,10 +47,10 @@ except:
HAVE_MUTAGEN = False
def have_ff(name: str) -> list[bytes]:
def have_ff(name: str) -> bytes:
uname = name.upper()
if os.environ.get("PRTY_NO_" + uname):
return []
return b""
ebin = os.environ.get("PRTY_%s_BIN" % (uname,))
try:
@ -68,12 +66,11 @@ def have_ff(name: str) -> list[bytes]:
bcmd = scmd.encode("utf-8")
try:
sp.Popen([bcmd, b"-version"], stdout=sp.PIPE, stderr=sp.PIPE).communicate()
return [bcmd]
return bcmd
except:
return []
return b""
else:
ret = shutil.which(scmd)
return [absreal(ret).encode("utf-8")] if ret else []
return (shutil.which(scmd) or "").encode("utf-8")
HAVE_FFMPEG = have_ff("ffmpeg")
@ -194,12 +191,6 @@ def au_unpk(
log(t)
fi = zf.open(using)
elif pk == "kra" or pk == "ora":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
fi = zf.open("mergedimage.png")
elif pk == "epub":
fi = get_cover_from_epub(log, abspath)
assert fi # !rm
@ -256,14 +247,14 @@ def ffprobe(
) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]], list[Any], dict[str, Any]]:
# ffprobe -hide_banner -show_streams -show_format --
bap = fsenc(abspath)
cmd = bwrap(HAVE_FFPROBE[0], bap, b"") + [
cmd = bwrap(HAVE_FFPROBE, bap, b"") + [
b"-hide_banner",
b"-show_streams",
b"-show_format",
b"--",
bap,
]
rc, so, se = runcmd(cmd, cwd=SCWD, timeout=timeout, nice=True, oom=200)
rc, so, se = runcmd(cmd, timeout=timeout, nice=True, oom=200)
if rc and TH_BWRAP:
bwrap_fail(se)
retchk(rc, cmd, se)

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import socket
import time

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import base64

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import errno
import hashlib

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import errno
import re

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import re
import stat

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import tempfile

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import atexit
@ -81,7 +81,6 @@ from .util import (
alltrace,
build_netmap,
expat_ver,
fsenc,
gzip,
html_escape,
load_ipr,
@ -133,19 +132,6 @@ VER_SHARES_DB = 2
CVE_SEVS = {"low": 1, "medium": 2, "moderate": 2, "high": 3, "critical": 4}
def _build_th_fset(f: str, fadd: str) -> set[str]:
fset = {x for x in f.replace(" ", "").split(",")}
rmset = {""}
for x in fadd.replace(" ", "").split(","):
if x.startswith("-"):
rmset.add(x.lstrip("-"))
else:
fset.add(x)
return fset - rmset
class SvcHub(object):
"""
Hosts all services which cannot be parallelized due to reliance on monolithic resources.
@ -247,14 +233,10 @@ class SvcHub(object):
self.lo1 = self.lo2 = ""
if args.lo:
do_xz = args.lo.replace("%R", "").lower().endswith(".xz")
if "%R" not in args.lo:
args.lo += "%R"
if args.rlo in ("", "no"):
args.rlo = ""
if do_xz and not args.rlo:
args.rlo = ".1"
if args.rlo in ("", "del", "over"):
args.lo = args.lo.replace("%R", "")
try:
self.lo1, self.lo2 = args.lo.split("%R")
@ -414,9 +396,6 @@ class SvcHub(object):
self.asrv = AuthSrv(self.args, self.log, dargs=self.dargs)
ramdisk_chk(self.asrv)
if ANYWIN and not self.args.unsafe_tools:
self._check_toolpaths()
if args.cgen:
self.asrv.cgen()
@ -454,20 +433,6 @@ class SvcHub(object):
zlss = [x.strip().lower().split("=", 1) for x in args.au_unpk.split(",")]
args.au_unpk = {x[0]: x[1] for x in zlss}
args.th_r_pil = _build_th_fset(args.th_r_pil, args.th_pil_add)
args.th_r_vips = _build_th_fset(args.th_r_vips, args.th_vips_add)
args.th_r_raw = _build_th_fset(args.th_r_raw, args.th_raw_add)
args.th_r_ffi = _build_th_fset(args.th_r_ffi, args.th_ffi_add)
args.th_r_ffv = _build_th_fset(args.th_r_ffv, args.th_ffv_add)
args.th_r_ffa = _build_th_fset(args.th_r_ffa, args.th_ffa_add)
th_extract = args.th_extract or []
args.th_extract = {}
for arg in th_extract:
exts, script = arg.split("=", 1)
for ext in exts.split(","):
args.th_extract[ext.strip().lower()] = script
self.args.th_dec = list(decs.keys())
self.thumbsrv = None
want_ff = False
@ -505,7 +470,7 @@ class SvcHub(object):
t = "invalid mp3 transcoding quality [%s] specified; only supports [0] to disable, a CBR value such as [192k], or a CQ/CRF value such as [v2]"
raise Exception(t % (args.q_mp3,))
else:
zss = args.th_r_ffa | args.th_r_ffv
zss = set(args.th_r_ffa.split(",") + args.th_r_ffv.split(","))
args.au_unpk = {
k: v for k, v in args.au_unpk.items() if v.split(".")[0] not in zss
}
@ -1056,18 +1021,6 @@ class SvcHub(object):
if zb:
Daemon(self.s6_notify, "s6-notify", (zb,))
def _check_toolpaths(self) -> None:
for vol in self.asrv.vfs.all_vols.values():
if not vol.realpath or not vol.axs.uwrite:
continue
ap_vol = (vol.realpath + os.sep).encode("utf-8")
for zsl in (HAVE_FFMPEG, HAVE_FFPROBE, HAVE_DCRAW):
if zsl and zsl[0].startswith(ap_vol):
zs = zsl[0].decode("utf-8", "replace")
t = "will not use [%s] because it is inside a writable volume [/%s] => [%s] and --unsafe-tools is not enabled"
self.log("root", t % (zs, vol.vpath, vol.realpath), 3)
del zsl[0]
def _feature_test(self) -> None:
fok = []
fng = []
@ -1076,8 +1029,8 @@ class SvcHub(object):
# fmt: off
to_check = [
(HAVE_SQLITE3, "sqlite", "sessions and file/media indexing"),
(HAVE_PIL, "pillow", "image thumbnails (good and fast)"),
(HAVE_VIPS, "vips", "image thumbnails (eats ram; will prefer pillow)"),
(HAVE_PIL, "pillow", "image thumbnails (plenty fast)"),
(HAVE_VIPS, "vips", "image thumbnails (faster, eats more ram)"),
(H_PIL_WEBP, "pillow-webp", "create thumbnails as webp files"),
(HAVE_FFMPEG, "ffmpeg", t_ff + ", good-but-slow image thumbnails"),
(HAVE_FFPROBE, "ffprobe", t_ff + ", read audio/media tags"),
@ -1154,23 +1107,14 @@ class SvcHub(object):
t = "not listening on any ip-addresses (only unix-sockets and/or FDs); cannot enable zeroconf/mdns/ssdp as requested"
self.log("root", t, 3)
if al.wopi and not al.wopi_url and not al.wopi_urls:
self.log("root", "disabling wopi; need either wopi-url or wopi-urls", 1)
al.wopi = False
if self.args.wopi or not self.args.no_dav:
if not self.args.no_dav:
from .dxml import DXML_OK
if not DXML_OK:
self.args.wopi = False
self.args.no_dav = True
t = "WARNING:\nDisabling WebDAV and WOPI support because dxml selftest failed. Please report this bug;\n%s\n...and include the following information in the bug-report:\n%s | expat %s\n"
self.log("root", t % (URL_BUG, VERSIONS, expat_ver()), 1)
if self.args.wopi and self.args.j != 1:
self.args.wopi = False
t = "WARNING: Disabling --wopi because -j is not 1 (the default and recommended -j value)"
self.log("root", t, 1)
if not self.args.no_dav:
self.args.no_dav = True
t = "WARNING:\nDisabling WebDAV support because dxml selftest failed. Please report this bug;\n%s\n...and include the following information in the bug-report:\n%s | expat %s\n"
self.log("root", t % (URL_BUG, VERSIONS, expat_ver()), 1)
if (
not E.scfg
@ -1239,6 +1183,14 @@ class SvcHub(object):
if al.rsp_jtr:
al.rsp_slp = 0.000001
zsl = al.th_covers.split(",")
zsl = [x.strip() for x in zsl]
zsl = [x for x in zsl if x]
al.th_covers = zsl
al.th_coversd = zsl + ["." + x for x in zsl]
al.th_covers_set = set(al.th_covers)
al.th_coversd_set = set(al.th_coversd)
zs = al.use_bwrap.strip().lower()
if zs == "a":
if not HAVE_BWRAP:
@ -1414,20 +1366,18 @@ class SvcHub(object):
self.args.name_html = zs
zs = al.u2sz
if zs in ("no", "1,1,1"):
zs = "0"
zsl = [x.strip() for x in zs.split(",")]
if len(zsl) not in (1, 2):
t = "invalid --u2sz; must be either one number, or a comma-separated list of two numbers (target,max)"
if len(zsl) not in (1, 3):
t = "invalid --u2sz; must be either one number, or a comma-separated list of three numbers (min,default,max)"
raise Exception(t)
if len(zsl) < 2:
zsl = [zs, zs]
zi2 = 0
if len(zsl) < 3:
zsl = ["1", zs, zs]
zi2 = 1
for zs in zsl:
zi = int(zs)
# arbitrary constraint (anything above 2 GiB is probably unintended)
if zi < 0 or zi > 2047:
raise Exception("invalid --u2sz; minimum is 0, max is 2047")
if zi < 1 or zi > 2047:
raise Exception("invalid --u2sz; minimum is 1, max is 2047")
if zi < zi2:
raise Exception("invalid --u2sz; values must be equal or ascending")
zi2 = zi
@ -1496,7 +1446,7 @@ class SvcHub(object):
def _setup_logfile(self) -> None:
base_fn = fn = self._logname()
sel_fn = fn + self.lo2
do_xz = sel_fn.replace("%R", "").lower().endswith(".xz")
do_xz = sel_fn.lower().endswith(".xz")
if "%R" in self.args.lo:
# yup this is a race; if started sufficiently concurrently, two
# copyparties can grab the same logfile (considered and ignored)
@ -1512,31 +1462,18 @@ class SvcHub(object):
except:
pass
if self.args.rlo in ("del", "over"):
tmode = "wt"
bmode = "w"
else:
tmode = "at"
bmode = "a"
if self.args.rlo == "del":
try:
bos.unlink(fn)
except:
pass
try:
if do_xz:
import lzma
lh = lzma.open(fn, tmode, encoding="utf-8", errors="replace", preset=0)
lh = lzma.open(fn, "wt", encoding="utf-8", errors="replace", preset=0)
self.args.no_logflush = True
else:
lh = open(fn, tmode, encoding="utf-8", errors="replace")
lh = open(fn, "wt", encoding="utf-8", errors="replace")
except:
import codecs
lh = codecs.open(fn, bmode, encoding="utf-8", errors="replace")
lh = codecs.open(fn, "w", encoding="utf-8", errors="replace")
if getattr(self.args, "free_umask", False):
os.fchmod(lh.fileno(), 0o644)
@ -1551,8 +1488,6 @@ class SvcHub(object):
printed = "".join(lprinted) + msg
t = "t0: {:.3f}\nargv: {}\n\n{}"
lh.write(t.format(self.E.t0, " ".join(argv), printed))
if not self.args.no_logflush:
lh.flush()
self.logf = lh
self.logf_base_fn = base_fn
print(msg, end="")

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import calendar
import stat

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import re

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
try:
from types import SimpleNamespace

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import errno
import os
@ -34,7 +34,6 @@ class ThumbCli(object):
self.cooldown = Cooldown(self.args.th_poke) if self.args.th_poke else None
self.thumbable = c["thumbable"]
self.thumbable_native = c["thumbable_native"]
self.fmt_pil = c["pil"]
self.fmt_vips = c["vips"]
self.fmt_raw = c["raw"]
@ -60,10 +59,6 @@ class ThumbCli(object):
if is_vid and "dvthumb" in dbv.flags:
return None
is_custom = ext not in self.thumbable_native
if is_custom and "dethumb" in dbv.flags:
return None
want_opus = fmt in EXTS_AC
is_au = ext in self.fmt_ffa
is_vau = want_opus and ext in self.fmt_ffv
@ -81,7 +76,7 @@ class ThumbCli(object):
elif want_opus:
return None
is_img = not is_vid and not is_au and not is_custom
is_img = not is_vid and not is_au
if is_img and "dithumb" in dbv.flags:
return None

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import hashlib
import io
@ -31,14 +31,12 @@ from .mtag import (
from .util import BytesIO # type: ignore
from .util import (
FFMPEG_URL,
SCWD,
VF_CAREFUL,
Cooldown,
Daemon,
afsenc,
atomic_move,
fsenc,
loadpy,
min_ex,
runcmd,
statdir,
@ -324,12 +322,24 @@ class ThumbSrv(object):
if self.args.th_clean:
Daemon(self.cleaner, "thumb.cln")
self.fmt_pil = self.args.th_r_pil
self.fmt_vips = self.args.th_r_vips
self.fmt_raw = self.args.th_r_raw
self.fmt_ffi = self.args.th_r_ffi
self.fmt_ffv = self.args.th_r_ffv
self.fmt_ffa = self.args.th_r_ffa
(
self.fmt_pil,
self.fmt_vips,
self.fmt_raw,
self.fmt_ffi,
self.fmt_ffv,
self.fmt_ffa,
) = [
set(y.split(","))
for y in [
self.args.th_r_pil,
self.args.th_r_vips,
self.args.th_r_raw,
self.args.th_r_ffi,
self.args.th_r_ffv,
self.args.th_r_ffa,
]
]
if not H_PIL_HEIF:
for f in "heif heifs heic heics".split(" "):
@ -348,26 +358,23 @@ class ThumbSrv(object):
self.fmt_pil.discard(f)
self.thumbable: set[str] = set()
self.thumbable_native: set[str] = set()
self._build_thumbable()
def _build_thumbable(self) -> None:
self.thumbable_native.clear()
self.thumbable.clear()
if "pil" in self.args.th_dec:
self.thumbable_native |= self.fmt_pil
self.thumbable |= self.fmt_pil
if "vips" in self.args.th_dec:
self.thumbable_native |= self.fmt_vips
self.thumbable |= self.fmt_vips
if "raw" in self.args.th_dec:
self.thumbable_native |= self.fmt_raw
self.thumbable |= self.fmt_raw
if "ff" in self.args.th_dec:
for zss in [self.fmt_ffi, self.fmt_ffv, self.fmt_ffa]:
self.thumbable_native |= zss
self.thumbable = self.thumbable_native | set(self.args.th_extract)
self.thumbable |= zss
def _log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("thumb", msg, c)
@ -459,7 +466,6 @@ class ThumbSrv(object):
def getcfg(self) -> dict[str, set[str]]:
return {
"thumbable": self.thumbable,
"thumbable_native": self.thumbable_native,
"pil": self.fmt_pil,
"vips": self.fmt_vips,
"raw": self.fmt_raw,
@ -542,36 +548,19 @@ class ThumbSrv(object):
png_ok = False
funs = []
tex = tpath.rsplit(".", 1)[-1]
want_mp3 = tex == "mp3"
want_opus = tex in ("opus", "owa", "caf")
want_flac = tex == "flac"
want_wav = tex == "wav"
want_png = tex == "png"
want_au = want_mp3 or want_opus or want_flac or want_wav
ap_extr = abspath
if (
ext in self.args.th_extract
and not want_au
and not want_png
and "dethumb" not in vn.flags
):
ap_extr = self.run_extractor(self.args.th_extract[ext], abspath, vn)
if ap_extr:
ext = ap_extr.rsplit(".", 1)[-1]
if ext in self.args.au_unpk:
ap_unpk = au_unpk(self.log, self.args.au_unpk, ap_extr, vn)
elif ext in self.thumbable_native:
ap_unpk = ap_extr
ap_unpk = au_unpk(self.log, self.args.au_unpk, abspath, vn)
else:
ap_unpk = ""
if ap_extr and ap_extr != ap_unpk and ap_extr != abspath:
wunlink(self.log, ap_extr, vn.flags)
ap_unpk = abspath
if ap_unpk and not bos.path.exists(tpath):
tex = tpath.rsplit(".", 1)[-1]
want_mp3 = tex == "mp3"
want_opus = tex in ("opus", "owa", "caf")
want_flac = tex == "flac"
want_wav = tex == "wav"
want_png = tex == "png"
want_au = want_mp3 or want_opus or want_flac or want_wav
for lib in self.args.th_dec:
can_au = lib == "ff" and (
ext in self.fmt_ffa or ext in self.fmt_ffv
@ -679,66 +668,6 @@ class ThumbSrv(object):
with self.mutex:
self.nthr -= 1
def run_extractor(self, extr: str, abspath: str, vn: VFS) -> str:
try:
mod = loadpy(extr, self.args.hot_th_extr)
except Exception as ex:
self.log("extractor import failed; " + min_ex(), 1)
return ""
fd = 0
ret = ""
stream = None
try:
res = mod.main(abspath, vn=vn, th_srv=self)
if not res:
t = "extractor %r returned no data for file %r"
self.log(t % (extr, abspath))
return ""
outext, stream, offset, whence, size = res
if (
outext not in self.thumbable_native
or outext in self.fmt_ffa
or outext in self.fmt_ffv
):
t = "extractor %r returned unsupported format %r"
self.log(t % (extr, outext))
return ""
remains = size if size is not None and size >= 0 else -1
stream.seek(offset, whence)
fd, ret = tempfile.mkstemp("." + outext)
bufsz = rsz = self.args.iobuf
maxsz = self.args.th_extr_sz * 1048576
outsz = 0
with os.fdopen(fd, "wb", rsz) as f:
fd = 0
while True:
if remains < rsz and remains >= 0:
bufsz = remains
buf = stream.read(bufsz)
if not buf:
break
remains -= len(buf)
outsz += len(buf)
if outsz >= maxsz:
raise Exception("too large")
f.write(buf)
return ret
except Exception as e:
if fd:
os.close(fd)
if ret:
wunlink(self.log, ret, vn.flags)
t = "failed to extract thumbnail from %r: %s"
self.log(t % (abspath, min_ex()))
return ""
finally:
if stream:
stream.close()
def fancy_pillow(self, im: "Image.Image", fmt: str, vn: VFS) -> "Image.Image":
# exif_transpose is expensive (loads full image + unconditional copy)
res = self.getres(vn, fmt)
@ -845,7 +774,7 @@ class ThumbSrv(object):
self.wait4ram(0.6, tpath)
bap = fsenc(abspath)
# fmt: off
cmd = bwrap(HAVE_DCRAW[0], bap, b"") + [
cmd = bwrap(HAVE_DCRAW, bap, b"") + [
b"-h", # halfsize
b"-o", b"1", # srgb
b"-s", b"0", # first frame
@ -853,7 +782,7 @@ class ThumbSrv(object):
bap,
]
# fmt: on
p = sp.Popen(cmd, stdout=sp.PIPE, cwd=SCWD)
p = sp.Popen(cmd, stdout=sp.PIPE)
try:
if HAVE_PIL:
self.conv_image_pil(Image.open(p.stdout), tpath, fmt, vn)
@ -904,7 +833,7 @@ class ThumbSrv(object):
def conv_ffmpeg(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
self.wait4ram(0.2, tpath)
ret, raw, strms, ctnr = ffprobe(abspath, int(vn.flags["convt"] / 2))
ret, _, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if not ret:
return
@ -912,13 +841,6 @@ class ThumbSrv(object):
# audio in a video trenchcoat
return self.conv_spec(abspath, tpath, fmt, vn)
for strm in strms:
if (
strm.get("codec_type") == "video"
and strm.get("DISPOSITION:attached_pic") == "1"
):
return self.conv_emb_cv(abspath, tpath, fmt, vn, strm)
ext = abspath.rsplit(".")[-1].lower()
if ext in ["h264", "h265"] or ext in self.fmt_ffi:
seek: list[bytes] = []
@ -948,7 +870,7 @@ class ThumbSrv(object):
bap_in = fsenc(abspath)
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner"
@ -989,7 +911,7 @@ class ThumbSrv(object):
def _run_ff(self, cmd: list[bytes], vn: VFS, kto: str, oom: int = 400) -> None:
# self.log((b" ".join(cmd)).decode("utf-8"))
ret, _, serr = runcmd(cmd, cwd=SCWD, timeout=vn.flags[kto], nice=True, oom=oom)
ret, _, serr = runcmd(cmd, timeout=vn.flags[kto], nice=True, oom=oom)
if not ret:
return
@ -1091,7 +1013,7 @@ class ThumbSrv(object):
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1172,7 +1094,7 @@ class ThumbSrv(object):
bap_out = fsenc(infile)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1206,7 +1128,7 @@ class ThumbSrv(object):
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1244,7 +1166,7 @@ class ThumbSrv(object):
# to not support opus then it's probably also super picky
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1275,7 +1197,7 @@ class ThumbSrv(object):
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1312,7 +1234,7 @@ class ThumbSrv(object):
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1375,7 +1297,7 @@ class ThumbSrv(object):
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1418,7 +1340,7 @@ class ThumbSrv(object):
bap_out = fsenc(tmp_opus)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1445,7 +1367,7 @@ class ThumbSrv(object):
bap_in = fsenc(abspath)
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
@ -1467,7 +1389,7 @@ class ThumbSrv(object):
bap_in = fsenc(tmp_opus)
bap_out = fsenc(tpath)
# fmt: off
cmd = bwrap(HAVE_FFMPEG[0], bap_in, bap_out) + [
cmd = bwrap(HAVE_FFMPEG, bap_in, bap_out) + [
b"-nostdin",
b"-v", b"error",
b"-hide_banner",

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import calendar
import os
@ -28,14 +28,6 @@ from .util import (
if HAVE_SQLITE3:
import sqlite3
try:
if os.environ.get("PRTY_NO_UNIDATA"):
raise Exception()
import unicodedata
except:
pass
try:
from pathlib import Path
except:
@ -51,14 +43,6 @@ if PY2:
range = xrange # type: ignore
def norm_icase(txt: str) -> str:
return txt.casefold() if txt else txt
def norm_nfkc(txt: str) -> str:
return unicodedata.normalize("NFKC", txt).casefold() if txt else txt
class U2idx(object):
def __init__(self, hsrv: "HttpSrv") -> None:
self.log_func = hsrv.log
@ -71,10 +55,6 @@ class U2idx(object):
return
if self.args.srch_icase:
if self.args.srch_nfkc:
self.normfun = norm_nfkc
else:
self.normfun = norm_icase
self._open_db = self._open_db_icase
else:
self._open_db = self._open_db_std
@ -102,7 +82,7 @@ class U2idx(object):
def _open_db_icase(self, *args, **kwargs):
db = self._open_db_std(*args, **kwargs)
db.create_function("casefold", 1, self.normfun)
db.create_function("casefold", 1, lambda x: x.casefold() if x else x)
return db
def shutdown(self) -> None:
@ -343,7 +323,7 @@ class U2idx(object):
if icase and "casefold(" in q:
try:
v = self.normfun(unicode(v))
v = unicode(v).casefold()
except:
v = unicode(v).lower()

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import errno
import hashlib
@ -74,7 +74,6 @@ from .util import (
vsplit,
w8b64dec,
w8b64enc,
winsparse,
wunlink,
)
@ -1160,7 +1159,7 @@ class Up2k(object):
ft = "\033[0;32m{}{:.0}"
ff = "\033[0;35m{}{:.0}"
fv = "\033[0;36m{}:\033[90m{}"
zs = "bcasechk du_iwho emb_all emb_lgs emb_mds ext_th_d html_head html_head_d html_head_s ls_q_m put_name2 mv_re_r mv_re_t rm_re_r rm_re_t oh_f oh_g rw_edit_set srch_re_dots srch_re_nodot th_coversd th_coversl th_covers_set th_coversd_set zipmax zipmaxn_v zipmaxs_v"
zs = "bcasechk du_iwho emb_all emb_lgs emb_mds ext_th_d html_head html_head_d html_head_s ls_q_m put_name2 mv_re_r mv_re_t rm_re_r rm_re_t oh_f oh_g rw_edit_set srch_re_dots srch_re_nodot zipmax zipmaxn_v zipmaxs_v"
fx = set(zs.split())
fd = vf_bmap()
fd.update(vf_cmap())
@ -1432,8 +1431,6 @@ class Up2k(object):
rtop,
rei,
reh,
vol.flags["th_coversd"],
vol.flags["th_coversd_set"],
n4g,
ffat,
[],
@ -1511,8 +1508,6 @@ class Up2k(object):
rcdir: str,
rei: Optional[Pattern[str]],
reh: Optional[Pattern[str]],
th_cvd: list[str],
th_cvds: set[str],
n4g: bool,
ffat: bool,
seen: list[str],
@ -1538,6 +1533,8 @@ class Up2k(object):
cv = vcv = acv = ""
e_d = {}
th_cvd = self.args.th_coversd
th_cvds = self.args.th_coversd_set
scan_pr_s = self.args.scan_pr_s
assert self.pp and self.mem_cur # !rm
@ -1604,8 +1601,6 @@ class Up2k(object):
rap,
rei,
reh,
th_cvd,
th_cvds,
n4g,
fat32,
seen,
@ -1636,7 +1631,7 @@ class Up2k(object):
rsz += sz
files.append((sz, lmod, iname))
if sz and th_cvd:
if sz:
liname = iname.lower()
ext = liname.rsplit(".", 1)[-1]
if (
@ -4122,7 +4117,31 @@ class Up2k(object):
with self.rescan_cond:
self.rescan_cond.notify_all()
if vflags and "nodirsz" not in vflags:
if rd and sz and fn.lower() in self.args.th_coversd_set:
# wasteful; db_add will re-index actual covers
# but that won't catch existing files
crd, cdn = rd.rsplit("/", 1) if "/" in rd else ("", rd)
try:
q = "select fn from cv where rd=? and dn=?"
db_cv = db.execute(q, (crd, cdn)).fetchone()[0]
db_lcv = db_cv.lower()
if db_lcv in self.args.th_coversd_set:
idx_db = self.args.th_coversd.index(db_lcv)
idx_fn = self.args.th_coversd.index(fn.lower())
add_cv = idx_fn < idx_db
else:
add_cv = True
except:
add_cv = True
if add_cv:
try:
db.execute("delete from cv where rd=? and dn=?", (crd, cdn))
db.execute("insert into cv values (?,?,?)", (crd, cdn, fn))
except:
pass
if "nodirsz" not in vflags:
try:
q = "update ds set nf=nf+1, sz=sz+? where rd=?"
q2 = "insert into ds values(?,?,1)"
@ -4735,7 +4754,7 @@ class Up2k(object):
fsize,
ip,
time.time(),
[dabs],
None,
)
t = hr.get("rejectmsg") or ""
if t or hr.get("rc") != 0:
@ -4778,7 +4797,7 @@ class Up2k(object):
fsize,
ip,
time.time(),
[sabs],
None,
)
return "k"
@ -4908,7 +4927,7 @@ class Up2k(object):
fsize,
ip,
time.time(),
[sabs],
None,
)
return "k"
@ -5316,12 +5335,9 @@ class Up2k(object):
and self.args.sparse * 1024 * 1024 <= sz
):
try:
winsparse(f)
# some kernels/etc are buggy; force sync:
f.close()
f = open(abspath, "rb+")
except Exception as ex:
self.log("could not sparse %r: %s" % (abspath, ex), 3)
sp.check_call(["fsutil", "sparse", "setflag", abspath])
except:
self.log("could not sparse %r" % (abspath,), 3)
relabel = True
sprs = False
@ -5708,7 +5724,7 @@ def up2k_chunksize(filesize: int) -> int:
stepsize = 512 * 1024
while True:
for mul in [1, 2]:
nchunks = -int(-filesize // chunksize)
nchunks = math.ceil(filesize * 1.0 / chunksize)
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
return chunksize

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import argparse
import base64
@ -429,7 +429,6 @@ IMPLICATIONS = [
["e2vu", "e2v"],
["e2vp", "e2v"],
["e2v", "e2d"],
["srch_nfkc", "srch_icase"],
["hardlink_only", "hardlink"],
["hardlink", "dedup"],
["tftpvv", "tftpv"],
@ -678,12 +677,6 @@ except:
HAVE_BWRAP = ""
if ANYWIN:
SCWD = os.environ["systemroot"]
else:
SCWD = None
def py_desc() -> str:
interp = platform.python_implementation()
py_ver = ".".join([str(x) for x in sys.version_info])
@ -1361,7 +1354,7 @@ class MTHash(object):
self.csz = chunksz
chunks: dict[int, tuple[str, int, int]] = {}
nchunks = -int(-fsz // chunksz)
nchunks = int(math.ceil(fsz / chunksz))
for nch in range(nchunks):
self.work_q.put(nch)
@ -2150,8 +2143,7 @@ class MultipartParser(object):
boundary = get_boundary(self.headers)
if boundary.startswith('"') and boundary.endswith('"'):
boundary = boundary[1:-1] # dillo uses quotes
if len(boundary) > 72: # rfc-2046 <=70
raise Pebkac(400, "boundary 2big: %d" % (len(boundary),))
self.log("boundary=%r" % (boundary,))
# spec says there might be junk before the first boundary,
# can't have the leading \r\n if that's not the case
@ -2955,25 +2947,6 @@ def trystat_shutil_copy2(log: "NamedLogger", src: bytes, dst: bytes) -> bytes:
raise
def trystat_shutil_move(log: "NamedLogger", src: bytes, dst: bytes) -> bytes:
try:
return shutil.move(src, dst)
except:
# ignore failed mtime on linux+ntfs; for example:
# shutil.py:437 <copy2>: copystat(src, dst, follow_symlinks=follow_symlinks)
# shutil.py:376 <copystat>: lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
# [PermissionError] [Errno 1] Operation not permitted, '/windows/_videos'
_, _, tb = sys.exc_info()
for _, _, fun, _ in traceback.extract_tb(tb):
if fun == "copystat":
if log:
t = "warning: failed to retain some file attributes (timestamp and/or permissions) during move from %r to %r:\n%s"
log(t % (src, dst, min_ex()), 3)
os.unlink(src)
return dst # close enough
raise
def _fs_mvrm(
log: "NamedLogger", src: str, dst: str, atomic: bool, flags: dict[str, Any]
) -> bool:
@ -3025,9 +2998,7 @@ def _fs_mvrm(
if not attempt and ex.errno == errno.EXDEV:
t = "using copy+delete (%s)\n %s\n %s"
log(t % (ex.strerror, src, dst))
if osfun in (os.replace, os.rename):
args.insert(0, log)
osfun = trystat_shutil_move
osfun = shutil.move
continue
if now - t0 > maxtime or attempt == 90209:
raise
@ -3064,7 +3035,7 @@ def atomic_move(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -
os.unlink(bdst)
except:
pass
trystat_shutil_move(log, bsrc, bdst) # type: ignore
shutil.move(bsrc, bdst) # type: ignore
def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
@ -4061,7 +4032,7 @@ def runihook(
t0 = time.time()
if fork:
Daemon(runcmd, cmd, [bcmd], ka=sp_ka)
Daemon(runcmd, cmd, bcmd, ka=sp_ka)
else:
rc, v, err = runcmd(bcmd, **sp_ka) # type: ignore
if chk and rc:
@ -4227,10 +4198,6 @@ def _runhook(
if src in ("xm", "xban"):
ja["txt"] = txt[0]
ja["body"] = txt[1]
elif src == "xbr":
ja["ap_to"] = txt[0]
elif src in ("xar.ln", "xar.mv"):
ja["ap_from"] = txt[0]
else:
ja["wark"] = txt[0] # acshually the dwark but less confusing
if imp:
@ -4239,7 +4206,7 @@ def _runhook(
return mod.main(ja)
arg = json.dumps(ja)
else:
arg = txt[0] if txt and src in ("xm", "xban") else ap
arg = txt[0] if txt else ap
if acmd[0].startswith("zmq:"):
zi, zs = _zmq_hook(log, verbose, src, acmd[0][4:].lower(), arg, wait, sp_ka)
@ -4534,18 +4501,6 @@ def hidedir(dp) -> None:
pass
def winsparse(f: typing.BinaryIO) -> None:
assert ctypes # !rm
assert wk32 # !rm
import msvcrt
fh = msvcrt.get_osfhandle(f.fileno())
if not wk32.DeviceIoControl(
fh, 0x900C4, None, 0, None, 0, ctypes.byref(ctypes.c_ulong()), None
):
raise ctypes.WinError(ctypes.get_last_error())
_flocks = {}

View file

@ -121,8 +121,6 @@
<h2 id="wfp"><a href="{{ r }}/?h" id="goh">control-panel</a></h2>
<p id="js_bork">Looks like javascript is broken; see "csp nonce" section in readme</p>
<a href="#" id="repl">π</a>
</div>

View file

@ -46,7 +46,6 @@ if (1)
["ctrl-V", "paste (move/copy) here"],
["Y", "download selected"],
["F2", "rename selected"],
["F4", "update/refresh"],
"file-list-sel",
["space", "toggle file selection"],
@ -758,7 +757,7 @@ if (window.glang && navigator.languages && !/\bcplng=/.test(document.cookie))
if (window.stop)
window.stop();
document.body.innerHTML = 'Loading ' + n;
setck("cplng=" + n, relod);
setck("cplng=" + n, location.reload.bind(location));
crashed = true;
throw 1;
}
@ -1246,11 +1245,8 @@ check_image_support('jxl', "data:image/jxl;base64,/woIAAAMABKIAgC4AF3lEgA=");
var img_re = APPLE ?
/\.(a?png|avif|bmp|gif|hei[cf]s?|jpe?g|jfif|svg|webp|webm|mkv|mp4|m4v|mov|svg)(\?|$)/i :
/\.(a?png|avif|bmp|gif|jpe?g|jfif|svg|webp|webm|mkv|mp4|m4v|mov|svg)(\?|$)/i;
var wopi_set = !window.have_wopi ? null :
new Set('odt fodt ott doc docx dotx rtf odm ods fods ots xls xlsx odp fodp otp ppt pptx ppsx odg fodg otg odf'.split(' '));
/\.(a?png|avif|bmp|gif|hei[cf]s?|jpe?g|jfif|svg|webp|webm|mkv|mp4|m4v|mov)(\?|$)/i :
/\.(a?png|avif|bmp|gif|jpe?g|jfif|svg|webp|webm|mkv|mp4|m4v|mov)(\?|$)/i;
function set_files_html(html) {
@ -4062,11 +4058,6 @@ var fileman = (function () {
}
shui.style.display = 'block';
var wr = has(perms, 'write') && !fns.length;
var nope = ['admin', 'move', 'delete', 'upget'];
if (!wr)
nope.push('write');
var html = [
'<div>',
'<table>',
@ -4087,10 +4078,10 @@ var fileman = (function () {
'<tr><td>perms</td><td class="sh_axs">',
];
for (var a = 0; a < perms.length; a++)
if (!has(nope, perms[a]))
if (!has(['admin', 'move', 'delete', 'upget'], perms[a]))
html.push('<a href="#" class="tgl btn">' + perms[a] + '</a>');
if (wr)
if (has(perms, 'write'))
html.push('<a href="#" class="btn">write-only</a>');
html.push('</td></tr></div');
@ -5694,8 +5685,8 @@ var thegrid = (function () {
fid = oth.getAttribute('id'),
aplay = ebi('a' + fid),
atext = ebi('t' + fid),
is_txt = atext && !/\.ts$/.test(href) && showfile.getlang(href),
is_img = img_re.test(href),
is_txt = atext && !is_img && !/\.ts$/.test(href) && showfile.getlang(href),
is_dir = href.endsWith('/'),
is_srch = !!ebi('unsearch'),
in_tree = is_dir && treectl.find(oth.textContent.slice(0, -1)),
@ -6439,9 +6430,6 @@ var ahotkeys = function (e) {
if (k == 'F2')
return fileman.rename();
if (k == 'F4')
return treectl.goto();
if (!treectl.hidden && (!sh || !thegrid.en)) {
if (kl == 'a')
return QS('#twig').click();
@ -7693,11 +7681,6 @@ var treectl = (function () {
tn.href = addq(tn.href, 'v');
}
if (wopi_set && wopi_set.has(tn.ext))
tn.lead = '<a href="?wopi=' + bhref +
'" rel="nofollow" target="_blank" name="' + hname + '">📄</a>';
if (tn.lead == '-')
tn.lead = '<a href="?doc=' + bhref + '" id="t' + id +
'" rel="nofollow" class="doc' + (lang ? ' bri' : '') +
@ -7790,7 +7773,6 @@ var treectl = (function () {
r.hydrate = function () {
qsr('#bbsw');
qsr('#js_bork');
srvinf = ebi('srv_info').innerHTML.slice(6, -7);
if (ls0 === null) {
r.ls_cb = showfile.addlinks;
@ -8626,7 +8608,7 @@ var setfszf = (function () {
setck('cplng=' + lang);
freshen();
var t = L.tt == 'English' ? '' : Ls.eng.lang_set;
modal.confirm(L.lang_set + "\n\n" + t, relod, null);
modal.confirm(L.lang_set + "\n\n" + t, location.reload.bind(location), null);
}
freshen();
@ -8877,7 +8859,7 @@ var msel = (function () {
r.evsel = function (e, fun) {
ev(e);
r.so = r.pr = null;
var trs = QSA('#files tbody tr' + (ebi('unsearch') ? ':not(.srch_hdr)' : ''));
var trs = QSA('#files tbody tr');
for (var a = 0, aa = trs.length; a < aa; a++)
clmod(trs[a], 'sel', fun);
r.selui();

View file

@ -325,9 +325,6 @@ var modpoll = new Modpoll();
window.onbeforeunload = function (e) {
if (window.gtfo)
return;
if ((ebi("save").className + '').indexOf('disabled') >= 0)
return; //nice (todo)

View file

@ -39,7 +39,6 @@ Ls.chi = {
["ctrl-V", "粘贴(移动/复制)到此处"],
["Y", "下载选中项"],
["F2", "重命名选中项"],
["F4", "刷新"], //m
"文件列表选择",
["space", "切换文件选择"],

View file

@ -39,7 +39,6 @@ Ls.cze = {
["ctrl-V", "vložit (přesunout/kopírovat) zde"],
["Y", "stáhnout vybrané"],
["F2", "přejmenovat vybrané"],
["F4", "obnovit"], //m
"výběr souborů",
["space", "přepnout výběr souboru"],

View file

@ -39,7 +39,6 @@ Ls.deu = {
["STRG-V", "Zwischenablage hier einfügen"],
["Y", "Auswahl herunterladen"],
["F2", "Auswahl umbenennen"],
["F4", "Aktualisieren"], //m
"file-list-sel",
["LEER", "Dateiauswahl aktivieren"],

View file

@ -39,7 +39,6 @@ Ls.epo = {
["stir-V", "alglui (movi/kopii) ĉi tien"],
["Y", "elŝuti elektitajn"],
["F2", "alinomi elektitajn"],
["F4", "refreŝigi"], //m
"file-list-sel",
["spacoklavo", "baskuli elektadon de dosieroj"],

View file

@ -39,7 +39,6 @@ Ls.fin = {
["ctrl-V", "siirrä tai kopioi tähän"],
["Y", "lataa valitut"],
["F2", "uudelleennimeä valitut"],
["F4", "päivitä"], //m
"file-list-sel",
["space", "vaihda tiedostonvalintatilaan"],

View file

@ -39,7 +39,6 @@ Ls.fra = {
["ctrl-V", "coller (déplacer/copier) ici"],
["Y", "télécharger la sélection"],
["F2", "renomer la sélection"],
["F4", "actualiser"], //m
"file-list-sel",
["Espace", "activer la sélection de fichiers"],

View file

@ -39,7 +39,6 @@ Ls.grc = {
["ctrl-V", "επικόλληση (μετακίνηση/αντιγραφή) εδώ"],
["Y", "λήψη επιλεγμένων"],
["F2", "μετονομασία επιλεγμένων"],
["F4", "ανανέωση"], //m
"λίστα αρχείων",
["space", "εναλλαγή επιλογής αρχείου"],

View file

@ -36,7 +36,6 @@ Ls.hun = {
['ctrl-V', 'beillesztés ide'],
['Y', 'kijelöltek letöltése'],
['F2', 'átnevezés'],
["F4", "frissítés"], //m
'kijelölés',
['space', 'fájl kijelölése'],

View file

@ -39,7 +39,6 @@ Ls.ita = {
["ctrl-V", "incolla (sposta/copia) qui"],
["Y", "scarica selezionati"],
["F2", "rinomina selezionati"],
["F4", "aggiorna"], //m
"file-list-sel",
["spazio", "alterna selezione file"],

View file

@ -39,7 +39,6 @@ Ls.jpn = {
["ctrl-V", "ここに貼り付け(移動/コピー)"],
["Y", "選択した項目をダウンロード"],
["F2", "選択した項目の名前を変更"],
["F4", "更新"], //m
"file-list-sel",
["space", "ファイル選択の切り替え"],

View file

@ -39,7 +39,6 @@ Ls.kor = {
["ctrl-V", "여기에 붙여넣기 (이동/복사)"],
["Y", "선택 항목 다운로드"],
["F2", "선택 항목 이름 바꾸기"],
["F4", "새로 고침"], //m
"파일 목록 선택",
["space", "파일 선택/해제"],

View file

@ -39,7 +39,6 @@ Ls.nld = {
["ctrl-V", "Hier plakken (verplaatsen/kopieëren)"],
["Y", "Download geselecteerde"],
["F2", "Hernoem geselecteerde"],
["F4", "vernieuwen"], //m
"bestand-lijst-selectie",
["space", "wissel bestand selectie"],

View file

@ -36,7 +36,6 @@ Ls.nno = {
["ctrl-V", "lim inn (flytt/kopiér)"],
["Y", "last ned valde"],
["F2", "endre namn på valde"],
["F4", "oppdater"],
"filmarkering",
["space", "markér fil"],

View file

@ -36,7 +36,6 @@ Ls.nor = {
["ctrl-V", "lim inn (flytt/kopiér)"],
["Y", "last ned valgte"],
["F2", "endre navn på valgte"],
["F4", "oppdater"],
"filmarkering",
["space", "marker fil"],

View file

@ -39,7 +39,6 @@ Ls.pol = {
["ctrl-V", "wklej (przenieś/skopiuj) tutaj"],
["Y", "pobierz zaznaczone"],
["F2", "zmień nazwę zaznaczonych"],
["F4", "odśwież"], //m
"file-list-sel",
["spacja", "przełącz zaznaczanie plików"],

View file

@ -39,7 +39,6 @@ Ls.por = {
["ctrl-V", "colar (mover/copiar) aqui"],
["Y", "baixar selecionado"],
["F2", "renomear selecionado"],
["F4", "atualizar"], //m
"seleção de lista de arquivos",
["espaço", "alternar seleção de arquivo"],

View file

@ -39,7 +39,6 @@ Ls.rus = {
["ctrl-V", "вставить (переместить/копировать) сюда"],
["Y", "скачать выделенное"],
["F2", "переименовать выделенное"],
["F4", "обновить"], //m
"выделение файлов",
["пробел", "выделить/снять выделение с текущего файла"],

View file

@ -39,7 +39,6 @@ Ls.spa = {
["ctrl-V", "pegar (mover/copiar) aquí"],
["Y", "descargar seleccionados"],
["F2", "renombrar seleccionados"],
["F4", "actualizar"], //m
"selección en lista de archivos",
["space", "alternar selección de archivo"],

View file

@ -39,7 +39,6 @@ Ls.swe = {
["ctrl-V", "klistra in (kopiera/flytta) hit"],
["Y", "ladda ner urval"],
["F2", "byt namn på urval"],
["F4", "uppdatera"], //m
"välja filer",
["Blanksteg", "växla val av fil"],

View file

@ -39,7 +39,6 @@ Ls.tur = {
["ctrl-V", "buraya yapıştır (taşı/kopyala)"],
["Y", "seçileni indir"],
["F2", "seçileni yeniden adlandır"],
["F4", "yenile"], //m
"dosya yönetimi seçimleri",
["boşluk", "seçimi değiştir"],

View file

@ -39,7 +39,6 @@ Ls.ukr = {
["ctrl-V", "вставити (перемістити/копіювати) сюди"],
["Y", "завантажити вибране"],
["F2", "перейменувати вибране"],
["F4", "оновити"], //m
"вибір файлів у списку",
["space", "перемкнути вибір файлу"],

View file

@ -36,7 +36,6 @@ Ls.vie = {
["ctrl-V", "dán (di chuyển/sao chép) tại đây"],
["Y", "tải xuống mục đã chọn"],
["F2", "đổi tên mục đã chọn"],
["F4", "làm mới"], //m
"file-list-sel",
["space", "chuyển đổi chọn tệp"],

View file

@ -888,8 +888,9 @@ function up2k_init(subtle) {
setmsg(suggest_up2k, 'msg');
var u2szs = u2sz.split(','),
u2sz_tgt = parseInt(u2szs[0]),
u2sz_max = parseInt(u2szs[1]);
u2sz_min = parseInt(u2szs[0]),
u2sz_tgt = parseInt(u2szs[1]),
u2sz_max = parseInt(u2szs[2]);
var parallel_uploads = ebi('nthread').value = icfg_get('nthread', u2j),
stitch_tgt = ebi('u2szg').value = icfg_get('u2sz', u2sz_tgt),
@ -997,14 +998,6 @@ function up2k_init(subtle) {
});
}
function unqueue_up(t) {
t.postlist = [];
var arr = st.todo.upload;
for (var a = arr.length - 1; a >= 0; a--)
if (arr[a].nfile == t.n)
arr.splice(a, 1);
}
var pvis = new U2pvis("bz", '#u2cards', uc, st),
donut = new Donut(uc, st);
@ -1076,28 +1069,25 @@ function up2k_init(subtle) {
return onovercmn(this, e, true);
}
function onovercmn(self, e, btn) {
var ok = false;
try {
var dt = e.dataTransfer.types;
for (var a = 0; a < dt.length && !ok; a++)
var ok = false, dt = e.dataTransfer.types;
for (var a = 0; a < dt.length; a++)
if (dt[a] == 'Files')
ok = true;
for (var a = 0; a < dt.length && !ok; a++)
if (dt[a] == 'text/uri-list') {
if (btn)
else if (dt[a] == 'text/uri-list') {
if (btn) {
ok = true;
if (toast.txt == L.u_uri)
toast.hide();
}
else
return toast.inf(10, L.u_uri) || true;
}
if (toast.txt == L.u_uri && ok)
toast.hide();
if (!ok)
return true;
}
catch (ex) { }
if (!ok)
return true;
ev(e);
try {
@ -2519,7 +2509,6 @@ function up2k_init(subtle) {
keepalive = t.keepalive,
me = Date.now();
while (apop(st.todo.handshake, t));
if (t.done)
return console.log('done; skip hs', t.name, t);
@ -2638,7 +2627,7 @@ function up2k_init(subtle) {
cbd.push(a == cdr_idx ? cdr_sz : chunksize);
}
unqueue_up(t);
t.postlist = [];
t.wark = response.wark;
var missing = response.hash;
for (var a = 0; a < missing.length; a++) {
@ -2676,7 +2665,7 @@ function up2k_init(subtle) {
ccs += stp; stp *= 2;
}
ocs = Math.floor(ocs / 1024 / 1024);
t.stitch_sz = Math.min(ocs, stitch_tgt || 1);
t.stitch_sz = Math.min(ocs, stitch_tgt);
}
for (var a = 0; a < t.postlist.length; a++) {
@ -2878,8 +2867,7 @@ function up2k_init(subtle) {
nparts = upt.nparts,
pcar = nparts[0],
pcdr = nparts[nparts.length - 1],
maxmsz = stitch_tgt == u2sz_tgt ? u2sz_max : stitch_tgt,
maxsz = (maxmsz > 0 ? maxmsz : 2040) * 1024 * 1024;
maxsz = (u2sz_max > 1 ? u2sz_max : 2040) * 1024 * 1024;
if (t.done)
return console.log('done; skip chunk', t.name, t);
@ -2928,7 +2916,8 @@ function up2k_init(subtle) {
var txt = unpre((xhr.response && xhr.response.err) || xhr.responseText);
if (txt.indexOf('upload blocked by x') + 1) {
apop(st.busy.upload, upt);
unqueue_up(t);
for (var a = pcar; a <= pcdr; a++)
apop(t.postlist, a);
pvis.seth(t.n, 1, "ERROR");
pvis.seth(t.n, 2, txt.split(/\n/)[0]);
pvis.move(t.n, 'ng');
@ -2951,13 +2940,8 @@ function up2k_init(subtle) {
st.etac.u++;
st.etac.t++;
}
else if (xhr.status == 413) {
modal.die('bad server config; see "u2sz" in readme');
}
else if (txt.indexOf('already got that') + 1) {
unqueue_up(t);
}
else if (txt.indexOf('already being written') + 1) {
else if (txt.indexOf('already got that') + 1 ||
txt.indexOf('already being written') + 1) {
t.nojoin = t.nojoin || t.postlist.length;
console.log("ignoring dupe-segment with backoff", t.nojoin, t.name, t);
if (!toast.visible && st.todo.upload.length < 4)
@ -2965,10 +2949,6 @@ function up2k_init(subtle) {
}
else {
xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), "404, target folder not found (???)", "warn", t);
if (txt.indexOf('unknown wark') == 0) {
t.cooldown = t.coolmul = 0;
unqueue_up(t);
}
chill(t);
}
orz2(xhr);
@ -2979,10 +2959,8 @@ function up2k_init(subtle) {
apop(t.postlist, a);
if (!t.postlist.length) {
t.t_uploaded = Date.now();
if (!has(st.busy.handshake, t)) {
pvis.seth(t.n, 1, 'verifying');
st.todo.handshake.unshift(t);
}
pvis.seth(t.n, 1, 'verifying');
st.todo.handshake.unshift(t);
}
tasker();
}
@ -3017,7 +2995,6 @@ function up2k_init(subtle) {
if (!toast.visible)
toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), t);
unqueue_up(t); // maybe unknown wark (ff drops undrained rsp)
t.nojoin = t.nojoin || t.postlist.length; // maybe rproxy postsize limit
console.log('chunkpit onerror,', t.name, t);
orz2(xhr);
@ -3166,7 +3143,7 @@ function up2k_init(subtle) {
var el = ebi('u2szg'), n = parseInt(el.value);
stitch_tgt = n = (
isNaN(n) ? u2sz_tgt :
n < 0 ? 0 :
n < u2sz_min ? u2sz_min :
n > u2sz_max ? u2sz_max : n
);
if (n == u2sz_tgt) sdrop('u2sz'); else swrite('u2sz', n);
@ -3403,7 +3380,7 @@ function up2k_init(subtle) {
function set_nosubtle(v) {
if (!WebAssembly)
return toast.err(10, L.u_nowork);
modal.confirm(L.lang_set, relod, null);
modal.confirm(L.lang_set, location.reload.bind(location), null);
}
function set_upnag(en) {
@ -3474,9 +3451,6 @@ function up2k_init(subtle) {
function warn_uploader_busy(e) {
if (window.gtfo)
return;
e.preventDefault();
e.returnValue = '';
return "upload in progress, click abort and use the file-tree to navigate instead";

View file

@ -108,13 +108,6 @@ function qsr(sel) {
}
function relod() {
crashed = 1;
window.gtfo = 1;
location.href = location.href;
}
// error handler for mobile devices
function esc(txt) {
return txt.replace(/[&"<>]/g, function (c) {
@ -296,7 +289,7 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
var x = ebi('exh_wipecfg');
if (x) x.onclick = function () {
localStorage.clear();
relod();
location.reload();
};
x = ebi('exh_ignex'); if (x) x.onclick = ignex;
x = ebi('exh_ignexa'); if (x) x.onclick = ignexa;
@ -1206,10 +1199,8 @@ function has(haystack, needle) {
function apop(arr, v) {
var ofs = arr.indexOf(v);
if (ofs !== -1) {
if (ofs !== -1)
arr.splice(ofs, 1);
return true;
}
}
@ -1475,7 +1466,7 @@ function Debounce(delay) {
};
r.rm = function (fun) {
while (apop(r.q, fun));
apop(r.q, fun);
};
r.run = function () {
@ -1535,7 +1526,7 @@ var timer = (function () {
};
r.rm = function (fun) {
while (apop(r.q, fun));
apop(r.q, fun);
};
var doevents = function () {
@ -2028,12 +2019,6 @@ var modal = (function () {
q.shift()();
}
r.die = function (html) {
crashed = 1;
if (r.busy)
r.hide();
_alert(lf2br(html), relod);
};
r.alert = function (html, cb, fun) {
q.push(function () {
_alert(lf2br(html), cb, fun);

View file

@ -1,42 +0,0 @@
<!DOCTYPE html>
<html lang="en" id="ht_wopi">
<head>
<meta charset="utf-8"/>
<title>{{ title|e }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="robots" content="noindex, nofollow">
<meta name="theme-color" content="#{{ tcolor }}">
<style>
body,#w {
margin: 0;
padding: 0;
border: 0;
width: 100%;
height: 100%;
}
body {
width: 100vw;
height: 100vh;
overflow: hidden;
}
</style></head><body>
<div style="display: none">
<form action="{{ url }}" enctype="multipart/form-data" method="post" target="w" id="submit-form">
<input name="access_token" value="{{ atoken }}" type="hidden" id="access-token"/>
<input name="access_token_ttl" value="{{ ttl }}" type="hidden" id="access-token_ttl"/>
<input type="submit" value="" />
</form>
</div>
<iframe id="w" name="w" allow="clipboard-read *; clipboard-write *; fullscreen *"></iframe>
<script nonce="{{ js_nonce }}">
document.getElementById("submit-form").submit();
</script>
</body>
</html>

View file

@ -1,78 +1,3 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2026-0727-2202 `v1.20.19` SECURITY: fix FTP upload outside volume
## ⚠️ ATTN: this release fixes a vulnerability in FTP and FTPS (not SFTP)
the FTP-server (default-disabled) would allow uploading to any folder that the copyparty process had permission to write to, but with certain limitations; see [GHSA-phv8-wgjp-g4p9](https://github.com/9001/copyparty/security/advisories/GHSA-phv8-wgjp-g4p9)
## recent important news
* [v1.20.19 (2026-07-27)](https://github.com/9001/copyparty/releases/tag/v1.20.19) fixed an FTP-server vuln (upload outside defined volumes)
* [v1.20.17 (2026-07-06)](https://github.com/9001/copyparty/releases/tag/v1.20.17) fixed a vuln when a volume has both filekeys and dirkeys enabled
* [v1.20.17 (2026-07-06)](https://github.com/9001/copyparty/releases/tag/v1.20.17) introduced csp nonces, possibly breaking some javascript-based plugins
## 🧪 new features
* #1495 wopi integration -- edit office documents in the web-ui (thx @brandon-doornbos!) d57bb0c7 10db4236
* currently only works with collabora online as wopi client #1574 and there's no docs/examples #1575
* thumbnails: use embedded cover-image in videos when available 59524018
* thumbnails: folder-thumbs can be disabled with `th-covers: no` (volflag or global) 856fada3
* #1555 macos: add [--srch-nfkc](https://copyparty.eu/cli/#g-srch-nfkc) to fix searching for filenames / paths in CJK languages a7c99094
* reduces search performance to around 30% when enabled
* hooks: `xbr` / `xar` did not include old/new abspath as parameters; now they do c122e103
## 🩹 bugfixes
* ftp: fix [GHSA-phv8-wgjp-g4p9](https://github.com/9001/copyparty/security/advisories/GHSA-phv8-wgjp-g4p9) b331bb1c
* #1563 moving files between volumes could fail depending on OS and underlying filesystem c70dc7ac
* fix drag-drop uploading in certain glitchy KDE environments (thx @tilse!) daf144af
* hooks: `xiu` crashed if the fork-flag was set (thx @stackxp!) aa862353
* hooks: `xau` without json-flag would be given the wark (file hash) instead of the abspath bae77b90
* shares: fix markdown-viewer (`?v`) inside shares 6a9437b7
* ftp: fix logging from `xbu` hooks 9912a951
* fix slow boot if a volume had lots of files in its toplevel folder cdb474c8
* python2.7: fix multithreaded file-hashing 0f2040c3
## 🔧 other changes
* #1556 the libvips thumbnailer was demoted to last-fallback due to frequently using excessive amounts of ram ed0be42a
* if [-lo](https://copyparty.eu/cli/#g-lo) points to an existing file, it will now be appended to instead of overwritten depending on [--rlo](https://copyparty.eu/cli/#g-help-rlo) b6abc33f
* #1530 nixos: the nix package now uses `ffmpeg-headless` instead of `ffmpeg-full` (thx @nyakase!) fface524
* slightly longer session cookie (was 20, now 24 chars) 537a99df
* Windows-specific:
* add a warning regarding the risks of DLL-hijacking when relevant bb40804f
* fix some trivial PATH-related footguns cea97ac6
* faster creation of sparse files bc45299b
* fix detection of filesystem characteristics 6226858b
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2026-0709-2254 `v1.20.18` bwrap off
## 🧪 new features
* [shadowing](https://github.com/9001/copyparty/#shadowing): filesystem-path `//NULL` will fully unmap a volume from the server's filesystem (no more `/var/lib/empties/4` jank) 88846061
## 🩹 bugfixes
* fix a false-positive warning on startup 12878e6c1cbf4417da42d46230dff686dd48760b
## 🔧 other changes
* #1552 #1551 default-disabled the bwrap ffmpeg sandbox; 85a8cc99
* sandboxing can fail in too many creative ways (funky linux distros with funky filesystem layouts and policies) so autoconfiguring bwrap is not feasible
* to enable it, set `use-bwrap: y` after adjusting [th-bwrap](https://copyparty.eu/cli/#g-th-bwrap) to match your OS/env
* see the default `th-bwrap` value in `--help` on your server for a best-effort guess
* always rotate (never overwrite) `-lo` logfiles on startup unless `--rlo no` 29e145f5
* thumbnails: save a tiny bit of I/O by not refreshing expiration-times if `th-clean` (expiration) is disabled anyways e15f2927
## 🌠 fun facts
* no
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2026-0706-1937 `v1.20.17` SECURITY: fix dirkeys

View file

@ -26,7 +26,7 @@ def up2k_chunksize(filesize):
stepsize = 512 * 1024
while True:
for mul in [1, 2]:
nchunks = -int(-filesize // chunksize)
nchunks = math.ceil(filesize * 1.0 / chunksize)
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
return chunksize

View file

@ -1,7 +1,6 @@
# you can include additional config like this
# (the space after the % is important)
#
# specify either a single file or a folder with .conf files;
# since copyparty.d is a folder, it'll include all *.conf
# files inside (not recursively) in alphabetical order
# (not necessarily same as numerical/natural order)

View file

@ -49,7 +49,7 @@ with image size after installation and when gzipped
* [`min`](https://hub.docker.com/r/copyparty/min) (57 MiB, 20 gz) is just copyparty itself
* [`im`](https://hub.docker.com/r/copyparty/im) (70 MiB, 25 gz) can thumbnail images with pillow, parse media files with mutagen
* [`ac` (163 MiB, 56 gz)](https://hub.docker.com/r/copyparty/ac) is `im` plus ffmpeg for video/audio thumbs + audio transcoding + better tags
* [`iv`](https://hub.docker.com/r/copyparty/iv) (211 MiB, 73 gz) is `ac` plus vips for more thumbnail formats
* [`iv`](https://hub.docker.com/r/copyparty/iv) (211 MiB, 73 gz) is `ac` plus vips for faster heif / avic / jxl thumbnails
* [`dj`](https://hub.docker.com/r/copyparty/dj) (309 MiB, 104 gz) is `iv` plus beatroot/keyfinder to detect musical keys and bpm
[`ac` is recommended](https://hub.docker.com/r/copyparty/ac) since the additional features available in `iv` and `dj` are rarely useful

View file

@ -15,7 +15,7 @@ command -v gsort >/dev/null && {
cd ../dist
kwds='-bind -accounts -auth -auth-ord -flags -thumb-ex -handlers -hooks -idp -urlform -exp -rlo -ls -dbd -chmod -pwhash -zm'
kwds='-bind -accounts -auth -auth-ord -flags -handlers -hooks -idp -urlform -exp -ls -dbd -chmod -pwhash -zm'
export PRTY_FULL_HELP=1
export PRTY_NO_PARAMIKO=1

View file

@ -74,7 +74,7 @@ def cnv(src):
continue
if '">uuid:' in ln:
ln = re.sub(r">uuid:[0-9a-f-]{36}<", ">autogenerated<", ln)
if "-salt " in ln and ("-salt SALT" in ln or "-salt TXT" in ln):
if "-salt SALT" in ln:
in_salt = 3
if in_salt:
in_salt -= 1

View file

@ -10,8 +10,8 @@ command -v gsort >/dev/null && {
( ( HLPTXT=CAT && cat || xsel -ob ) | sed -r '
s`/home/ed/`~/`;
s/uuid:[0-9a-f-]{36}/autogenerated/;
s/(-salt (SALT|TXT).*default: )[0-9a-zA-Z/+]{24}\)/\124-character-autogenerated)/;
s/(-salt (SALT|TXT).*default: )[0-9a-zA-Z/+]{40}\)/\140-character-autogenerated)/;
s/(-salt SALT.*default: )[0-9a-zA-Z/+]{24}\)/\124-character-autogenerated)/;
s/(-salt SALT.*default: )[0-9a-zA-Z/+]{40}\)/\140-character-autogenerated)/;
s/(--name TXT.*default: )[^)]+/\1hostname/;
s/(--hash-mt CORES.*default: )[0-9]+/\1numCores if 5 or less/;
s/(--mtag-mt|th-mt)( CORES.*default: )[0-9]+/\1\2numCores/;

View file

@ -16,7 +16,7 @@ uname -s | grep WOW64 && m=64 || m=32
uname -s | grep NT-10 && w10=1 || w7=1
[ $w7 ] && [ -e up2k.sh ] && [ ! "$1" ] && ./up2k.sh
[ $w7 ] && pyv=37 || pyv=314
[ $w7 ] && pyv=37 || pyv=313
[ $w7 ] && sfx=en || sfx=sfx
esuf=
[ $w7 ] && [ $m = 32 ] && esuf=32
@ -58,7 +58,6 @@ rm -rf mods/magic/
sed -ri '/(bccache|PackageLoader)/d' mods/jinja2/__init__.py
af '/^class/{s=0}/^class PackageLoader/{s=1}!s' mods/jinja2/loaders.py
sed -ri 's/from url.*Request, urlopen.*/pass/' mods/copyparty/svchub.py
sed -ri 's/from urllib.* import urlopen.*/pass/' mods/copyparty/httpcli.py
sed -ri 's/(.*"--vc-.*, help=).*/\1argparse.SUPPRESS)/' mods/copyparty/__main__.py
}
[ $w10 ] && {
@ -100,14 +99,9 @@ excl=(
)
[ $w10 ] && excl+=(
_pyrepl
bz2 # 51k
compression.zstd # 195k
concurrent.interpreters # 12k
distutils
setuptools
ftplib # 23k
PIL._avif
PIL._imagingft # 861k (freetype)
PIL.ImageQt
PIL.ImageShow
PIL.ImageTk

View file

@ -1,7 +1,7 @@
76e42806c52319081201d9fd9101827d36d9dbdd34e04f8a48736f2e3b37d68e79e8f52e43260d3eaaa46fe149d95b54ef1355af24361d0bf190dced93cd77bb altgraph-0.17.5-py2.py3-none-any.whl
f117016b1e6a7d7e745db30d3e67f1acf7957c443a0dd301b6c5e10b8368f2aa4db6be9782d2d3f84beadd139bfeef4982e40f21ca5d9065cb794eeb0e473e82 altgraph-0.17.4-py2.py3-none-any.whl
17ce52ba50692a9d964f57a23ac163fb74c77fdeb2ca988a6d439ae1fe91955ff43730c073af97a7b3223093ffea3479a996b9b50ee7fba0869247a56f74baa6 pefile-2023.2.7-py3-none-any.whl
b297ff66ec50cf5a1abcf07d6ac949644c5150ba094ffac974c5d27c81574c3e97ed814a47547f4b03a4c83ea0fb8f026433fca06a3f08e32742dc5c024f3d07 pywin32_ctypes-0.2.3-py3-none-any.whl
76724b04d6fdfadb46f27b6fd5753429d597642d1035ff7bf52fe1b6f5bc18f129567b738a40436adf03afd3a38708dc268de3c1b5d4ef9a89968335805ead72 upx-5.2.0-win32.zip
fd88d00c7231f9fcbe0fb9037217b6ae5dcb5b06a98be56243400542671a32f003d594b2e1852f77d316e9327ad4c643321e1f9f3ea20cc8500ac5bba8ae170e upx-5.0.2-win32.zip
# win7
d9d30ff960365307833425e340bf0a642fd54e35155f18ff1d9746bc1ecf8ecb864aad24948735e8fd994503b659dc8968e6ba337eeb4568217ebdf97dd292dd Git-2.46.2-32-bit.exe
3253e86471e6f9fa85bfdb7684cd2f964ed6e35c6a4db87f81cca157c049bef43e66dfcae1e037b2fb904567b1e028aaeefe8983ba3255105df787406d2aa71e en_windows_7_professional_with_sp1_x86_dvd_u_677056.iso
@ -21,15 +21,15 @@ ea73aa54cc6d5db20dfb127e54562dabf890e4cd6171a91b10a51af2bcfc76e1d64cbdce4546df2d
479a63e14586ab2f2228208116fc149ed8ee7b1e4ff360754f5bda4bf765c61af2e04b5ef123976623d04df4976b7886e0445647269da81436bd0a7b5671d361 windows6.1-kb2533623-x86.msu
ac96786e5d35882e0c5b724794329c9125c2b86ae7847f17acfc49f0d294312c6afc1c3f248655de3f0ccb4ca426d7957d02ba702f4a15e9fcd7e2c314e72c19 zipp-3.15.0-py3-none-any.whl
# win10
ef723f34ce8c0c1f8ef9eb7746f77e4ab6129aa7c7dcf501bee56f522ff2c57ff013f9a69db141ad04c9d1b90b4f1bbd22b6028218e3ae68eb4efec861d63f65 Git-2.55.0.3-64-bit.exe
f4b4e330995ebe96c0bd06e16e5b26062ece9473f06d369775aa68eab261dedcf32dfdd159acaa227502bbf9fa2cd8bbe57cddb89fc6f2196fef7a9ed5a80ae9 Git-2.51.0-64-bit.exe
0a2cd4cadf0395f0374974cd2bc2407e5cc65c111275acdffb6ecc5a2026eee9e1bb3da528b35c7f0ff4b64563a74857d5c2149051e281cc09ebd0d1968be9aa en-us_windows_10_enterprise_ltsc_2021_x64_dvd_d289cf96.iso
4f13070cc9241fa342deab4ebfac360565030580ff77b6e5f1951a64627621e5da4abfd30e1e46ca8bae2bb7dd4ff98141aff424142c9629a5876a61283962e5 virtio-win-0.1.285.iso
9a7f40edc6f9209a2acd23793f3cbd6213c94f36064048cb8bf6eb04f1bdb2c2fe991cb09f77fe8b13e5cd85c618ef23573e79813b2fef899ab2f290cd129779 jinja2-3.1.6-py3-none-any.whl
52adad30ac608695a6345468144048e5134696313ab775ef72f41ca082f8fa19953e2b9924fb5970f829bf9188183fe616b3942c4705eeddfe4ce223a5940186 markupsafe-3.0.3-cp314-cp314-win_amd64.whl
c7668214758cb79a6ba0f7dbffb4b1b6f7cbf98d8ab6cbe77ab42c891916de9339bf17e3de3c3f2d20f6065b49f97c6d521419cc0a2b6c6b8940cf4317172d71 mutagen-1.48.1-py3-none-any.whl
e9492eedaac7cfddde6d84389ef7df86c68275e8f29d838fc431d9bf379401bf6d82ec70ac02461995982613c80592cd8de74c16a3654d7d8fdc06f96057d6e7 packaging-26.3-py3-none-any.whl
8ea9946d8484a6de833a97d3cd545c2b67475207944856281cffb2c513ab14bf50e2b9d31d7cca8f50a6ca8b6711fb8e45cbadabd566cab28f7d58ee058dc650 pillow-12.3.0-cp314-cp314-win_amd64.whl
6f7ed5dc47ade9ad56f6f0499bc5d8eb485bd454d47189133297fa5aa633cf70dad18656cd673318c664f96097c39003d94d9c9703c8337d62840b4564695a04 pyinstaller-6.21.0-py3-none-win_amd64.whl
0a7c1ae7063a2ee46a1c5c057fe73838d7dc5c59800e56037b3fb9dbd1c363915375f989160e44f21436e1071713682bb14473aa8112128beb670f5b110dc8e2 pyinstaller_hooks_contrib-2026.6-py3-none-any.whl
a4adbefaa216571ade7fda561ed3404c762ff03029696ce1745fd4e23b066cb49b1fef99792ce3a594a04f9260a2f05a9e2f851bcc83ed4404f00ce05709c56a python-3.14.7-amd64.exe
bfda5996cdf908edf971df38421ed154e2f2c58f38731ad3be22a88e78c933c64efd35fbc6daf82845dc0b59b5211e925038b430749b2c851887096578c68219 setuptools-83.0.0-py3-none-any.whl
00731cfdd9d5c12efef04a7161c90c1e5ed1dc4677aa88a1d4054aff836f3430df4da5262ed4289c21637358a9e10e5df16f76743cbf5a29bb3a44b146c19cf3 MarkupSafe-3.0.2-cp313-cp313-win_amd64.whl
8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e mutagen-1.47.0-py3-none-any.whl
a726fb46cce24f781fc8b55a3e6dea0a884ebc3b2b400ea74aa02333699f4955a5dc1e2ec5927ac72f35a624401f3f3b442882ba1cc4cadaf9c88558b5b8bdae packaging-25.0-py3-none-any.whl
efcb9da8e6f50cb5ea8e1d41e631e89b6f852b68a2568619a6ec151267e7fac94aba1e36317341c4fc5df822215b7ab9dd6b33cfb738e1b8423d4f0f6cff9525 pillow-12.3.0-cp313-cp313-win_amd64.whl
b9b98714dfca6fa80b0b3f222965724d63be9c54d19435d1fe768e07016913d6db8d6e043fcb185b55a9bd6fe370a80cf961814fc096046a5f4640d99ed575ef pyinstaller-6.15.0-py3-none-win_amd64.whl
cad0f7cf39de691813b1d4abc7d33f8bda99a87d9c5886039b814752e8690364150da26fb61b3e28d5698ff57a90e6dcd619ed2b64b04f72b5aadb75e201bdb0 pyinstaller_hooks_contrib-2025.8-py3-none-any.whl
d0d23fd4ae8900764df664803844155994b253ed449b9ed54ea020fbfa7aaea8800ca1928e4a7f26b9278613f8bce27ef9a424cfcc4ca15551c7e4e1b78f3991 python-3.13.14-amd64.exe
2a0420f7faaa33d2132b82895a8282688030e939db0225ad8abb95a47bdb87b45318f10985fc3cee271a9121441c1526caa363d7f2e4a4b18b1a674068766e87 setuptools-80.9.0-py3-none-any.whl

Binary file not shown.

Before

Width:  |  Height:  |  Size: 555 B

After

Width:  |  Height:  |  Size: 554 B

View file

@ -29,21 +29,21 @@ uname -s | grep NT-10 && w10=1 || {
w7=1; uname -s | grep WOW64 && w7x64=1 || w7x32=1
}
fns=(
altgraph-0.17.5-py2.py3-none-any.whl
altgraph-0.17.4-py2.py3-none-any.whl
pefile-2023.2.7-py3-none-any.whl
pywin32_ctypes-0.2.3-py3-none-any.whl
upx-5.2.0-win32.zip
upx-5.0.2-win32.zip
)
[ $w10 ] && fns+=(
jinja2-3.1.6-py3-none-any.whl
markupsafe-3.0.3-cp314-cp314-win_amd64.whl
mutagen-1.48.1-py3-none-any.whl
packaging-26.3-py3-none-any.whl
pillow-12.3.0-cp314-cp314-win_amd64.whl
pyinstaller-6.21.0-py3-none-win_amd64.whl
pyinstaller_hooks_contrib-2026.6-py3-none-any.whl
python-3.14.7-amd64.exe
setuptools-83.0.0-py3-none-any.whl
MarkupSafe-3.0.2-cp313-cp313-win_amd64.whl
mutagen-1.47.0-py3-none-any.whl
packaging-25.0-py3-none-any.whl
pillow-12.3.0-cp313-cp313-win_amd64.whl
pyinstaller-6.15.0-py3-none-win_amd64.whl
pyinstaller_hooks_contrib-2025.8-py3-none-any.whl
python-3.13.14-amd64.exe
setuptools-80.9.0-py3-none-any.whl
)
[ $w7 ] && fns+=(
future-1.0.0-py3-none-any.whl
@ -83,7 +83,7 @@ close and reopen git-bash so python is in PATH
===[ copy-paste into git-bash ]================================
uname -s | grep NT-10 && w10=1 || w7=1
[ $w7 ] && pyv=37 || pyv=314
[ $w7 ] && pyv=37 || pyv=313
appd=$(cygpath.exe "$APPDATA")
cd ~/Downloads &&
yes | unzip upx-*-win32.zip &&
@ -91,7 +91,7 @@ mv upx-*/upx.exe . &&
python -m ensurepip &&
{ [ $w10 ] || python -m pip install --user -U pip-*.whl; } &&
python -m pip install --user -U packaging-*.whl &&
{ [ $w7 ] || python -m pip install --user -U {setuptools,mutagen,pillow,jinja2,markupsafe}-*.whl; } &&
{ [ $w7 ] || python -m pip install --user -U {setuptools,mutagen,pillow,jinja2,MarkupSafe}-*.whl; } &&
{ [ $w10 ] || python -m pip install --user -U future-*.whl importlib_metadata-*.whl typing_extensions-*.whl zipp-*.whl; } &&
python -m pip install --user -U pyinstaller-*.whl pefile-*.whl pywin32_ctypes-*.whl pyinstaller_hooks_contrib-*.whl altgraph-*.whl &&
sed -ri 's/--lzma/--best/' $appd/Python/Python$pyv/site-packages/pyinstaller/building/utils.py &&

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 1.4 KiB

View file

@ -141,4 +141,3 @@ copyparty/web/tl/vie.js,
copyparty/web/ui.css,
copyparty/web/up2k.js,
copyparty/web/util.js,
copyparty/web/wopi.html,

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: latin-1
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import re, os, sys, time, shutil, signal, tarfile, hashlib, platform, tempfile, traceback
import subprocess as sp

View file

@ -1,5 +1,5 @@
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import sys

View file

@ -71,7 +71,6 @@ Ls.hmn = {
["ctrl-V", "paste (move/copy) here"],
["Y", "download selected"],
["F2", "rename selected"],
["F4", "update/refresh"],
"file-list-sel",
["space", "toggle file selection"],

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import io
import os

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import shutil

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import json
import os

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import io
import json

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import re
import unittest

View file

@ -1,6 +1,6 @@
#!/usr/bin/env python3
# coding: utf-8
from __future__ import division, print_function, unicode_literals
from __future__ import print_function, unicode_literals
import os
import shutil

Some files were not shown because too many files have changed in this diff Show more