.github/ISSUE_TEMPLATE/bug_report.md

@@ -8,42 +8,33 @@ assignees: '9001'
 ---
 
 NOTE:
-**please use english, or include an english translation.** aside from that,  
 all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md
 
 
-### Describe the bug
+**Describe the bug**
 a description of what the bug is
 
-### To Reproduce
+**To Reproduce**
 List of steps to reproduce the issue, or, if it's hard to reproduce, then at least a detailed explanation of what you did to run into it
 
-### Expected behavior
+**Expected behavior**
 a description of what you expected to happen
 
-### Screenshots
+**Screenshots**
 if applicable, add screenshots to help explain your problem, such as the kickass crashpage :^)
 
-### Server details (if you are using docker/podman)
-remove the ones that are not relevant:
-* **server OS / version:** 
-* **how you're running copyparty:** (docker/podman/something-else)
-* **docker image:** (variant, version, and arch if you know)
-* **copyparty arguments and/or config-file:** 
+**Server details**
+if the issue is possibly on the server-side, then mention some of the following:
+* server OS / version: 
+* python version: 
+* copyparty arguments: 
+* filesystem (`lsblk -f` on linux): 
 
-### Server details (if you're NOT using docker/podman)
-remove the ones that are not relevant:
-* **server OS / version:** 
-* **what copyparty did you grab:** (sfx/exe/pip/arch/...)
-* **how you're running it:** (in a terminal, as a systemd-service, ...)
-* run copyparty with `--version` and grab the last 3 lines (they start with `copyparty`, `CPython`, `sqlite`) and paste them below this line:
-* **copyparty arguments and/or config-file:** 
-
-### Client details
+**Client details**
 if the issue is possibly on the client-side, then mention some of the following:
 * the device type and model: 
 * OS version: 
 * browser version: 
 
-### Additional context
+**Additional context**
 any other context about the problem here

.github/ISSUE_TEMPLATE/feature_request.md

@@ -7,8 +7,6 @@ assignees: '9001'
 
 ---
 
-NOTE:
-**please use english, or include an english translation.** aside from that,  
 all of the below are optional, consider them as inspiration, delete and rewrite at will
 
 **is your feature request related to a problem? Please describe.**

.gitignore

@@ -43,6 +43,3 @@ scripts/docker/*.err
 
 # nix build output link
 result
-
-# IDEA config
-.idea/

CONTRIBUTING.md

@@ -1,21 +1,8 @@
-* **found a bug?** [create an issue!](https://github.com/9001/copyparty/issues) or let me know in the [discord](https://discord.gg/25J8CdTT6G) :>
-* **fixed a bug?** create a PR or post a patch! big thx in advance :>
-* **have a cool idea?** let's discuss it! anywhere's fine, you choose.
+* do something cool
 
-but please:
-
-
-
-# do not use AI / LMM when writing code
-
-copyparty is 100% organic, free-range, human-written software!
-
-> ⚠ you are now entering a no-copilot zone
-
-the *only* place where LMM/AI *may* be accepted is for [localization](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#translations) if you are fluent and have confirmed that the translation is accurate.
-
-sorry for the harsh tone, but this is important to me 🙏
+really tho, send a PR or an issue or whatever, all appreciated, anything goes, just behave aight 👍👍
 
+but to be more specific,
 
 
 # contribution ideas
@@ -41,8 +28,6 @@ aside from documentation and ideas, some other things that would be cool to have
 
 * **translations** -- the copyparty web-UI has translations for english and norwegian at the top of [browser.js](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.js); if you'd like to add a translation for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>
 
-  * but please note that support for [RTL (Right-to-Left) languages](https://en.wikipedia.org/wiki/Right-to-left_script) is currently not planned, since the javascript is a bit too jank for that
-
 * **UI ideas** -- at some point I was thinking of rewriting the UI in react/preact/something-not-vanilla-javascript, but I'll admit the comfiness of not having any build stage combined with raw performance has kinda convinced me otherwise :p but I'd be very open to ideas on how the UI could be improved, or be more intuitive.
 
 * **docker improvements** -- I don't really know what I'm doing when it comes to containers, so I'm sure there's a *huge* room for improvement here, mainly regarding how you're supposed to use the container with kubernetes / docker-compose / any of the other popular ways to do things. At some point I swear I'll start learning about docker so I can pick up clach04's [docker-compose draft](https://github.com/9001/copyparty/issues/38) and learn how that stuff ticks, unless someone beats me to it!

bin/README.md

@@ -78,6 +78,3 @@ cd /mnt/nas/music/.hist
 # [`prisonparty.sh`](prisonparty.sh)
 * run copyparty in a chroot, preventing any accidental file access
 * creates bindmounts for /bin, /lib, and so on, see `sysdirs=`
-
-# [`bubbleparty.sh`](bubbleparty.sh)
-* run copyparty in an isolated process, preventing any accidental file access and more

bin/bubbleparty.sh

@@ -1,19 +0,0 @@
-#!/bin/sh
-# usage: ./bubbleparty.sh ./copyparty-sfx.py ....
-bwrap \
-  --unshare-all \
-  --ro-bind /usr /usr \
-  --ro-bind /bin /bin \
-  --ro-bind /lib /lib \
-  --ro-bind /etc/resolv.conf /etc/resolv.conf \
-  --dev-bind /dev /dev \
-  --dir /tmp \
-  --dir /var \
-  --bind $(pwd) $(pwd) \
-  --share-net \
-  --die-with-parent \
-  --file 11 /etc/passwd \
-  --file 12 /etc/group \
-  "$@" \
-  11< <(getent passwd $(id -u) 65534) \
-  12< <(getent group $(id -g) 65534) 

bin/handlers/README.md

@@ -20,8 +20,6 @@ each plugin must define a `main()` which takes 3 arguments;
 
 ## on404
 
-* [redirect.py](redirect.py) sends an HTTP 301 or 302, redirecting the client to another page/file
-* [randpic.py](randpic.py) redirects `/foo/bar/randpic.jpg` to a random pic in `/foo/bar/`
 * [sorry.py](answer.py) replies with a custom message instead of the usual 404
 * [nooo.py](nooo.py) replies with an endless noooooooooooooo
 * [never404.py](never404.py) 100% guarantee that 404 will never be a thing again as it automatically creates dummy files whenever necessary

bin/handlers/randpic.py

@@ -1,35 +0,0 @@
-import os
-import random
-from urllib.parse import quote
-
-
-# assuming /foo/bar/ is a valid URL but /foo/bar/randpic.png does not exist,
-# hijack the 404 with a redirect to a random pic in that folder
-#
-# thx to lia & kipu for the idea
-
-
-def main(cli, vn, rem):
-    req_fn = rem.split("/")[-1]
-    if not cli.can_read or not req_fn.startswith("randpic"):
-        return
-
-    req_abspath = vn.canonical(rem)
-    req_ap_dir = os.path.dirname(req_abspath)
-    files_in_dir = os.listdir(req_ap_dir)
-
-    if "." in req_fn:
-        file_ext = "." + req_fn.split(".")[-1]
-        files_in_dir = [x for x in files_in_dir if x.lower().endswith(file_ext)]
-
-    if not files_in_dir:
-        return
-
-    selected_file = random.choice(files_in_dir)
-
-    req_url = "/".join([vn.vpath, rem]).strip("/")
-    req_dir = req_url.rsplit("/", 1)[0]
-    new_url = "/".join([req_dir, quote(selected_file)]).strip("/")
-
-    cli.reply(b"redirecting...", 302, headers={"Location": "/" + new_url})
-    return "true"

bin/handlers/redirect.py

@@ -1,52 +0,0 @@
-# if someone hits a 404, redirect them to another location
-
-
-def send_http_302_temporary_redirect(cli, new_path):
-    """
-    replies with an HTTP 302, which is a temporary redirect;
-    "new_path" can be any of the following:
-      - "http://a.com/" would redirect to another website,
-      - "/foo/bar" would redirect to /foo/bar on the same server;
-          note the leading '/' in the location which is important
-    """
-    cli.reply(b"redirecting...", 302, headers={"Location": new_path})
-
-
-def send_http_301_permanent_redirect(cli, new_path):
-    """
-    replies with an HTTP 301, which is a permanent redirect;
-    otherwise identical to send_http_302_temporary_redirect
-    """
-    cli.reply(b"redirecting...", 301, headers={"Location": new_path})
-
-
-def send_errorpage_with_redirect_link(cli, new_path):
-    """
-    replies with a website explaining that the page has moved;
-    "new_path" must be an absolute location on the same server
-    but without a leading '/', so for example "foo/bar"
-    would redirect to "/foo/bar"
-    """
-    cli.redirect(new_path, click=False, msg="this page has moved")
-
-
-def main(cli, vn, rem):
-    """
-    this is the function that gets called by copyparty;
-    note that vn.vpath and cli.vpath does not have a leading '/'
-    so we're adding the slash in the debug messages below
-    """
-    print(f"this client just hit a 404: {cli.ip}")
-    print(f"they were accessing this volume: /{vn.vpath}")
-    print(f"and the original request-path (straight from the URL) was /{cli.vpath}")
-    print(f"...which resolves to the following filesystem path: {vn.canonical(rem)}")
-
-    new_path = "/foo/bar/"
-    print(f"will now redirect the client to {new_path}")
-
-    # uncomment one of these:
-    send_http_302_temporary_redirect(cli, new_path)
-    #send_http_301_permanent_redirect(cli, new_path)
-    #send_errorpage_with_redirect_link(cli, new_path)
-
-    return "true"

bin/hooks/README.md

@@ -14,8 +14,6 @@ run copyparty with `--help-hooks` for usage details / hook type explanations (xm
 * [discord-announce.py](discord-announce.py) announces new uploads on discord using webhooks ([example](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png))
 * [reject-mimetype.py](reject-mimetype.py) rejects uploads unless the mimetype is acceptable
 * [into-the-cache-it-goes.py](into-the-cache-it-goes.py) avoids bugs in caching proxies by immediately downloading each file that is uploaded
-* [podcast-normalizer.py](podcast-normalizer.py) creates a second file with dynamic-range-compression whenever an audio file is uploaded
-  * good example of the `idx` [hook effect](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#hook-effects) to tell copyparty about additional files to scan/index
 
 
 # upload batches
@@ -27,11 +25,9 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
 # before upload
 * [reject-extension.py](reject-extension.py) rejects uploads if they match a list of file extensions
 * [reloc-by-ext.py](reloc-by-ext.py) redirects an upload to another destination based on the file extension
-  * good example of the `reloc` [hook effect](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#hook-effects)
 
 
 # on message
 * [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
 * [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
-* [usb-eject.py](usb-eject.py) adds web-UI buttons to safe-remove usb flashdrives shared through copyparty
 * [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder

bin/hooks/podcast-normalizer.py

@@ -1,121 +0,0 @@
-#!/usr/bin/env python3
-
-import json
-import os
-import sys
-import subprocess as sp
-
-
-_ = r"""
-sends all uploaded audio files through an aggressive
-dynamic-range-compressor to even out the volume levels
-
-dependencies:
-    ffmpeg
-
-being an xau hook, this gets eXecuted After Upload completion
-    but before copyparty has started hashing/indexing the file, so
-    we'll create a second normalized copy in a subfolder and tell
-    copyparty to hash/index that additional file as well
-
-example usage as global config:
-    -e2d -e2t --xau j,c1,bin/hooks/podcast-normalizer.py
-
-parameters explained,
-    e2d/e2t = enable database and metadata indexing
-    xau = execute after upload
-    j   = this hook needs upload information as json (not just the filename)
-    c1  = this hook returns json on stdout, so tell copyparty to read that
-
-example usage as a volflag (per-volume config):
-    -v srv/inc/pods:inc/pods:r:rw,ed:c,xau=j,c1,bin/hooks/podcast-normalizer.py
-                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-    (share fs-path srv/inc/pods at URL /inc/pods,
-     readable by all, read-write for user ed,
-     running this xau (exec-after-upload) plugin for all uploaded files)
-
-example usage as a volflag in a copyparty config file:
-    [/inc/pods]
-      srv/inc/pods
-      accs:
-        r: *
-        rw: ed
-      flags:
-        e2d  # enables file indexing
-        e2t  # metadata tags too
-        xau: j,c1,bin/hooks/podcast-normalizer.py
-
-"""
-
-########################################################################
-### CONFIG
-
-# filetypes to process; ignores everything else
-EXTS = "mp3 flac ogg oga opus m4a aac wav wma"
-
-# the name of the subdir to put the normalized files in
-SUBDIR = "normalized"
-
-########################################################################
-
-
-# try to enable support for crazy filenames
-try:
-    from copyparty.util import fsenc
-except:
-
-    def fsenc(p):
-        return p.encode("utf-8")
-
-
-def main():
-    # read info from copyparty
-    inf = json.loads(sys.argv[1])
-    vpath = inf["vp"]
-    abspath = inf["ap"]
-
-    # check if the file-extension is on the to-be-processed list
-    ext = abspath.lower().split(".")[-1]
-    if ext not in EXTS.split():
-        return
-
-    # jump into the folder where the file was uploaded
-    # and create the subfolder to place the normalized copy inside
-    dirpath, filename = os.path.split(abspath)
-    os.chdir(fsenc(dirpath))
-    os.makedirs(SUBDIR, exist_ok=True)
-
-    # the input and output filenames to give ffmpeg
-    fname_in = fsenc(f"./{filename}")
-    fname_out = fsenc(f"{SUBDIR}/{filename}.opus")
-
-    # fmt: off
-    # create and run the ffmpeg command
-    cmd = [
-        b"ffmpeg",
-        b"-nostdin",
-        b"-hide_banner",
-        b"-i", fname_in,
-        b"-af", b"dynaudnorm=f=100:g=9",  # the normalizer config
-        b"-c:a", b"libopus",
-        b"-b:a", b"128k",
-        fname_out,
-    ]
-    # fmt: on
-    sp.check_output(cmd)
-
-    # and finally, tell copyparty about the new file
-    # so it appears in the database and rss-feed:
-    vpath = f"{SUBDIR}/{filename}.opus"
-    print(json.dumps({"idx": {"vp": [vpath]}}))
-
-    # (it's fine to give it a relative path like that; it gets
-    #  resolved relative to the folder the file was uploaded into)
-
-
-if __name__ == "__main__":
-    try:
-        main()
-    except Exception as ex:
-        print("podcast-normalizer failed; %r" % (ex,))

bin/hooks/reloc-by-ext.py

@@ -71,9 +71,6 @@ def main():
     ## selecting it inside the print at the end:
     ##
 
-    # move all uploads to one specific folder
-    into_junk = {"vp": "/junk"}
-
     # create a subfolder named after the filetype and move it into there
     into_subfolder = {"vp": ext}
 
@@ -95,8 +92,8 @@ def main():
         by_category = {}  # no action
 
     # now choose the default effect to apply; can be any of these:
-    # into_junk  into_subfolder  into_toplevel  into_sibling  by_category
-    effect = into_sibling
+    # into_subfolder  into_toplevel  into_sibling  by_category
+    effect = {"vp": "/junk"}
 
     ##
     ## but we can keep going, adding more speicifc rules

bin/hooks/usb-eject.js

@@ -1,62 +0,0 @@
-// see usb-eject.py for usage
-
-function usbclick() {
-    var o = QS('#treeul a[dst="/usb/"]') || QS('#treepar a[dst="/usb/"]');
-    if (o)
-        o.click();
-}
-
-function eject_cb() {
-    var t = ('' + this.responseText).trim();
-    if (t.indexOf('can be safely unplugged') < 0 && t.indexOf('Device can be removed') < 0)
-        return toast.err(30, 'usb eject failed:\n\n' + t);
-
-    toast.ok(5, esc(t.replace(/ - /g, '\n\n')).trim());
-    usbclick(); setTimeout(usbclick, 10);
-};
-
-function add_eject_2(a) {
-    var aw = a.getAttribute('href').split(/\//g);
-    if (aw.length != 4 || aw[3])
-        return;
-
-    var v = aw[2],
-        k = 'umount_' + v;
-
-    for (var b = 0; b < 9; b++) {
-        var o = ebi(k);
-        if (!o)
-            break;
-        o.parentNode.removeChild(o);
-    }
-
-    a.appendChild(mknod('span', k, '⏏'), a);
-    o = ebi(k);
-    o.style.cssText = 'position:absolute; right:1em; margin-top:-.2em; font-size:1.3em';
-    o.onclick = function (e) {
-        ev(e);
-        var xhr = new XHR();
-        xhr.open('POST', get_evpath(), true);
-        xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded;charset=UTF-8');
-        xhr.send('msg=' + uricom_enc(':usb-eject:' + v + ':'));
-        xhr.onload = xhr.onerror = eject_cb;
-        toast.inf(10, "ejecting " + v + "...");
-    };
-};
-
-function add_eject() {
-    var o = QSA('#treeul a[href^="/usb/"]') || QSA('#treepar a[href^="/usb/"]');
-    for (var a = o.length - 1; a > 0; a--)
-        add_eject_2(o[a]);
-};
-
-(function() {
-    var f0 = treectl.rendertree;
-    treectl.rendertree = function (res, ts, top0, dst, rst) {
-        var ret = f0(res, ts, top0, dst, rst);
-        add_eject();
-        return ret;
-    };
-})();
-
-setTimeout(add_eject, 50);

bin/hooks/usb-eject.py

@@ -1,62 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import stat
-import subprocess as sp
-import sys
-from urllib.parse import unquote_to_bytes as unquote
-
-
-"""
-if you've found yourself using copyparty to serve flashdrives on a LAN
-and your only wish is that the web-UI had a button to unmount / safely
-remove those flashdrives, then boy howdy are you in the right place :D
-
-put usb-eject.js in the webroot (or somewhere else http-accessible)
-then run copyparty with these args:
-
-   -v /run/media/egon:/usb:A:c,hist=/tmp/junk
-   --xm=c1,bin/hooks/usb-eject.py
-   --js-browser=/usb-eject.js
-
-which does the following respectively,
-
-  * share all of /run/media/egon as /usb with admin for everyone
-     and put the histpath somewhere it won't cause trouble
-  * run the usb-eject hook with stdout redirect to the web-ui
-  * add the complementary usb-eject.js to the browser
-
-"""
-
-
-MOUNT_BASE = b"/run/media/egon/"
-
-
-def main():
-    try:
-        label = sys.argv[1].split(":usb-eject:")[1].split(":")[0]
-        mp = MOUNT_BASE + unquote(label)
-        # print("ejecting [%s]... " % (mp,), end="")
-        mp = os.path.abspath(os.path.realpath(mp))
-        st = os.lstat(mp)
-        if not stat.S_ISDIR(st.st_mode) or not mp.startswith(MOUNT_BASE):
-            raise Exception("not a regular directory")
-
-        # if you're running copyparty as root (thx for the faith)
-        # you'll need something like this to make dbus talkative
-        cmd = b"sudo -u egon DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus gio mount -e"
-
-        # but if copyparty and the ui-session is running
-        # as the same user (good) then this is plenty
-        cmd = b"gio mount -e"
-
-        cmd = cmd.split(b" ") + [mp]
-        ret = sp.check_output(cmd).decode("utf-8", "replace")
-        print(ret.strip() or (label + " can be safely unplugged"))
-
-    except Exception as ex:
-        print("unmount failed: %r" % (ex,))
-
-
-if __name__ == "__main__":
-    main()

bin/mtag/README.md

@@ -31,9 +31,6 @@ plugins in this section should only be used with appropriate precautions:
 * [very-bad-idea.py](./very-bad-idea.py) combined with [meadup.js](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/meadup.js) converts copyparty into a janky yet extremely flexible chromecast clone
   * also adds a virtual keyboard by @steinuil to the basic-upload tab for comfy couch crowd control
   * anything uploaded through the [android app](https://github.com/9001/party-up) (files or links) are executed on the server, meaning anyone can infect your PC with malware... so protect this with a password and keep it on a LAN!
-  * [kamelåså](https://github.com/steinuil/kameloso) is a much better (and MUCH safer) alternative to this plugin
-    * powered by [chicken-curry-banana-pineapple-peanut pizza](https://a.ocv.me/pub/g/i/2025/01/298437ce-8351-4c8c-861c-fa131d217999.jpg?cache) so you know it's good
-    * and, unlike this plugin, kamelåså even has windows support (nice)
 
 
 # dependencies

bin/mtag/cksum.py

@@ -2,15 +2,11 @@
 
 import sys
 import json
+import zlib
 import struct
 import base64
 import hashlib
 
-try:
-    from zlib_ng import zlib_ng as zlib
-except:
-    import zlib
-
 try:
     from copyparty.util import fsenc
 except:

bin/mtag/install-deps.sh

@@ -22,8 +22,6 @@ set -e
 #   modifies the keyfinder python lib to load the .so in ~/pe
 
 
-export FORCE_COLOR=1
-
 linux=1
 
 win=
@@ -188,15 +186,12 @@ install_keyfinder() {
 		echo "so not found at $sop"
 		exit 1
 	}
-
-	x=${-//[^x]/}; set -x; cat /etc/alpine-release
+	
 	# rm -rf /Users/ed/Library/Python/3.9/lib/python/site-packages/*keyfinder*
 	CFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
-	CXXFLAGS="-I$h/pe/keyfinder/include -I/opt/local/include -I/usr/include/ffmpeg" \
 	LDFLAGS="-L$h/pe/keyfinder/lib -L$h/pe/keyfinder/lib64 -L/opt/local/lib" \
-	PKG_CONFIG_PATH="/c/msys64/mingw64/lib/pkgconfig:$h/pe/keyfinder/lib/pkgconfig" \
+	PKG_CONFIG_PATH=/c/msys64/mingw64/lib/pkgconfig \
 	$pybin -m pip install --user keyfinder
-	[ "$x" ] || set +x
 
 	pypath="$($pybin -c 'import keyfinder; print(keyfinder.__file__)')"
 	for pyso in "${pypath%/*}"/*.so; do

bin/mtag/very-bad-idea.py

@@ -6,11 +6,6 @@ WARNING -- DANGEROUS PLUGIN --
   running this plugin, they can execute malware on your machine
   so please keep this on a LAN and protect it with a password
 
-here is a MUCH BETTER ALTERNATIVE (which also works on Windows):
-  https://github.com/steinuil/kameloso
-
-----------------------------------------------------------------------
-
 use copyparty as a chromecast replacement:
   * post a URL and it will open in the default browser
   * upload a file and it will open in the default application

bin/u2c.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
 from __future__ import print_function, unicode_literals
 
-S_VERSION = "2.11"
-S_BUILD_DT = "2025-05-18"
+S_VERSION = "2.7"
+S_BUILD_DT = "2024-12-06"
 
 """
 u2c.py: upload to copyparty
@@ -52,7 +52,6 @@ if PY2:
 
     sys.dont_write_bytecode = True
     bytes = str
-    files_decoder = lambda s: unicode(s, "utf8")
 else:
     from urllib.parse import quote_from_bytes as quote
     from urllib.parse import unquote_to_bytes as unquote
@@ -62,7 +61,6 @@ else:
     from queue import Queue
 
     unicode = str
-    files_decoder = unicode
 
 
 WTF8 = "replace" if PY2 else "surrogateescape"
@@ -236,10 +234,6 @@ CLEN = "Content-Length"
 
 web = None  # type: HCli
 
-links = []  # type: list[str]
-linkmtx = threading.Lock()
-linkfile = None
-
 
 class File(object):
     """an up2k upload task; represents a single file"""
@@ -767,29 +761,6 @@ def get_hashlist(file, pcb, mth):
             file.kchunks[k] = [v1, v2]
 
 
-def printlink(ar, purl, name, fk):
-    if not name:
-        url = purl  # srch
-    else:
-        name = quotep(name.encode("utf-8", WTF8)).decode("utf-8")
-        if fk:
-            url = "%s%s?k=%s" % (purl, name, fk)
-        else:
-            url = "%s%s" % (purl, name)
-
-    url = "%s/%s" % (ar.burl, url.lstrip("/"))
-
-    with linkmtx:
-        if ar.u:
-            links.append(url)
-        if ar.ud:
-            print(url)
-        if linkfile:
-            zs = "%s\n" % (url,)
-            zb = zs.encode("utf-8", "replace")
-            linkfile.write(zb)
-
-
 def handshake(ar, file, search):
     # type: (argparse.Namespace, File, bool) -> tuple[list[str], bool]
     """
@@ -809,9 +780,7 @@ def handshake(ar, file, search):
     else:
         if ar.touch:
             req["umod"] = True
-        if ar.owo:
-            req["replace"] = "mt"
-        elif ar.ow:
+        if ar.ow:
             req["replace"] = True
 
     file.recheck = False
@@ -863,17 +832,12 @@ def handshake(ar, file, search):
         raise Exception(txt)
 
     if search:
-        if ar.uon and r["hits"]:
-            printlink(ar, r["hits"][0]["rp"], "", "")
         return r["hits"], False
 
     file.url = quotep(r["purl"].encode("utf-8", WTF8)).decode("utf-8")
     file.name = r["name"]
     file.wark = r["wark"]
 
-    if ar.uon and not r["hash"]:
-        printlink(ar, file.url, r["name"], r.get("fk"))
-
     return r["hash"], r["sprs"]
 
 
@@ -1139,7 +1103,7 @@ class Ctl(object):
             nleft = self.nfiles - self.up_f
             tail = "\033[K\033[u" if VT100 and not self.ar.ns else "\r"
 
-            t = "%s eta @ %s/s, %s, %d# left" % (self.eta, spd, sleft, nleft)
+            t = "%s eta @ %s/s, %s, %d# left\033[K" % (self.eta, spd, sleft, nleft)
             if not self.hash_b:
                 t = " now hashing..."
             eprint(txt + "\033]0;{0}\033\\\r{0}{1}".format(t, tail))
@@ -1285,13 +1249,13 @@ class Ctl(object):
                         for n, zsii in enumerate(file.cids)
                     ]
                     print("chs: %s\n%s" % (vp, "\n".join(zsl)))
-                zsl = [self.ar.wsalt, str(file.size)] + [x[0] for x in file.cids]
+                zsl = [self.ar.wsalt, str(file.size)] + [x[0] for x in file.kchunks]
                 zb = hashlib.sha512("\n".join(zsl).encode("utf-8")).digest()[:33]
                 wark = ub64enc(zb).decode("utf-8")
                 if self.ar.jw:
                     print("%s  %s" % (wark, vp))
                 else:
-                    zd = datetime.datetime.fromtimestamp(max(0, file.lmod), UTC)
+                    zd = datetime.datetime.fromtimestamp(file.lmod, UTC)
                     dt = "%04d-%02d-%02d %02d:%02d:%02d" % (
                         zd.year,
                         zd.month,
@@ -1508,7 +1472,7 @@ class APF(argparse.ArgumentDefaultsHelpFormatter, argparse.RawDescriptionHelpFor
 
 
 def main():
-    global web, linkfile
+    global web
 
     time.strptime("19970815", "%Y%m%d")  # python#7980
     "".encode("idna")  # python#29288
@@ -1534,7 +1498,7 @@ source file/folder selection uses rsync syntax, meaning that:
 """)
 
     ap.add_argument("url", type=unicode, help="server url, including destination folder")
-    ap.add_argument("files", type=files_decoder, nargs="+", help="files and/or folders to process")
+    ap.add_argument("files", type=unicode, nargs="+", help="files and/or folders to process")
     ap.add_argument("-v", action="store_true", help="verbose")
     ap.add_argument("-a", metavar="PASSWD", help="password or $filepath")
     ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
@@ -1542,15 +1506,9 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
     ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
     ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
-    ap.add_argument("--owo", action="store_true", help="overwrite existing files if server-file is older")
     ap.add_argument("--spd", action="store_true", help="print speeds for each file")
     ap.add_argument("--version", action="store_true", help="show version and exit")
 
-    ap = app.add_argument_group("print links")
-    ap.add_argument("-u", action="store_true", help="print list of download-links after all uploads finished")
-    ap.add_argument("-ud", action="store_true", help="print download-link after each upload finishes")
-    ap.add_argument("-uf", type=unicode, metavar="PATH", help="print list of download-links to file")
-
     ap = app.add_argument_group("compatibility")
     ap.add_argument("--cls", action="store_true", help="clear screen before start")
     ap.add_argument("--rh", type=int, metavar="TRIES", default=0, help="resolve server hostname before upload (good for buggy networks, but TLS certs will break)")
@@ -1636,10 +1594,6 @@ source file/folder selection uses rsync syntax, meaning that:
     ar.x = "|".join(ar.x or [])
 
     setattr(ar, "wlist", ar.url == "-")
-    setattr(ar, "uon", ar.u or ar.ud or ar.uf)
-
-    if ar.uf:
-        linkfile = open(ar.uf, "wb")
 
     for k in "dl dr drd wlist".split():
         errs = []
@@ -1702,12 +1656,6 @@ source file/folder selection uses rsync syntax, meaning that:
         ar.z = True
         ctl = Ctl(ar, ctl.stats)
 
-    if links:
-        print()
-        print("\n".join(links))
-    if linkfile:
-        linkfile.close()
-
     if ctl.errs:
         print("WARNING: %d errors" % (ctl.errs))
 

bin/zmq-recv.py

@@ -1,76 +0,0 @@
-#!/usr/bin/env python3
-
-import sys
-import zmq
-
-"""
-zmq-recv.py: demo zmq receiver
-2025-01-22, v1.0, ed <irc.rizon.net>, MIT-Licensed
-https://github.com/9001/copyparty/blob/hovudstraum/bin/zmq-recv.py
-
-basic zmq-server to receive events from copyparty; try one of
-the below and then "send a message to serverlog" in the web-ui:
-
-1) dumb fire-and-forget to any and all listeners;
-run this script with "sub" and run copyparty with this:
-  --xm zmq:pub:tcp://*:5556
-
-2) one lucky listener gets the message, blocks if no listeners:
-run this script with "pull" and run copyparty with this:
-  --xm t3,zmq:push:tcp://*:5557
-
-3) blocking syn/ack mode, client must ack each message;
-run this script with "rep" and run copyparty with this:
-  --xm t3,zmq:req:tcp://localhost:5555
-
-note: to conditionally block uploads based on message contents,
-use rep_server to answer with "return 1" and run copyparty with
-  --xau t3,c,zmq:req:tcp://localhost:5555
-"""
-
-
-ctx = zmq.Context()
-
-
-def sub_server():
-    # PUB/SUB allows any number of servers/clients, and
-    # messages are fire-and-forget
-    sck = ctx.socket(zmq.SUB)
-    sck.connect("tcp://localhost:5556")
-    sck.setsockopt_string(zmq.SUBSCRIBE, "")
-    while True:
-        print("copyparty says %r" % (sck.recv_string(),))
-
-
-def pull_server():
-    # PUSH/PULL allows any number of servers/clients, and
-    # each message is sent to a exactly one PULL client
-    sck = ctx.socket(zmq.PULL)
-    sck.connect("tcp://localhost:5557")
-    while True:
-        print("copyparty says %r" % (sck.recv_string(),))
-
-
-def rep_server():
-    # REP/REQ is a server/client pair where each message must be
-    # acked by the other before another message can be sent, so
-    # copyparty will do a blocking-wait for the ack
-    sck = ctx.socket(zmq.REP)
-    sck.bind("tcp://*:5555")
-    while True:
-        print("copyparty says %r" % (sck.recv_string(),))
-        reply = b"thx"
-        # reply = b"return 1"  # non-zero to block an upload
-        sck.send(reply)
-
-
-mode = sys.argv[1].lower() if len(sys.argv) > 1 else ""
-
-if mode == "sub":
-    sub_server()
-elif mode == "pull":
-    pull_server()
-elif mode == "rep":
-    rep_server()
-else:
-    print("specify mode as first argument:  SUB | PULL | REP")

contrib/README.md

@@ -12,19 +12,14 @@
 * assumes the webserver and copyparty is running on the same server/IP
 * modify `10.13.1.1` as necessary if you wish to support browsers without javascript
 
-### [`sharex.sxcu`](sharex.sxcu) - Windows screenshot uploader
-* [sharex](https://getsharex.com/) config file to upload screenshots and grab the URL
+### [`sharex.sxcu`](sharex.sxcu)
+* sharex config file to upload screenshots and grab the URL
 * `RequestURL`: full URL to the target folder
 * `pw`: password (remove the `pw` line if anon-write)
 * the `act:bput` thing is optional since copyparty v1.9.29
 * using an older sharex version, maybe sharex v12.1.1 for example? dw fam i got your back 👉😎👉 [`sharex12.sxcu`](sharex12.sxcu)
 
-### [`ishare.iscu`](ishare.iscu) - MacOS screenshot uploader
-* [ishare](https://isharemac.app/) config file to upload screenshots and grab the URL
-* `RequestURL`: full URL to the target folder
-* `pw`: password (remove the `pw` line if anon-write)
-
-### [`flameshot.sh`](flameshot.sh) - Linux screenshot uploader
+### [`flameshot.sh`](flameshot.sh)
 * takes a screenshot with [flameshot](https://flameshot.org/) on Linux, uploads it, and writes the URL to clipboard
 
 ### [`send-to-cpp.contextlet.json`](send-to-cpp.contextlet.json)
@@ -50,9 +45,6 @@
 * give a 3rd argument to install it to your copyparty config
 * systemd service at [`systemd/cfssl.service`](systemd/cfssl.service)
 
-### [`zfs-tune.py`](zfs-tune.py)
-* optimizes databases for optimal performance when stored on a zfs filesystem; also see [openzfs docs](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#database-workloads) and specifically the SQLite subsection
-
 # OS integration
 init-scripts to start copyparty as a service
 * [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
@@ -61,10 +53,5 @@ init-scripts to start copyparty as a service
 * [`openrc/copyparty`](openrc/copyparty)
 
 # Reverse-proxy
-copyparty supports running behind another webserver
-* [`apache/copyparty.conf`](apache/copyparty.conf)
-* [`haproxy/copyparty.conf`](haproxy/copyparty.conf)
-* [`lighttpd/subdomain.conf`](lighttpd/subdomain.conf)
-* [`lighttpd/subpath.conf`](lighttpd/subpath.conf)
-* [`nginx/copyparty.conf`](nginx/copyparty.conf) -- recommended
-* [`traefik/copyparty.yaml`](traefik/copyparty.yaml)
+copyparty has basic support for running behind another webserver
+* [`nginx/copyparty.conf`](nginx/copyparty.conf)

contrib/apache/copyparty.conf

@@ -1,29 +1,14 @@
-# if you would like to use unix-sockets (recommended),
-# you must run copyparty with one of the following:
+# when running copyparty behind a reverse proxy,
+# the following arguments are recommended:
 #
-#   -i unix:777:/dev/shm/party.sock
-#   -i unix:777:/dev/shm/party.sock,127.0.0.1
+#   -i 127.0.0.1    only accept connections from nginx
 #
 # if you are doing location-based proxying (such as `/stuff` below)
 # you must run copyparty with --rp-loc=stuff
 #
 # on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
 
-
 LoadModule proxy_module modules/mod_proxy.so
-
+ProxyPass "/stuff" "http://127.0.0.1:3923/stuff"
+# do not specify ProxyPassReverse
 RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
-# NOTE: do not specify ProxyPassReverse
-
-
-##
-## then, enable one of the below:
-
-# use subdomain proxying to unix-socket (best)
-ProxyPass "/" "unix:///dev/shm/party.sock|http://whatever/"
-
-# use subdomain proxying to 127.0.0.1 (slower)
-#ProxyPass "/" "http://127.0.0.1:3923/"
-
-# use subpath proxying to 127.0.0.1 (slow and maybe buggy)
-#ProxyPass "/stuff" "http://127.0.0.1:3923/stuff"

contrib/haproxy/copyparty.conf

@@ -1,24 +0,0 @@
-# this config is essentially two separate examples;
-#
-#   foo1 connects to copyparty using tcp, and
-#   foo2 uses unix-sockets for 27% higher performance
-#
-# to use foo2 you must run copyparty with one of the following:
-#
-#   -i unix:777:/dev/shm/party.sock
-#   -i unix:777:/dev/shm/party.sock,127.0.0.1
-
-defaults
-  mode http
-  option forwardfor
-  timeout connect 1s
-  timeout client 610s
-  timeout server 610s
-
-listen foo1
-  bind *:8081
-  server srv1 127.0.0.1:3923 maxconn 512
-
-listen foo2
-  bind *:8082
-  server srv1 /dev/shm/party.sock maxconn 512

contrib/ishare.iscu

@@ -1,10 +0,0 @@
-{
-  "Name": "copyparty",
-  "RequestURL": "http://127.0.0.1:3923/screenshots/",
-  "Headers": {
-    "pw": "PUT_YOUR_PASSWORD_HERE_MY_DUDE",
-    "accept": "json"
-  },
-  "FileFormName": "f",
-  "ResponseURL": "{{fileurl}}"
-}

contrib/lighttpd/subdomain.conf

@@ -1,24 +0,0 @@
-# example usage for benchmarking:
-#
-#   taskset -c 1 lighttpd -Df ~/dev/copyparty/contrib/lighttpd/subdomain.conf
-#
-# lighttpd can connect to copyparty using either tcp (127.0.0.1)
-# or a unix-socket, but unix-sockets are 37% faster because
-# lighttpd doesn't reuse tcp connections, so we're doing unix-sockets
-#
-# this means we must run copyparty with one of the following:
-#
-#   -i unix:777:/dev/shm/party.sock
-#   -i unix:777:/dev/shm/party.sock,127.0.0.1
-#
-# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
-
-server.port = 80
-server.document-root = "/var/empty"
-server.upload-dirs = ( "/dev/shm", "/tmp" )
-server.modules = ( "mod_proxy" )
-proxy.forwarded = ( "for" => 1, "proto" => 1 )
-proxy.server = ( "" => ( ( "host" => "/dev/shm/party.sock" ) ) )
-
-# if you really need to use tcp instead of unix-sockets, do this instead:
-#proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "3923" ) ) )

contrib/lighttpd/subpath.conf

@@ -1,31 +0,0 @@
-# example usage for benchmarking:
-#
-#   taskset -c 1 lighttpd -Df ~/dev/copyparty/contrib/lighttpd/subpath.conf
-#
-# lighttpd can connect to copyparty using either tcp (127.0.0.1)
-# or a unix-socket, but unix-sockets are 37% faster because
-# lighttpd doesn't reuse tcp connections, so we're doing unix-sockets
-#
-# this means we must run copyparty with one of the following:
-#
-#   -i unix:777:/dev/shm/party.sock
-#   -i unix:777:/dev/shm/party.sock,127.0.0.1
-#
-# also since this example proxies a subpath instead of the
-# recommended subdomain-proxying, we must also specify this:
-#
-#   --rp-loc files
-#
-# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
-
-server.port = 80
-server.document-root = "/var/empty"
-server.upload-dirs = ( "/dev/shm", "/tmp" )
-server.modules = ( "mod_proxy" )
-$HTTP["url"] =~ "^/files" {
-    proxy.forwarded = ( "for" => 1, "proto" => 1 )
-    proxy.server = ( "" => ( ( "host" => "/dev/shm/party.sock" ) ) )
-
-    # if you really need to use tcp instead of unix-sockets, do this instead:
-    #proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "3923" ) ) )
-}

contrib/nginx/copyparty.conf

@@ -2,38 +2,19 @@
 # not accept more consecutive clients than what copyparty is able to;
 # nginx default is 512  (worker_processes 1, worker_connections 512)
 #
-# ======================================================================
-#
-# to reverse-proxy a specific path/subpath/location below a domain
-# (rather than a complete subdomain), for example "/qw/er", you must
-# run copyparty with --rp-loc /qw/as and also change the following:
-# 	location / {
-# 		proxy_pass http://cpp_tcp;
-# to this:
-# 	location /qw/er/ {
-# 		proxy_pass http://cpp_tcp/qw/er/;
-#
-# ======================================================================
-#
 # rarely, in some extreme usecases, it can be good to add -j0
 # (40'000 requests per second, or 20gbps upload/download in parallel)
 # but this is usually counterproductive and slightly buggy
 #
-# ======================================================================
-#
 # on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
 #
-# ======================================================================
-#
-# if you are behind cloudflare (or another CDN/WAF/protection service),
+# if you are behind cloudflare (or another protection service),
 # remember to reject all connections which are not coming from your
 # protection service -- for cloudflare in particular, you can
 # generate the list of permitted IP ranges like so:
 #   (curl -s https://www.cloudflare.com/ips-v{4,6} | sed 's/^/allow /; s/$/;/'; echo; echo "deny all;") > /etc/nginx/cloudflare-only.conf
 #
 # and then enable it below by uncomenting the cloudflare-only.conf line
-#
-# ======================================================================
 
 
 upstream cpp_tcp {
@@ -55,9 +36,9 @@ upstream cpp_uds {
 	# but there must be at least one unix-group which both
 	# nginx and copyparty is a member of; if that group is
 	# "www" then run copyparty with the following args:
-	# -i unix:770:www:/dev/shm/party.sock
+	# -i unix:770:www:/tmp/party.sock
 
-	server unix:/dev/shm/party.sock fail_timeout=1s;
+	server unix:/tmp/party.sock fail_timeout=1s;
 	keepalive 1;
 }
 
@@ -80,18 +61,14 @@ server {
 		client_max_body_size 0;
 		proxy_buffering off;
 		proxy_request_buffering off;
-		# improve download speed from 600 to 1500 MiB/s
-		proxy_buffers 32 8k;
-		proxy_buffer_size 16k;
-		proxy_busy_buffers_size 24k;
 
-		proxy_set_header   Connection        "Keep-Alive";
 		proxy_set_header   Host              $host;
 		proxy_set_header   X-Real-IP         $remote_addr;
-		proxy_set_header   X-Forwarded-Proto $scheme;
 		proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
-		# NOTE: with cloudflare you want this X-Forwarded-For instead:
+		# NOTE: with cloudflare you want this instead:
 		#proxy_set_header   X-Forwarded-For   $http_cf_connecting_ip;
+		proxy_set_header   X-Forwarded-Proto $scheme;
+		proxy_set_header   Connection        "Keep-Alive";
 	}
 }
 

contrib/nixos/modules/copyparty.nix

@@ -1,28 +1,23 @@
-{
-  config,
-  pkgs,
-  lib,
-  ...
-}:
+{ config, pkgs, lib, ... }:
+
 with lib;
+
 let
-  mkKeyValue =
-    key: value:
+  mkKeyValue = key: value:
     if value == true then
-      # sets with a true boolean value are coerced to just the key name
+    # sets with a true boolean value are coerced to just the key name
       key
     else if value == false then
-      # or omitted completely when false
+    # or omitted completely when false
       ""
     else
       (generators.mkKeyValueDefault { inherit mkValueString; } ": " key value);
 
   mkAttrsString = value: (generators.toKeyValue { inherit mkKeyValue; } value);
 
-  mkValueString =
-    value:
+  mkValueString = value:
     if isList value then
-      (concatStringsSep "," (map mkValueString value))
+      (concatStringsSep ", " (map mkValueString value))
     else if isAttrs value then
       "\n" + (mkAttrsString value)
     else
@@ -54,14 +49,13 @@ let
     ${concatStringsSep "\n" (mapAttrsToList mkVolume cfg.volumes)}
   '';
 
+  name = "copyparty";
   cfg = config.services.copyparty;
-  configFile = pkgs.writeText "copyparty.conf" configStr;
-  runtimeConfigPath = "/run/copyparty/copyparty.conf";
-  externalCacheDir = "/var/cache/copyparty";
-  externalStateDir = "/var/lib/copyparty";
-  defaultShareDir = "${externalStateDir}/data";
-in
-{
+  configFile = pkgs.writeText "${name}.conf" configStr;
+  runtimeConfigPath = "/run/${name}/${name}.conf";
+  home = "/var/lib/${name}";
+  defaultShareDir = "${home}/data";
+in {
   options.services.copyparty = {
     enable = mkEnableOption "web-based file manager";
 
@@ -74,35 +68,6 @@ in
       '';
     };
 
-    mkHashWrapper = mkOption {
-      type = types.bool;
-      default = true;
-      description = ''
-        Make a shell script wrapper called 'copyparty-hash' with all options set here,
-        that launches the hashing cli.
-      '';
-    };
-
-    user = mkOption {
-      type = types.str;
-      default = "copyparty";
-      description = ''
-        The user that copyparty will run under.
-
-        If changed from default, you are responsible for making sure the user exists.
-      '';
-    };
-
-    group = mkOption {
-      type = types.str;
-      default = "copyparty";
-      description = ''
-        The group that copyparty will run under.
-
-        If changed from default, you are responsible for making sure the user exists.
-      '';
-    };
-
     openFilesLimit = mkOption {
       default = 4096;
       type = types.either types.int types.str;
@@ -114,41 +79,33 @@ in
       description = ''
         Global settings to apply.
         Directly maps to values in the [global] section of the copyparty config.
-        Cannot set "c" or "hist", those are set by this module.
         See `${getExe cfg.package} --help` for more details.
       '';
       default = {
         i = "127.0.0.1";
         no-reload = true;
-        hist = externalCacheDir;
       };
       example = literalExpression ''
         {
           i = "0.0.0.0";
           no-reload = true;
-          hist = ${externalCacheDir};
         }
       '';
     };
 
     accounts = mkOption {
-      type = types.attrsOf (
-        types.submodule (
-          { ... }:
-          {
-            options = {
-              passwordFile = mkOption {
-                type = types.str;
-                description = ''
-                  Runtime file path to a file containing the user password.
-                  Must be readable by the copyparty user.
-                '';
-                example = "/run/keys/copyparty/ed";
-              };
-            };
-          }
-        )
-      );
+      type = types.attrsOf (types.submodule ({ ... }: {
+        options = {
+          passwordFile = mkOption {
+            type = types.str;
+            description = ''
+              Runtime file path to a file containing the user password.
+              Must be readable by the copyparty user.
+            '';
+            example = "/run/keys/copyparty/ed";
+          };
+        };
+      }));
       description = ''
         A set of copyparty accounts to create.
       '';
@@ -161,81 +118,74 @@ in
     };
 
     volumes = mkOption {
-      type = types.attrsOf (
-        types.submodule (
-          { ... }:
-          {
-            options = {
-              path = mkOption {
-                type = types.path;
-                description = ''
-                  Path of a directory to share.
-                '';
-              };
-              access = mkOption {
-                type = types.attrs;
-                description = ''
-                  Attribute list of permissions and the users to apply them to.
+      type = types.attrsOf (types.submodule ({ ... }: {
+        options = {
+          path = mkOption {
+            type = types.str;
+            description = ''
+              Path of a directory to share.
+            '';
+          };
+          access = mkOption {
+            type = types.attrs;
+            description = ''
+              Attribute list of permissions and the users to apply them to.
 
-                  The key must be a string containing any combination of allowed permission:
-                    "r" (read):   list folder contents, download files
-                    "w" (write):  upload files; need "r" to see the uploads
-                    "m" (move):   move files and folders; need "w" at destination
-                    "d" (delete): permanently delete files and folders
-                    "g" (get):    download files, but cannot see folder contents
-                    "G" (upget):  "get", but can see filekeys of their own uploads
-                    "h" (html):   "get", but folders return their index.html
-                    "a" (admin):  can see uploader IPs, config-reload
+              The key must be a string containing any combination of allowed permission:
+                "r" (read):   list folder contents, download files
+                "w" (write):  upload files; need "r" to see the uploads
+                "m" (move):   move files and folders; need "w" at destination
+                "d" (delete): permanently delete files and folders
+                "g" (get):    download files, but cannot see folder contents
+                "G" (upget):  "get", but can see filekeys of their own uploads
+                "h" (html):   "get", but folders return their index.html
+                "a" (admin):  can see uploader IPs, config-reload
 
-                  For example: "rwmd"
+              For example: "rwmd"
 
-                  The value must be one of:
-                    an account name, defined in `accounts`
-                    a list of account names
-                    "*", which means "any account"
-                '';
-                example = literalExpression ''
-                  {
-                    # wG = write-upget = see your own uploads only
-                    wG = "*";
-                    # read-write-modify-delete for users "ed" and "k"
-                    rwmd = ["ed" "k"];
-                  };
-                '';
+              The value must be one of:
+                an account name, defined in `accounts`
+                a list of account names
+                "*", which means "any account"
+            '';
+            example = literalExpression ''
+              {
+                # wG = write-upget = see your own uploads only
+                wG = "*";
+                # read-write-modify-delete for users "ed" and "k"
+                rwmd = ["ed" "k"];
               };
-              flags = mkOption {
-                type = types.attrs;
-                description = ''
-                  Attribute list of volume flags to apply.
-                  See `${getExe cfg.package} --help-flags` for more details.
-                '';
-                example = literalExpression ''
-                  {
-                    # "fk" enables filekeys (necessary for upget permission) (4 chars long)
-                    fk = 4;
-                    # scan for new files every 60sec
-                    scan = 60;
-                    # volflag "e2d" enables the uploads database
-                    e2d = true;
-                    # "d2t" disables multimedia parsers (in case the uploads are malicious)
-                    d2t = true;
-                    # skips hashing file contents if path matches *.iso
-                    nohash = "\.iso$";
-                  };
-                '';
-                default = { };
+            '';
+          };
+          flags = mkOption {
+            type = types.attrs;
+            description = ''
+              Attribute list of volume flags to apply.
+              See `${getExe cfg.package} --help-flags` for more details.
+            '';
+            example = literalExpression ''
+              {
+                # "fk" enables filekeys (necessary for upget permission) (4 chars long)
+                fk = 4;
+                # scan for new files every 60sec
+                scan = 60;
+                # volflag "e2d" enables the uploads database
+                e2d = true;
+                # "d2t" disables multimedia parsers (in case the uploads are malicious)
+                d2t = true;
+                # skips hashing file contents if path matches *.iso
+                nohash = "\.iso$";
               };
-            };
-          }
-        )
-      );
+            '';
+            default = { };
+          };
+        };
+      }));
       description = "A set of copyparty volumes to create";
       default = {
         "/" = {
           path = defaultShareDir;
-          access = {
-            r = "*";
-          };
+          access = { r = "*"; };
         };
       };
       example = literalExpression ''
@@ -254,122 +204,80 @@ in
     };
   };
 
-  config = mkIf cfg.enable (
-    let
-      command = "${getExe cfg.package} -c ${runtimeConfigPath}";
-    in
-    {
-      systemd.services.copyparty = {
-        description = "http file sharing hub";
-        wantedBy = [ "multi-user.target" ];
+  config = mkIf cfg.enable {
+    systemd.services.copyparty = {
+      description = "http file sharing hub";
+      wantedBy = [ "multi-user.target" ];
 
-        environment = {
-          PYTHONUNBUFFERED = "true";
-          XDG_CONFIG_HOME = externalStateDir;
-        };
-
-        preStart =
-          let
-            replaceSecretCommand =
-              name: attrs:
-              "${getExe pkgs.replace-secret} '${passwordPlaceholder name}' '${attrs.passwordFile}' ${runtimeConfigPath}";
-          in
-          ''
-            set -euo pipefail
-            install -m 600 ${configFile} ${runtimeConfigPath}
-            ${concatStringsSep "\n" (mapAttrsToList replaceSecretCommand cfg.accounts)}
-          '';
-
-        serviceConfig = {
-          Type = "simple";
-          ExecStart = command;
-          # Hardening options
-          User = cfg.user;
-          Group = cfg.group;
-          RuntimeDirectory = [ "copyparty" ];
-          RuntimeDirectoryMode = "0700";
-          StateDirectory = [ "copyparty" ];
-          StateDirectoryMode = "0700";
-          CacheDirectory = lib.mkIf (cfg.settings ? hist) [ "copyparty" ];
-          CacheDirectoryMode = lib.mkIf (cfg.settings ? hist) "0700";
-          WorkingDirectory = externalStateDir;
-          BindReadOnlyPaths = [
-            "/nix/store"
-            "-/etc/resolv.conf"
-            "-/etc/nsswitch.conf"
-            "-/etc/group"
-            "-/etc/hosts"
-            "-/etc/localtime"
-          ] ++ (mapAttrsToList (k: v: "-${v.passwordFile}") cfg.accounts);
-          BindPaths =
-            (if cfg.settings ? hist then [ cfg.settings.hist ] else [ ])
-            ++ [ externalStateDir ]
-            ++ (mapAttrsToList (k: v: v.path) cfg.volumes);
-          # ProtectSystem = "strict";
-          # Note that unlike what 'ro' implies,
-          # this actually makes it impossible to read anything in the root FS,
-          # except for things explicitly mounted via `RuntimeDirectory`, `StateDirectory`, `CacheDirectory`, and `BindReadOnlyPaths`.
-          # This is because TemporaryFileSystem creates a *new* *empty* filesystem for the process, so only bindmounts are visible.
-          TemporaryFileSystem = "/:ro";
-          PrivateTmp = true;
-          PrivateDevices = true;
-          ProtectKernelTunables = true;
-          ProtectControlGroups = true;
-          RestrictSUIDSGID = true;
-          PrivateMounts = true;
-          ProtectKernelModules = true;
-          ProtectKernelLogs = true;
-          ProtectHostname = true;
-          ProtectClock = true;
-          ProtectProc = "invisible";
-          ProcSubset = "pid";
-          RestrictNamespaces = true;
-          RemoveIPC = true;
-          UMask = "0077";
-          LimitNOFILE = cfg.openFilesLimit;
-          NoNewPrivileges = true;
-          LockPersonality = true;
-          RestrictRealtime = true;
-          MemoryDenyWriteExecute = true;
-        };
+      environment = {
+        PYTHONUNBUFFERED = "true";
+        XDG_CONFIG_HOME = "${home}/.config";
       };
 
-      # ensure volumes exist:
-      systemd.tmpfiles.settings."copyparty" = (
-        lib.attrsets.mapAttrs' (
-          name: value:
-          lib.attrsets.nameValuePair (value.path) {
-            d = {
-              #: in front of things means it wont change it if the directory already exists.
-              group = ":${cfg.group}";
-              user = ":${cfg.user}";
-              mode = ":755";
-            };
-          }
-        ) cfg.volumes
-      );
+      preStart = let
+        replaceSecretCommand = name: attrs:
+          "${getExe pkgs.replace-secret} '${
+            passwordPlaceholder name
+          }' '${attrs.passwordFile}' ${runtimeConfigPath}";
+      in ''
+        set -euo pipefail
+        install -m 600 ${configFile} ${runtimeConfigPath}
+        ${concatStringsSep "\n"
+        (mapAttrsToList replaceSecretCommand cfg.accounts)}
+      '';
 
-      users.groups.copyparty = lib.mkIf (cfg.user == "copyparty" && cfg.group == "copyparty") { };
-      users.users.copyparty = lib.mkIf (cfg.user == "copyparty" && cfg.group == "copyparty") {
-        description = "Service user for copyparty";
-        group = "copyparty";
-        home = externalStateDir;
-        isSystemUser = true;
+      serviceConfig = {
+        Type = "simple";
+        ExecStart = "${getExe cfg.package} -c ${runtimeConfigPath}";
+
+        # Hardening options
+        User = "copyparty";
+        Group = "copyparty";
+        RuntimeDirectory = name;
+        RuntimeDirectoryMode = "0700";
+        StateDirectory = [ name "${name}/data" "${name}/.config" ];
+        StateDirectoryMode = "0700";
+        WorkingDirectory = home;
+        TemporaryFileSystem = "/:ro";
+        BindReadOnlyPaths = [
+          "/nix/store"
+          "-/etc/resolv.conf"
+          "-/etc/nsswitch.conf"
+          "-/etc/hosts"
+          "-/etc/localtime"
+        ] ++ (mapAttrsToList (k: v: "-${v.passwordFile}") cfg.accounts);
+        BindPaths = [ home ] ++ (mapAttrsToList (k: v: v.path) cfg.volumes);
+        # Would re-mount paths ignored by temporary root
+        #ProtectSystem = "strict";
+        ProtectHome = true;
+        PrivateTmp = true;
+        PrivateDevices = true;
+        ProtectKernelTunables = true;
+        ProtectControlGroups = true;
+        RestrictSUIDSGID = true;
+        PrivateMounts = true;
+        ProtectKernelModules = true;
+        ProtectKernelLogs = true;
+        ProtectHostname = true;
+        ProtectClock = true;
+        ProtectProc = "invisible";
+        ProcSubset = "pid";
+        RestrictNamespaces = true;
+        RemoveIPC = true;
+        UMask = "0077";
+        LimitNOFILE = cfg.openFilesLimit;
+        NoNewPrivileges = true;
+        LockPersonality = true;
+        RestrictRealtime = true;
       };
-      environment.systemPackages = lib.mkIf cfg.mkHashWrapper [
-        (pkgs.writeShellScriptBin "copyparty-hash" ''
-          set -a  # automatically export variables
-          # set same environment variables as the systemd service
-          ${lib.pipe config.systemd.services.copyparty.environment [
-            (lib.filterAttrs (n: v: v != null && n != "PATH"))
-            (lib.mapAttrs (_: v: "${v}"))
-            (lib.toShellVars)
-          ]}
-          PATH=${config.systemd.services.copyparty.environment.PATH}:$PATH
+    };
 
-          exec ${command} --ah-cli
-        '')
-      ];
-    }
-  );
+    users.groups.copyparty = { };
+    users.users.copyparty = {
+      description = "Service user for copyparty";
+      group = "copyparty";
+      home = home;
+      isSystemUser = true;
+    };
+  };
 }

contrib/package/arch/PKGBUILD

@@ -1,48 +1,56 @@
 # Maintainer: icxes <dev.null@need.moe>
-# Contributor: Morgan Adamiec <morganamilo@archlinux.org>
-# NOTE: You generally shouldn't use this PKGBUILD on Arch, as it is mainly for testing purposes. Install copyparty using pacman instead.
-
 pkgname=copyparty
-pkgver="1.19.4"
+pkgver="1.16.5"
 pkgrel=1
 pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
-depends=("bash" "python" "lsof" "python-jinja")
+depends=("python" "lsof" "python-jinja")
 makedepends=("python-wheel" "python-setuptools" "python-build" "python-installer" "make" "pigz")
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
-            "cfssl: generate TLS certificates on startup"
-            "python-mutagen: music tags (alternative)"
-            "python-pillow: thumbnails for images"
-            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)"
-            "libkeyfinder: detection of musical keys"
-            "python-pyopenssl: ftps functionality"
-            "python-pyzmq: send zeromq messages from event-hooks"
-            "python-argon2-cffi: hashed passwords in config"
+            "cfssl: generate TLS certificates on startup (pointless when reverse-proxied)"
+            "python-mutagen: music tags (alternative)" 
+            "python-pillow: thumbnails for images" 
+            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)" 
+            "libkeyfinder-git: detection of musical keys" 
+            "qm-vamp-plugins: BPM detection" 
+            "python-pyopenssl: ftps functionality" 
+            "python-argon2_cffi: hashed passwords in config" 
+            "python-impacket-git: smb support (bad idea)"
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
-backup=("etc/${pkgname}/copyparty.conf" )
-sha256sums=("b0e84a78eb2701cb7447b6023afcec280c550617dde67b6f0285bb23483111eb")
+backup=("etc/${pkgname}.d/init" )
+sha256sums=("2830086bd872aaa5174c2ca73ba395439e85c883d85438263bd89521c5f37d9c")
 
 build() {
-    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
-    make
-
     cd "${srcdir}/${pkgname}-${pkgver}"
-    python -m build --wheel --no-isolation
+    
+    pushd copyparty/web
+    make -j$(nproc)
+    rm Makefile
+    popd
+    
+    python3 -m build -wn
 }
 
 package() {
     cd "${srcdir}/${pkgname}-${pkgver}"
-    python -m installer --destdir="$pkgdir" dist/*.whl
+    python3 -m installer -d "$pkgdir" dist/*.whl
 
-    install -dm755 "${pkgdir}/etc/${pkgname}"
+    install -dm755 "${pkgdir}/etc/${pkgname}.d"
     install -Dm755 "bin/prisonparty.sh" "${pkgdir}/usr/bin/prisonparty"
-    install -Dm644 "contrib/systemd/${pkgname}.conf" "${pkgdir}/etc/${pkgname}/copyparty.conf"
-    install -Dm644 "contrib/systemd/${pkgname}@.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}@.service"
-    install -Dm644 "contrib/systemd/${pkgname}-user.service" "${pkgdir}/usr/lib/systemd/user/${pkgname}.service"
-    install -Dm644 "contrib/systemd/prisonparty@.service" "${pkgdir}/usr/lib/systemd/system/prisonparty@.service"
-    install -Dm644 "contrib/systemd/index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
+    install -Dm644 "contrib/package/arch/${pkgname}.conf" "${pkgdir}/etc/${pkgname}.d/init"
+    install -Dm644 "contrib/package/arch/${pkgname}.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}.service"
+    install -Dm644 "contrib/package/arch/prisonparty.service" "${pkgdir}/usr/lib/systemd/system/prisonparty.service"
+    install -Dm644 "contrib/package/arch/index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
     install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
+
+    find /etc/${pkgname}.d -iname '*.conf' 2>/dev/null | grep -qE . && return
+    echo "┏━━━━━━━━━━━━━━━──-"
+    echo "┃ Configure ${pkgname} by adding .conf files into /etc/${pkgname}.d/"
+    echo "┃ and maybe copy+edit one of the following to /etc/systemd/system/:"
+    echo "┣━♦ /usr/lib/systemd/system/${pkgname}.service   (standard)"
+    echo "┣━♦ /usr/lib/systemd/system/prisonparty.service (chroot)"
+    echo "┗━━━━━━━━━━━━━━━──-"
 }

contrib/package/makedeb-mpr/PKGBUILD

@@ -1,44 +0,0 @@
-# Contributor: Beethoven <beethovenisadog@protonmail.com>
-
-
-pkgname=copyparty
-pkgver=1.19.4
-pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
-arch=("any")
-url="https://github.com/9001/${pkgname}"
-license=('MIT')
-depends=("bash" "python3" "lsof" "python3-jinja2")
-makedepends=("python3-wheel" "python3-setuptools" "python3-build" "python3-installer" "make" "pigz")
-optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
-            "golang-cfssl: generate TLS certificates on startup"
-            "python3-mutagen: music tags (alternative)"
-            "python3-pil: thumbnails for images"
-            "python3-openssl: ftps functionality"
-            "python3-zmq: send zeromq messages from event-hooks"
-            "python3-argon2: hashed passwords in config"
-)
-source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
-backup=("/etc/${pkgname}.d/init" )
-sha256sums=("b0e84a78eb2701cb7447b6023afcec280c550617dde67b6f0285bb23483111eb")
-
-build() {
-    cd "${srcdir}/${pkgname}-${pkgver}/copyparty/web"
-    make
-
-    cd "${srcdir}/${pkgname}-${pkgver}"
-    python -m build --wheel --no-isolation
-}
-
-package() {
-    cd "${srcdir}/${pkgname}-${pkgver}"
-    python -m installer --destdir="$pkgdir" dist/*.whl
-
-    install -dm755 "${pkgdir}/etc/${pkgname}.d"
-    install -Dm755 "bin/prisonparty.sh" "${pkgdir}/usr/bin/prisonparty"
-    install -Dm644 "contrib/package/makedeb-mpr/${pkgname}.conf" "${pkgdir}/etc/${pkgname}.d/init"
-    install -Dm644 "contrib/package/makedeb-mpr/${pkgname}.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}.service"
-    install -Dm644 "contrib/package/makedeb-mpr/prisonparty.service" "${pkgdir}/usr/lib/systemd/system/prisonparty.service"
-    install -Dm644 "contrib/package/makedeb-mpr/index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
-    install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
-}

contrib/package/nix/copyparty/default.nix

@@ -1,118 +1,63 @@
-{
-  lib,
-  buildPythonApplication,
-  fetchurl,
-  util-linux,
-  python,
-  setuptools,
-  jinja2,
-  impacket,
-  pyopenssl,
-  cfssl,
-  argon2-cffi,
-  pillow,
-  pyvips,
-  pyzmq,
-  ffmpeg,
-  mutagen,
-  pyftpdlib,
-  magic,
-  partftpy,
-  fusepy, # for partyfuse
+{ lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, argon2-cffi, pillow, pyvips, ffmpeg, mutagen,
 
-  # use argon2id-hashed passwords in config files (sha2 is always available)
-  withHashedPasswords ? true,
+# use argon2id-hashed passwords in config files (sha2 is always available)
+withHashedPasswords ? true,
 
-  # generate TLS certificates on startup (pointless when reverse-proxied)
-  withCertgen ? false,
+# generate TLS certificates on startup (pointless when reverse-proxied)
+withCertgen ? false,
 
-  # create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
-  withThumbnails ? true,
+# create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
+withThumbnails ? true,
 
-  # create thumbnails with PyVIPS; even faster, uses more memory
-  # -- can be combined with Pillow to support more filetypes
-  withFastThumbnails ? false,
+# create thumbnails with PyVIPS; even faster, uses more memory
+# -- can be combined with Pillow to support more filetypes
+withFastThumbnails ? false,
 
-  # enable FFmpeg; thumbnails for most filetypes (also video and audio), extract audio metadata, transcode audio to opus
-  # -- possibly dangerous if you allow anonymous uploads, since FFmpeg has a huge attack surface
-  # -- can be combined with Thumbnails and/or FastThumbnails, since FFmpeg is slower than both
-  withMediaProcessing ? true,
+# enable FFmpeg; thumbnails for most filetypes (also video and audio), extract audio metadata, transcode audio to opus
+# -- possibly dangerous if you allow anonymous uploads, since FFmpeg has a huge attack surface
+# -- can be combined with Thumbnails and/or FastThumbnails, since FFmpeg is slower than both
+withMediaProcessing ? true,
 
-  # if MediaProcessing is not enabled, you probably want this instead (less accurate, but much safer and faster)
-  withBasicAudioMetadata ? false,
+# if MediaProcessing is not enabled, you probably want this instead (less accurate, but much safer and faster)
+withBasicAudioMetadata ? false,
 
-  # send ZeroMQ messages from event-hooks
-  withZeroMQ ? true,
+# enable FTPS support in the FTP server
+withFTPS ? false,
 
-  # enable FTP server
-  withFTP ? true,
-
-  # enable FTPS support in the FTP server
-  withFTPS ? false,
-
-  # enable TFTP server
-  withTFTP ? false,
-
-  # samba/cifs server; dangerous and buggy, enable if you really need it
-  withSMB ? false,
-
-  # enables filetype detection for nameless uploads
-  withMagic ? false,
-
-  # extra packages to add to the PATH
-  extraPackages ? [ ],
-
-  # function that accepts a python packageset and returns a list of packages to
-  # be added to the python venv. useful for scripts and such that require
-  # additional dependencies
-  extraPythonPackages ? (_p: [ ]),
+# samba/cifs server; dangerous and buggy, enable if you really need it
+withSMB ? false,
 
 }:
 
 let
   pinData = lib.importJSON ./pin.json;
-  runtimeDeps = ([ util-linux ] ++ extraPackages ++ lib.optional withMediaProcessing ffmpeg);
-in
-buildPythonApplication {
-  pname = "copyparty";
-  inherit (pinData) version;
-  src = fetchurl {
-    inherit (pinData) url hash;
-  };
-  dependencies =
-    [
+  pyEnv = python.withPackages (ps:
+    with ps; [
       jinja2
-      fusepy
     ]
     ++ lib.optional withSMB impacket
-    ++ lib.optional withFTP pyftpdlib
     ++ lib.optional withFTPS pyopenssl
-    ++ lib.optional withTFTP partftpy
     ++ lib.optional withCertgen cfssl
     ++ lib.optional withThumbnails pillow
     ++ lib.optional withFastThumbnails pyvips
     ++ lib.optional withMediaProcessing ffmpeg
     ++ lib.optional withBasicAudioMetadata mutagen
     ++ lib.optional withHashedPasswords argon2-cffi
-    ++ lib.optional withZeroMQ pyzmq
-    ++ lib.optional withMagic magic
-    ++ (extraPythonPackages python.pkgs);
-  makeWrapperArgs = [ "--prefix PATH : ${lib.makeBinPath runtimeDeps}" ];
-
-  pyproject = true;
-  build-system = [
-    setuptools
-  ];
-  meta = {
-    description = "Turn almost any device into a file server";
-    longDescription = ''
-      Portable file server with accelerated resumable uploads, dedup, WebDAV,
-      FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
-    '';
-    homepage = "https://github.com/9001/copyparty";
-    changelog = "https://github.com/9001/copyparty/releases/tag/v${pinData.version}";
-    license = lib.licenses.mit;
-    mainProgram = "copyparty";
-    sourceProvenance = [ lib.sourceTypes.fromSource ];
+    );
+in stdenv.mkDerivation {
+  pname = "copyparty";
+  version = pinData.version;
+  src = fetchurl {
+    url = pinData.url;
+    hash = pinData.hash;
   };
+  buildInputs = [ makeWrapper ];
+  dontUnpack = true;
+  dontBuild = true;
+  installPhase = ''
+    install -Dm755 $src $out/share/copyparty-sfx.py
+    makeWrapper ${pyEnv.interpreter} $out/bin/copyparty \
+      --set PATH '${lib.makeBinPath ([ utillinux ] ++ lib.optional withMediaProcessing ffmpeg)}:$PATH' \
+      --add-flags "$out/share/copyparty-sfx.py"
+  '';
 }

contrib/package/nix/copyparty/pin.json

@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.19.4/copyparty-1.19.4.tar.gz",
-    "version": "1.19.4",
-    "hash": "sha256-sOhKeOsnAct0R7YCOvzsKAxVBhfd5ntvAoW7I0gxEes="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.16.5/copyparty-sfx.py",
+    "version": "1.16.5",
+    "hash": "sha256-rfZ76ujA6bLXYW52qP2pK8gdwDdED91mLOF2gzquG8E="
 }

contrib/package/nix/copyparty/update.py

@@ -11,14 +11,14 @@ import base64
 import json
 import hashlib
 import sys
-import tarfile
+import re
 from pathlib import Path
 
 OUTPUT_FILE = Path("pin.json")
-TARGET_ASSET = lambda version: f"copyparty-{version}.tar.gz"
+TARGET_ASSET = "copyparty-sfx.py"
 HASH_TYPE = "sha256"
 LATEST_RELEASE_URL = "https://api.github.com/repos/9001/copyparty/releases/latest"
-DOWNLOAD_URL = lambda version: f"https://github.com/9001/copyparty/releases/download/v{version}/{TARGET_ASSET(version)}"
+DOWNLOAD_URL = lambda version: f"https://github.com/9001/copyparty/releases/download/v{version}/{TARGET_ASSET}"
 
 
 def get_formatted_hash(binary):
@@ -29,13 +29,11 @@ def get_formatted_hash(binary):
     return f"{HASH_TYPE}-{encoded_hash}"
 
 
-def version_from_tar_gz(path):
-    with tarfile.open(path) as tarball:
-        release_name = tarball.getmembers()[0].name
-        prefix = "copyparty-"
+def version_from_sfx(binary):
+    result = re.search(b'^VER = "(.*)"$', binary, re.MULTILINE)
+    if result:
+        return result.groups(1)[0].decode("ascii")
 
-        if release_name.startswith(prefix):
-            return release_name.replace(prefix, "")
     raise ValueError("version not found in provided file")
 
 
@@ -44,7 +42,7 @@ def remote_release_pin():
 
     response = requests.get(LATEST_RELEASE_URL).json()
     version = response["tag_name"].lstrip("v")
-    asset_info = [a for a in response["assets"] if a["name"] == TARGET_ASSET(version)][0]
+    asset_info = [a for a in response["assets"] if a["name"] == TARGET_ASSET][0]
     download_url = asset_info["browser_download_url"]
     asset = requests.get(download_url)
     formatted_hash = get_formatted_hash(asset.content)
@@ -54,9 +52,10 @@ def remote_release_pin():
 
 
 def local_release_pin(path):
-    version = version_from_tar_gz(path)
+    asset = path.read_bytes()
+    version = version_from_sfx(asset)
     download_url = DOWNLOAD_URL(version)
-    formatted_hash = get_formatted_hash(path.read_bytes())
+    formatted_hash = get_formatted_hash(asset)
 
     result = {"url": download_url, "version": version, "hash": formatted_hash}
     return result

contrib/package/nix/overlay.nix

@@ -1,11 +0,0 @@
-final: prev: {
-  copyparty = final.python3.pkgs.callPackage ./copyparty {
-    ffmpeg = final.ffmpeg-full;
-  };
-
-  python3 = prev.python3.override {
-    packageOverrides = pyFinal: pyPrev: {
-      partftpy = pyFinal.callPackage ./partftpy { };
-    };
-  };
-}

contrib/package/nix/partftpy/default.nix

@@ -1,30 +0,0 @@
-{
-  lib,
-  buildPythonPackage,
-  fetchurl,
-  setuptools,
-}:
-let
-  pinData = lib.importJSON ./pin.json;
-in
-
-buildPythonPackage rec {
-  pname = "partftpy";
-  inherit (pinData) version;
-  pyproject = true;
-
-  src = fetchurl {
-    inherit (pinData) url hash;
-  };
-
-  build-system = [ setuptools ];
-
-  pythonImportsCheck = [ "partftpy.TftpServer" ];
-
-  meta = {
-    description = "Pure Python TFTP library  (copyparty edition)";
-    homepage = "https://github.com/9001/partftpy";
-    changelog = "https://github.com/9001/partftpy/releases/tag/${version}";
-    license = lib.licenses.mit;
-  };
-}

contrib/package/nix/partftpy/pin.json

@@ -1,5 +0,0 @@
-{
-    "url": "https://github.com/9001/partftpy/releases/download/v0.4.0/partftpy-0.4.0.tar.gz",
-    "version": "0.4.0",
-    "hash": "sha256-5Q2zyuJ892PGZmb+YXg0ZPW/DK8RDL1uE0j5HPd4We0="
-}

contrib/package/nix/partftpy/update.py

@@ -1,50 +0,0 @@
-#!/usr/bin/env python3
-
-# Update the Nix package pin
-#
-# Usage: ./update.sh
-
-import base64
-import json
-import hashlib
-import sys
-from pathlib import Path
-
-OUTPUT_FILE = Path("pin.json")
-TARGET_ASSET = lambda version: f"partftpy-{version}.tar.gz"
-HASH_TYPE = "sha256"
-LATEST_RELEASE_URL = "https://api.github.com/repos/9001/partftpy/releases/latest"
-
-
-def get_formatted_hash(binary):
-    hasher = hashlib.new("sha256")
-    hasher.update(binary)
-    asset_hash = hasher.digest()
-    encoded_hash = base64.b64encode(asset_hash).decode("ascii")
-    return f"{HASH_TYPE}-{encoded_hash}"
-
-
-def remote_release_pin():
-    import requests
-
-    response = requests.get(LATEST_RELEASE_URL).json()
-    version = response["tag_name"].lstrip("v")
-    asset_info = [a for a in response["assets"] if a["name"] == TARGET_ASSET(version)][0]
-    download_url = asset_info["browser_download_url"]
-    asset = requests.get(download_url)
-    formatted_hash = get_formatted_hash(asset.content)
-
-    result = {"url": download_url, "version": version, "hash": formatted_hash}
-    return result
-
-
-def main():
-    result = remote_release_pin()
-
-    print(result)
-    json_result = json.dumps(result, indent=4)
-    OUTPUT_FILE.write_text(json_result)
-
-
-if __name__ == "__main__":
-    main()

contrib/package/rpm/copyparty.spec

@@ -1,62 +0,0 @@
-Name:           copyparty
-Version:        $pkgver
-Release:        $pkgrel
-License:        MIT
-Group:          Utilities
-URL:            https://github.com/9001/copyparty
-Source0:        copyparty-$pkgver.tar.gz
-Summary:        File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++
-BuildArch:      noarch
-BuildRequires:  python3, python3-devel, pyproject-rpm-macros, python-setuptools, python-wheel, make
-Requires:       python3, (python3-jinja2 or python-jinja2), lsof
-Recommends:     ffmpeg, (golang-github-cloudflare-cfssl or cfssl), python-mutagen, python-pillow, python-pyvips
-Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-impacket
-
-%description
-Portable file server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
-
-See release at https://github.com/9001/copyparty/releases
-
-%global debug_package %{nil}
-
-%generate_buildrequires
-%pyproject_buildrequires
-
-%prep
-%setup -q
-
-%build
-cd "copyparty/web"
-make
-cd -
-%pyproject_wheel
-
-%install
-mkdir -p %{buildroot}%{_bindir}
-mkdir -p %{buildroot}%{_libdir}/systemd/{system,user}
-mkdir -p %{buildroot}/etc/%{name}
-mkdir -p %{buildroot}/var/lib/%{name}-jail
-mkdir -p %{buildroot}%{_datadir}/licenses/%{name}
-
-%pyproject_install
-%pyproject_save_files copyparty
-
-install -m 0755 bin/prisonparty.sh                       %{buildroot}%{_bindir}/prisonpary.sh
-install -m 0644 contrib/systemd/%{name}.conf             %{buildroot}/etc/%{name}/%{name}.conf
-install -m 0644 contrib/systemd/%{name}@.service         %{buildroot}%{_libdir}/systemd/system/%{name}@.service
-install -m 0644 contrib/systemd/%{name}-user.service     %{buildroot}%{_libdir}/systemd/user/%{name}.service
-install -m 0644 contrib/systemd/prisonparty@.service     %{buildroot}%{_libdir}/systemd/system/prisonparty@.service
-install -m 0644 contrib/systemd/index.md                 %{buildroot}/var/lib/%{name}-jail/README.md
-install -m 0644 LICENSE                                  %{buildroot}%{_datadir}/licenses/%{name}/LICENSE
-
-%files -n copyparty -f %{pyproject_files}
-%license LICENSE
-%{_bindir}/copyparty
-%{_bindir}/partyfuse
-%{_bindir}/u2c
-%{_bindir}/prisonpary.sh
-/etc/%{name}/%{name}.conf
-%{_libdir}/systemd/system/%{name}@.service
-%{_libdir}/systemd/user/%{name}.service
-%{_libdir}/systemd/system/prisonparty@.service
-/var/lib/%{name}-jail/README.md

contrib/plugins/README.md

@@ -15,7 +15,6 @@ save one of these as `.epilogue.html` inside a folder to customize it:
 point `--js-browser` to one of these by URL:
 
 * [`minimal-up2k.js`](minimal-up2k.js) is similar to the above `minimal-up2k.html` except it applies globally to all write-only folders
-* [`quickmove.js`](quickmove.js) adds a hotkey to move selected files into a subfolder
 * [`up2k-hooks.js`](up2k-hooks.js) lets you specify a ruleset for files to skip uploading
   * [`up2k-hook-ytid.js`](up2k-hook-ytid.js) is a more specific example checking youtube-IDs against some API
 

contrib/plugins/graft-thumbs.js

@@ -1,117 +0,0 @@
-// USAGE:
-//   place this file somewhere in the webroot and then
-//   python3 -m copyparty --js-browser /.res/graft-thumbs.js
-//
-// DESCRIPTION:
-//   this is a gridview plugin which, for each file in a folder,
-//   looks for another file with the same filename (but with a
-//   different file extension)
-//
-//   if one of those files is an image and the other is not,
-//   then this plugin assumes the image is a "sidecar thumbnail"
-//   for the other file, and it will graft the image thumbnail
-//   onto the non-image file (for example an mp3)
-//
-//   optional feature 1, default-enabled:
-//   the image-file is then hidden from the directory listing
-//
-//   optional feature 2, default-enabled:
-//   when clicking the audio file, the image will also open
-
-
-(function() {
-
-	// `graft_thumbs` assumes the gridview has just been rendered;
-	// it looks for sidecars, and transplants those thumbnails onto
-	// the other file with the same basename (filename sans extension)
-
-	var graft_thumbs = function () {
-		if (!thegrid.en)
-			return;  // not in grid mode
-
-		var files = msel.getall(),
-			pairs = {};
-
-		console.log(files);
-
-		for (var a = 0; a < files.length; a++) {
-			var file = files[a],
-				is_pic = /\.(jpe?g|png|gif|webp)$/i.exec(file.vp),
-				is_audio = re_au_all.exec(file.vp),
-				basename = file.vp.replace(/\.[^\.]+$/, ""),
-				entry = pairs[basename];
-
-			if (!entry)
-				// first time seeing this basename; create a new entry in pairs
-				entry = pairs[basename] = {};
-
-			if (is_pic)
-				entry.thumb = file;
-			else if (is_audio)
-				entry.audio = file;
-		}
-
-		var basenames = Object.keys(pairs);
-		for (var a = 0; a < basenames.length; a++)
-			(function(a) {
-				var pair = pairs[basenames[a]];
-
-				if (!pair.thumb || !pair.audio)
-					return;  // not a matching pair of files
-
-				var img_thumb = QS('#ggrid a[ref="' + pair.thumb.id + '"] img[onload]'),
-					img_audio = QS('#ggrid a[ref="' + pair.audio.id + '"] img[onload]');
-
-				if (!img_thumb || !img_audio)
-					return;  // something's wrong... let's bail
-
-				// alright, graft the thumb...
-				img_audio.src = img_thumb.src;
-
-				// ...and hide the sidecar
-				img_thumb.closest('a').style.display = 'none';
-
-				// ...and add another onclick-handler to the audio,
-				// so it also opens the pic while playing the song
-				img_audio.addEventListener('click', function() {
-					img_thumb.click();
-					return false;  // let it bubble to the next listener
-				});
-
-			})(a);
-	};
-
-	// ...and then the trick! near the end of loadgrid,
-	// thegrid.bagit is called to initialize the baguettebox
-	// (image/video gallery); this is the perfect function to
-	// "hook" (hijack) so we can run our code :^)
-
-	// need to grab a backup of the original function first,
-	var orig_func = thegrid.bagit;
-
-	// and then replace it with our own:
-	thegrid.bagit = function (isrc) {
-
-		if (isrc !== '#ggrid')
-			// we only want to modify the grid, so
-			// let the original function handle this one
-			return orig_func(isrc);
-
-		graft_thumbs();
-
-		// when changing directories, the grid is
-		// rendered before msel returns the correct
-		// filenames, so schedule another run:
-		setTimeout(graft_thumbs, 1);
-
-		// and finally, call the original thegrid.bagit function
-		return orig_func(isrc);
-	};
-
-	if (ls0) {
-		// the server included an initial listing json (ls0),
-		// so the grid has already been rendered without our hook
-		graft_thumbs();
-	}
-
-})();

contrib/plugins/minimal-up2k.js

@@ -12,23 +12,6 @@ almost the same as minimal-up2k.html except this one...:
 
  -- looks slightly better
 
-
-========================
-== USAGE INSTRUCTIONS ==
-
-1. create a volume which anyone can read from (if you haven't already)
-2. copy this file into that volume, so anyone can download it
-3. enable the plugin by telling the webbrowser to load this file;
-    assuming the URL to the public volume is /res/, and
-    assuming you're using config-files, then add this to your config:
-
-    [global]
-      js-browser: /res/minimal-up2k.js
-
-alternatively, if you're not using config-files, then
-add the following commandline argument instead:
-  --js-browser=/res/minimal-up2k.js
-
 */
 
 var u2min = `

contrib/plugins/quickmove.js

@@ -1,140 +0,0 @@
-"use strict";
-
-
-// USAGE:
-//   place this file somewhere in the webroot, 
-//   for example in a folder named ".res" to hide it, and then
-//   python3 copyparty-sfx.py -v .::A --js-browser /.res/quickmove.js
-//
-// DESCRIPTION:
-//   the command above launches copyparty with one single volume;
-//   ".::A" = current folder as webroot, and everyone has Admin
-//
-//   the plugin adds hotkey "W" which moves all selected files
-//   into a subfolder named "foobar" inside the current folder
-
-
-(function() {
-
-    var action_to_perform = ask_for_confirmation_and_then_move;
-    // this decides what the new hotkey should do;
-    //  ask_for_confirmation_and_then_move  =  show a yes/no box,
-    //  move_selected_files  =  just move the files immediately
-
-    var move_destination = "foobar";
-    // this is the target folder to move files to;
-    // by default it is a subfolder of the current folder,
-    // but it can also be an absolute path like "/foo/bar"
-
-    // ===
-    // ===   END OF CONFIG
-    // ===
-
-    var main_hotkey_handler,  // copyparty's original hotkey handler
-        plugin_enabler,  // timer to engage this plugin when safe
-        files_to_move;  // list of files to move
-
-    function ask_for_confirmation_and_then_move() {
-        var num_files = msel.getsel().length,
-            msg = "move the selected " + num_files + " files?";
-
-        if (!num_files)
-            return toast.warn(2, 'no files were selected to be moved');
-
-        modal.confirm(msg, move_selected_files, null);
-    }
-
-    function move_selected_files() {
-        var selection = msel.getsel();
-
-        if (!selection.length)
-            return toast.warn(2, 'no files were selected to be moved');
-
-        if (thegrid.bbox) {
-            // close image/video viewer
-            thegrid.bbox = null;
-            baguetteBox.destroy();
-        }
-
-        files_to_move = [];
-        for (var a = 0; a < selection.length; a++)
-            files_to_move.push(selection[a].vp);
-
-        move_next_file();
-    }
-
-    function move_next_file() {
-        var num_files = files_to_move.length,
-            filepath = files_to_move.pop(),
-            filename = vsplit(filepath)[1];
-
-        toast.inf(10, "moving " + num_files + " files...\n\n" + filename);
-
-        var dst = move_destination;
-
-        if (!dst.endsWith('/'))
-            // must have a trailing slash, so add it
-            dst += '/';
-
-        if (!dst.startsWith('/'))
-            // destination is a relative path, so prefix current folder path
-            dst = get_evpath() + dst;
-
-        // and finally append the filename
-        dst += '/' + filename;
-
-        // prepare the move-request to be sent
-        var xhr = new XHR();
-        xhr.onload = xhr.onerror = function() {
-            if (this.status !== 201)
-                return toast.err(30, 'move failed: ' + esc(this.responseText));
-
-            if (files_to_move.length)
-                return move_next_file();  // still more files to go
-
-            toast.ok(1, 'move OK');
-            treectl.goto(); // reload the folder contents
-        };
-        xhr.open('POST', filepath + '?move=' + dst);
-        xhr.send();
-    }
-
-    function our_hotkey_handler(e) {
-        // bail if either ALT, CTRL, or SHIFT is pressed
-        if (e.altKey || e.shiftKey || e.isComposing || ctrl(e))
-            return main_hotkey_handler(e);  // let copyparty handle this keystroke
-
-        var key_name = (e.code || e.key) + '',
-    		ae = document.activeElement,
-		    aet = ae && ae != document.body ? ae.nodeName.toLowerCase() : '';
-
-        // check the current aet (active element type),
-        // only continue if one of the following currently has input focus:
-        //   nothing | link | button | table-row | table-cell | div | text
-        if (aet && !/^(a|button|tr|td|div|pre)$/.test(aet))
-            return main_hotkey_handler(e);  // let copyparty handle this keystroke
-
-        if (key_name == 'KeyW') {
-            // okay, this one's for us... do the thing
-            action_to_perform();
-            return ev(e);
-        }
-
-        return main_hotkey_handler(e);  // let copyparty handle this keystroke
-    }
-
-    function enable_plugin() {
-        if (!window.hotkeys_attached)
-            return console.log('quickmove is waiting for the page to finish loading');
-
-        clearInterval(plugin_enabler);
-        main_hotkey_handler = document.onkeydown;
-        document.onkeydown = our_hotkey_handler;
-        console.log('quickmove is now enabled');
-    }
-
-    // copyparty doesn't enable its hotkeys until the page
-    // has finished loading, so we'll wait for that too
-    plugin_enabler = setInterval(enable_plugin, 100);
-
-})();

contrib/systemd/copyparty-user.service

@@ -1,26 +0,0 @@
-# this will start `/usr/bin/copyparty`
-# and read config from `$HOME/.config/copyparty.conf`
-#
-# unless you add -q to disable logging, you may want to remove the
-#   following line to allow buffering (slightly better performance):
-#   Environment=PYTHONUNBUFFERED=x
-
-[Unit]
-Description=copyparty file server
-
-[Service]
-Type=notify
-SyslogIdentifier=copyparty
-WorkingDirectory=/var/lib/copyparty-jail
-Environment=PYTHONUNBUFFERED=x
-Environment=PRTY_CONFIG=%h/.config/copyparty/copyparty.conf
-ExecReload=/bin/kill -s USR1 $MAINPID
-
-# ensure there is a config
-ExecStartPre=/bin/bash -c 'if [[ ! -f %h/.config/copyparty/copyparty.conf ]]; then mkdir -p %h/.config/copyparty; cp /etc/copyparty/copyparty.conf %h/.config/copyparty/copyparty.conf; fi'
-
-# run copyparty
-ExecStart=/usr/bin/python3 /usr/bin/copyparty
-
-[Install]
-WantedBy=default.target

contrib/systemd/copyparty.conf

@@ -1,13 +1,42 @@
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
+
+# put this file in /etc/
+
+
 [global]
-  i: 127.0.0.1
+  e2dsa  # enable file indexing and filesystem scanning
+  e2ts   # and enable multimedia indexing
+  ansi   # and colors in log messages
+
+  # disable logging to stdout/journalctl and log to a file instead;
+  # $LOGS_DIRECTORY is usually /var/log/copyparty (comes from systemd)
+  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
+  # full path will be something like /var/log/copyparty/2023-1130.txt
+  # (note: enable compression by adding .xz at the end)
+  q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
+
+  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
+  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
+  # ftp: 3921        # enable ftp server on port 3921
+  # p: 3939          # listen on another port
+  # df: 16           # stop accepting uploads if less than 16 GB free disk space
+  # ver              # show copyparty version in the controlpanel
+  # grid             # show thumbnails/grid-view by default
+  # theme: 2         # monokai
+  # name: datasaver  # change the server-name that's displayed in the browser
+  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
+  # no-robots, force-js  # make it harder for search engines to read your server
+
 
 [accounts]
-  user: password
+  ed: wark  # username: password
 
-[/]
-  /var/lib/copyparty-jail
+
+[/]            # create a volume at "/" (the webroot), which will
+  /mnt         # share the contents of the "/mnt" folder
   accs:
-    r: *
-    rwdma: user
-  flags:
-    grid
+    rw: *      # everyone gets read-write access, but
+    rwmda: ed  # the user "ed" gets read-write-move-delete-admin

contrib/systemd/copyparty.example.conf

@@ -1,42 +0,0 @@
-# not actually YAML but lets pretend:
-# -*- mode: yaml -*-
-# vim: ft=yaml:
-
-
-# put this file in /etc/
-
-
-[global]
-  e2dsa  # enable file indexing and filesystem scanning
-  e2ts   # and enable multimedia indexing
-  ansi   # and colors in log messages
-
-  # disable logging to stdout/journalctl and log to a file instead;
-  # $LOGS_DIRECTORY is usually /var/log/copyparty (comes from systemd)
-  # and copyparty replaces %Y-%m%d with Year-MonthDay, so the
-  # full path will be something like /var/log/copyparty/2023-1130.txt
-  # (note: enable compression by adding .xz at the end)
-  q, lo: $LOGS_DIRECTORY/%Y-%m%d.log
-
-  # p: 80,443,3923   # listen on 80/443 as well (requires CAP_NET_BIND_SERVICE)
-  # i: 127.0.0.1     # only allow connections from localhost (reverse-proxies)
-  # ftp: 3921        # enable ftp server on port 3921
-  # p: 3939          # listen on another port
-  # df: 16           # stop accepting uploads if less than 16 GB free disk space
-  # ver              # show copyparty version in the controlpanel
-  # grid             # show thumbnails/grid-view by default
-  # theme: 2         # monokai
-  # name: datasaver  # change the server-name that's displayed in the browser
-  # stats, nos-dup   # enable the prometheus endpoint, but disable the dupes counter (too slow)
-  # no-robots, force-js  # make it harder for search engines to read your server
-
-
-[accounts]
-  ed: wark  # username: password
-
-
-[/]            # create a volume at "/" (the webroot), which will
-  /mnt         # share the contents of the "/mnt" folder
-  accs:
-    rw: *      # everyone gets read-write access, but
-    rwmda: ed  # the user "ed" gets read-write-move-delete-admin

contrib/systemd/copyparty@.service

@@ -1,30 +0,0 @@
-# this will start `/usr/bin/copyparty`
-# and read config from `/etc/copyparty/copyparty.conf`
-#
-# the %i refers to whatever you put after the copyparty@
-#   so with copyparty@foo.service, %i == foo
-#
-# unless you add -q to disable logging, you may want to remove the
-#   following line to allow buffering (slightly better performance):
-#   Environment=PYTHONUNBUFFERED=x
-
-[Unit]
-Description=copyparty file server
-
-[Service]
-Type=notify
-SyslogIdentifier=copyparty
-WorkingDirectory=/var/lib/copyparty-jail
-Environment=PYTHONUNBUFFERED=x
-Environment=PRTY_CONFIG=/etc/copyparty/copyparty.conf
-ExecReload=/bin/kill -s USR1 $MAINPID
-
-# user to run as + where the TLS certificate is (if any)
-User=%i
-Environment=XDG_CONFIG_HOME=/home/%i/.config
-
-# run copyparty
-ExecStart=/usr/bin/python3 /usr/bin/copyparty
-
-[Install]
-WantedBy=multi-user.target

contrib/systemd/index.md

@@ -1,10 +0,0 @@
-this is `/var/lib/copyparty-jail`, the fallback webroot when copyparty has not yet been configured
-
-please edit `/etc/copyparty/copyparty.conf` (if running as a system service)
-or `$HOME/.config/copyparty/copyparty.conf` if running as a user service
-
-a basic configuration example is available at https://github.com/9001/copyparty/blob/hovudstraum/contrib/systemd/copyparty.example.conf
-a configuration example that explains most flags is available at https://github.com/9001/copyparty/blob/hovudstraum/docs/chungus.conf
-
-the full list of configuration options can be seen at https://ocv.me/copyparty/helptext.html 
-or by running `copyparty --help`

contrib/systemd/prisonparty@.service

@@ -1,38 +0,0 @@
-# this will start `/usr/bin/copyparty`
-# in a chroot, preventing accidental access elsewhere,
-# and read copyparty config from `/etc/copyparty/copyparty.conf`
-#
-# expose additional filesystem locations to copyparty
-#   by listing them between the last `%i` and `--`
-#
-# `%i %i` = user/group to run copyparty as; can be IDs (1000 1000)
-#   the %i refers to whatever you put after the prisonparty@
-#   so with prisonparty@foo.service, %i == foo
-#
-# unless you add -q to disable logging, you may want to remove the
-#   following line to allow buffering (slightly better performance):
-#   Environment=PYTHONUNBUFFERED=x
-
-[Unit]
-Description=copyparty file server
-
-[Service]
-Type=notify
-SyslogIdentifier=prisonparty
-WorkingDirectory=/var/lib/copyparty-jail
-Environment=PYTHONUNBUFFERED=x
-Environment=PRTY_CONFIG=/etc/copyparty/copyparty.conf
-ExecReload=/bin/kill -s USR1 $MAINPID
-
-# user to run as + where the TLS certificate is (if any)
-User=%i
-Environment=XDG_CONFIG_HOME=/home/%i/.config
-
-# run copyparty
-ExecStart=/bin/bash /usr/bin/prisonparty /var/lib/copyparty-jail %i %i \
-  /etc/copyparty \
-  -- \
-  /usr/bin/python3 /usr/bin/copyparty
-
-[Install]
-WantedBy=multi-user.target

contrib/traefik/copyparty.yaml

@@ -1,25 +0,0 @@
-# ./traefik --configFile=copyparty.yaml
-
-entryPoints:
-  web:
-    address: :8080
-    transport:
-      # don't disconnect during big uploads
-      respondingTimeouts:
-        readTimeout: "0s"
-log:
-  level: DEBUG
-providers:
-  file:
-    # WARNING: must be same filename as current file
-    filename: "copyparty.yaml"
-http:
-  services:
-    service-cpp:
-      loadBalancer:
-        servers:
-          - url: "http://127.0.0.1:3923/"
-  routers:
-    my-router:
-      rule: "PathPrefix(`/`)"
-      service: service-cpp

contrib/zfs-tune.py

@@ -1,107 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sqlite3
-import sys
-import traceback
-
-
-"""
-when the up2k-database is stored on a zfs volume, this may give
-slightly higher performance (actual gains not measured yet)
-
-NOTE: must be applied in combination with the related advice in the openzfs documentation;
-https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#database-workloads
-and see specifically the SQLite subsection
-
-it is assumed that all databases are stored in a single location,
-for example with `--hist /var/store/hists`
-
-three alternatives for running this script:
-
-1. copy it into /var/store/hists and run "python3 zfs-tune.py s"
-    (s = modify all databases below folder containing script)
-
-2. cd into /var/store/hists and run "python3 ~/zfs-tune.py w"
-    (w = modify all databases below current working directory)
-
-3. python3 ~/zfs-tune.py /var/store/hists
-
-if you use docker, run copyparty with `--hist /cfg/hists`, copy this script into /cfg, and run this:
-podman run --rm -it --entrypoint /usr/bin/python3 ghcr.io/9001/copyparty-ac /cfg/zfs-tune.py s
-
-"""
-
-
-PAGESIZE = 65536
-
-
-# borrowed from copyparty; short efficient stacktrace for errors
-def min_ex(max_lines: int = 8, reverse: bool = False) -> str:
-    et, ev, tb = sys.exc_info()
-    stb = traceback.extract_tb(tb) if tb else traceback.extract_stack()[:-1]
-    fmt = "%s:%d <%s>: %s"
-    ex = [fmt % (fp.split(os.sep)[-1], ln, fun, txt) for fp, ln, fun, txt in stb]
-    if et or ev or tb:
-        ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
-    return "\n".join(ex[-max_lines:][:: -1 if reverse else 1])
-
-
-def set_pagesize(db_path):
-    try:
-        # check current page_size
-        with sqlite3.connect(db_path) as db:
-            v = db.execute("pragma page_size").fetchone()[0]
-            if v == PAGESIZE:
-                print(" `-- OK")
-                return
-
-        # https://www.sqlite.org/pragma.html#pragma_page_size
-        #  `- disable wal; set pagesize; vacuum
-        #      (copyparty will reenable wal if necessary)
-
-        with sqlite3.connect(db_path) as db:
-            db.execute("pragma journal_mode=delete")
-            db.commit()
-
-        with sqlite3.connect(db_path) as db:
-            db.execute(f"pragma page_size = {PAGESIZE}")
-            db.execute("vacuum")
-
-        print(" `-- new pagesize OK")
-
-    except Exception:
-        err = min_ex().replace("\n", "\n -- ")
-        print(f"FAILED: {db_path}\n -- {err}")
-
-
-def main():
-    top = os.path.dirname(os.path.abspath(__file__))
-    cwd = os.path.abspath(os.getcwd())
-    try:
-        x = sys.argv[1]
-    except:
-        print(f"""
-this script takes one mandatory argument:
-specify 's' to start recursing from folder containing this script file ({top})
-specify 'w' to start recursing from the current working directory ({cwd})
-specify a path to start recursing from there
-""")
-        sys.exit(1)
-
-    if x.lower() == "w":
-        top = cwd
-    elif x.lower() != "s":
-        top = x
-
-    for dirpath, dirs, files in os.walk(top):
-        for fname in files:
-            if not fname.endswith(".db"):
-                continue
-            db_path = os.path.join(dirpath, fname)
-            print(db_path)
-            set_pagesize(db_path)
-
-
-if __name__ == "__main__":
-    main()

copyparty/__init__.py

@@ -63,6 +63,10 @@ web/browser.js
 web/browser2.html
 web/cf.html
 web/copyparty.gif
+web/dd/2.png
+web/dd/3.png
+web/dd/4.png
+web/dd/5.png
 web/deps/busy.mp3
 web/deps/easymde.css
 web/deps/easymde.js
@@ -76,7 +80,6 @@ web/deps/prismd.css
 web/deps/scp.woff2
 web/deps/sha512.ac.js
 web/deps/sha512.hw.js
-web/idp.html
 web/iiam.gif
 web/md.css
 web/md.html

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 19, 4)
-CODENAME = "usernames"
-BUILD_DT = (2025, 8, 17)
+VERSION = (1, 16, 6)
+CODENAME = "COPYparty"
+BUILD_DT = (2024, 12, 19)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/bos/bos.py

@@ -9,11 +9,8 @@ from . import path as path
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Optional
 
-MKD_755 = {"chmod_d": 0o755}
-MKD_700 = {"chmod_d": 0o700}
-
-_ = (path, MKD_755, MKD_700)
-__all__ = ["path", "MKD_755", "MKD_700"]
+_ = (path,)
+__all__ = ["path"]
 
 # grep -hRiE '(^|[^a-zA-Z_\.-])os\.' . | gsed -r 's/ /\n/g;s/\(/(\n/g' | grep -hRiE '(^|[^a-zA-Z_\.-])os\.' | sort | uniq -c
 # printf 'os\.(%s)' "$(grep ^def bos/__init__.py | gsed -r 's/^def //;s/\(.*//' | tr '\n' '|' | gsed -r 's/.$//')"
@@ -23,39 +20,19 @@ def chmod(p: str, mode: int) -> None:
     return os.chmod(fsenc(p), mode)
 
 
-def chown(p: str, uid: int, gid: int) -> None:
-    return os.chown(fsenc(p), uid, gid)
-
-
 def listdir(p: str = ".") -> list[str]:
     return [fsdec(x) for x in os.listdir(fsenc(p))]
 
 
-def makedirs(name: str, vf: dict[str, Any] = MKD_755, exist_ok: bool = True) -> bool:
-    # os.makedirs does 777 for all but leaf; this does mode on all
-    todo = []
+def makedirs(name: str, mode: int = 0o755, exist_ok: bool = True) -> bool:
     bname = fsenc(name)
-    while bname:
-        if os.path.isdir(bname):
-            break
-        todo.append(bname)
-        bname = os.path.dirname(bname)
-    if not todo:
-        if not exist_ok:
-            os.mkdir(bname)  # to throw
-        return False
-    mode = vf["chmod_d"]
-    chown = "chown" in vf
-    for zb in todo[::-1]:
-        try:
-            os.mkdir(zb, mode)
-            if chown:
-                os.chown(zb, vf["uid"], vf["gid"])
-        except:
-            if os.path.isdir(zb):
-                continue
+    try:
+        os.makedirs(bname, mode)
+        return True
+    except:
+        if not exist_ok or not os.path.isdir(bname):
             raise
-    return True
+        return False
 
 
 def mkdir(p: str, mode: int = 0o755) -> None:

copyparty/cert.py

@@ -1,11 +1,13 @@
 import calendar
 import errno
+import filecmp
 import json
 import os
+import shutil
 import time
 
 from .__init__ import ANYWIN
-from .util import Netdev, atomic_move, load_resource, runcmd, wunlink
+from .util import Netdev, load_resource, runcmd, wrename, wunlink
 
 HAVE_CFSSL = not os.environ.get("PRTY_NO_CFSSL")
 
@@ -120,7 +122,7 @@ def _gen_ca(log: "RootLogger", args):
         wunlink(nlog, bname + ".key", VF)
     except:
         pass
-    atomic_move(nlog, bname + "-key.pem", bname + ".key", VF)
+    wrename(nlog, bname + "-key.pem", bname + ".key", VF)
     wunlink(nlog, bname + ".csr", VF)
 
     log("cert", "new ca OK", 2)
@@ -213,7 +215,7 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
         wunlink(nlog, bname + ".key", VF)
     except:
         pass
-    atomic_move(nlog, bname + "-key.pem", bname + ".key", VF)
+    wrename(nlog, bname + "-key.pem", bname + ".key", VF)
     wunlink(nlog, bname + ".csr", VF)
 
     with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:

copyparty/cfg.py

@@ -5,9 +5,6 @@ from __future__ import print_function, unicode_literals
 zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
 onedash = set(zs.split())
 
-# verify that all volflags are documented here:
-# grep volflag= __main__.py | sed -r 's/.*volflag=//;s/\).*//' | sort | uniq | while IFS= read -r x; do grep -E "\"$x(=[^ \"]+)?\": \"" cfg.py || printf '%s\n' "$x"; done
-
 
 def vf_bmap() -> dict[str, str]:
     """argv-to-volflag: simple bools"""
@@ -22,7 +19,6 @@ def vf_bmap() -> dict[str, str]:
         "no_forget": "noforget",
         "no_pipe": "nopipe",
         "no_robots": "norobots",
-        "no_tail": "notail",
         "no_thumb": "dthumb",
         "no_vthumb": "dvthumb",
         "no_athumb": "dathumb",
@@ -44,7 +40,6 @@ def vf_bmap() -> dict[str, str]:
         "gsel",
         "hardlink",
         "magic",
-        "no_db_ip",
         "no_sb_md",
         "no_sb_lg",
         "nsort",
@@ -52,14 +47,10 @@ def vf_bmap() -> dict[str, str]:
         "og_no_head",
         "og_s_title",
         "rand",
-        "reflink",
-        "rmagic",
         "rss",
-        "wo_up_readme",
         "xdev",
         "xlink",
         "xvol",
-        "zipmaxu",
     ):
         ret[k] = k
     return ret
@@ -68,7 +59,6 @@ def vf_bmap() -> dict[str, str]:
 def vf_vmap() -> dict[str, str]:
     """argv-to-volflag: simple values"""
     ret = {
-        "ac_convt": "aconvt",
         "no_hash": "nohash",
         "no_idx": "noidx",
         "re_maxage": "scan",
@@ -79,20 +69,12 @@ def vf_vmap() -> dict[str, str]:
         "th_x3": "th3x",
     }
     for k in (
-        "bup_ck",
-        "chmod_d",
-        "chmod_f",
         "dbd",
-        "forget_ip",
         "hsortn",
         "html_head",
         "lg_sbf",
         "md_sbf",
-        "lg_sba",
-        "md_sba",
-        "md_hist",
         "nrand",
-        "u2ow",
         "og_desc",
         "og_site",
         "og_th",
@@ -102,29 +84,13 @@ def vf_vmap() -> dict[str, str]:
         "og_title_i",
         "og_tpl",
         "og_ua",
-        "put_ck",
-        "put_name",
         "mv_retry",
         "rm_retry",
         "sort",
-        "tail_fd",
-        "tail_rate",
-        "tail_tmax",
-        "tail_who",
         "tcolor",
-        "th_spec_p",
-        "txt_eol",
         "unlist",
         "u2abort",
         "u2ts",
-        "uid",
-        "gid",
-        "unp_who",
-        "ups_who",
-        "zip_who",
-        "zipmaxn",
-        "zipmaxs",
-        "zipmaxt",
     ):
         ret[k] = k
     return ret
@@ -136,7 +102,6 @@ def vf_cmap() -> dict[str, str]:
     for k in (
         "exp_lg",
         "exp_md",
-        "ext_th",
         "mte",
         "mth",
         "mtp",
@@ -175,24 +140,14 @@ flagcats = {
         "dedup": "enable symlink-based file deduplication",
         "hardlink": "enable hardlink-based file deduplication,\nwith fallback on symlinks when that is impossible",
         "hardlinkonly": "dedup with hardlink only, never symlink;\nmake a full copy if hardlink is impossible",
-        "reflink": "enable reflink-based file deduplication,\nwith fallback on full copy when that is impossible",
         "safededup": "verify on-disk data before using it for dedup",
         "noclone": "take dupe data from clients, even if available on HDD",
         "nodupe": "rejects existing files (instead of linking/cloning them)",
-        "chmod_d=755": "unix-permission for new dirs/folders",
-        "chmod_f=644": "unix-permission for new files",
-        "uid=573": "change owner of new files/folders to unix-user 573",
-        "gid=999": "change owner of new files/folders to unix-group 999",
         "sparse": "force use of sparse files, mainly for s3-backed storage",
-        "nosparse": "deny use of sparse files, mainly for slow storage",
         "daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
         "nosub": "forces all uploads into the top folder of the vfs",
         "magic": "enables filetype detection for nameless uploads",
-        "put_name": "fallback filename for nameless uploads",
-        "put_ck": "default checksum-hasher for PUT/WebDAV uploads",
-        "bup_ck": "default checksum-hasher for bup/basic uploads",
-        "gz": "allows server-side gzip compression of uploads with ?gz",
-        "xz": "allows server-side lzma compression of uploads with ?xz",
+        "gz": "allows server-side gzip of uploads with ?gz (also c,xz)",
         "pk": "forces server-side compression, optional arg: xz,9",
     },
     "upload rules": {
@@ -201,10 +156,8 @@ flagcats = {
         "vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
         "vmaxn=4k": "max 4096 files in volume (suffixes: b, k, m, g, t)",
         "medialinks": "return medialinks for non-up2k uploads (not hotlinks)",
-        "wo_up_readme": "write-only users can upload logues without getting renamed",
         "rand": "force randomized filenames, 9 chars long by default",
         "nrand=N": "randomized filenames are N chars long",
-        "u2ow=N": "overwrite existing files? 0=no 1=if-older 2=always",
         "u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
         "u2abort=1": "allow aborting unfinished uploads? 0=no 1=strict 2=ip-chk 3=acct-chk",
         "sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
@@ -221,24 +174,17 @@ flagcats = {
         "e2dsa": "scans all folders for new files on startup; also sets -e2d",
         "e2t": "enable multimedia indexing; makes it possible to search for tags",
         "e2ts": "scan existing files for tags on startup; also sets -e2t",
-        "e2tsr": "delete all metadata from DB (full rescan); also sets -e2ts",
+        "e2tsa": "delete all metadata from DB (full rescan); also sets -e2ts",
         "d2ts": "disables metadata collection for existing files",
-        "e2v": "verify integrity on startup by hashing files and comparing to db",
-        "e2vu": "when e2v fails, update the db (assume on-disk files are good)",
-        "e2vp": "when e2v fails, panic and quit copyparty",
         "d2ds": "disables onboot indexing, overrides -e2ds*",
         "d2t": "disables metadata collection, overrides -e2t*",
         "d2v": "disables file verification, overrides -e2v*",
         "d2d": "disables all database stuff, overrides -e2*",
         "hist=/tmp/cdb": "puts thumbnails and indexes at that location",
-        "dbpath=/tmp/cdb": "puts indexes at that location",
-        "landmark=foo": "disable db if file foo doesn't exist",
         "scan=60": "scan for new files every 60sec, same as --re-maxage",
         "nohash=\\.iso$": "skips hashing file contents if path matches *.iso",
         "noidx=\\.iso$": "fully ignores the contents at paths matching *.iso",
         "noforget": "don't forget files when deleted from disk",
-        "forget_ip=43200": "forget uploader-IP after 30 days (GDPR)",
-        "no_db_ip": "never store uploader-IP in the db; disables unpost",
         "fat32": "avoid excessive reindexing on android sdcardfs",
         "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
         "xlink": "cross-volume dupe detection / linking (dangerous)",
@@ -249,8 +195,6 @@ flagcats = {
         "srch_excl": "exclude search results with URL matching this regex",
     },
     'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
-        "mte=artist,title": "media-tags to index/display",
-        "mth=fmt,res,ac": "media-tags to hide by default",
         "mtp=.bpm=f,audio-bpm.py": 'uses the "audio-bpm.py" program to\ngenerate ".bpm" tags from uploads (f = overwrite tags)',
         "mtp=ahash,vhash=media-hash.py": "collects two tags at once",
     },
@@ -263,10 +207,7 @@ flagcats = {
         "thsize": "thumbnail res; WxH",
         "crop": "center-cropping (y/n/fy/fn)",
         "th3x": "3x resolution (y/n/fy/fn)",
-        "convt": "convert-to-image timeout in seconds",
-        "aconvt": "convert-to-audio timeout in seconds",
-        "th_spec_p=1": "make spectrograms? 0=never 1=fallback 2=always",
-        "ext_th=s=/b.png": "use /b.png as thumbnail for file-extension s",
+        "convt": "conversion timeout in seconds",
     },
     "handlers\n(better explained in --help-handlers)": {
         "on404=PY": "handle 404s by executing PY file",
@@ -289,71 +230,22 @@ flagcats = {
         "grid": "show grid/thumbnails by default",
         "gsel": "select files in grid by ctrl-click",
         "sort": "default sort order",
-        "nsort": "natural-sort of leading digits in filenames",
-        "hsortn": "number of sort-rules to add to media URLs",
         "unlist": "dont list files matching REGEX",
         "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
-        "tcolor=#fc0": "theme color (a hint for webbrowsers, discord, etc.)",
-        "nodirsz": "don't show total folder size",
         "robots": "allows indexing by search engines (default)",
         "norobots": "kindly asks search engines to leave",
-        "unlistcr": "don't list read-access in controlpanel",
-        "unlistcw": "don't list write-access in controlpanel",
         "no_sb_md": "disable js sandbox for markdown files",
         "no_sb_lg": "disable js sandbox for prologue/epilogue",
         "sb_md": "enable js sandbox for markdown files (default)",
         "sb_lg": "enable js sandbox for prologue/epilogue (default)",
         "md_sbf": "list of markdown-sandbox safeguards to disable",
         "lg_sbf": "list of *logue-sandbox safeguards to disable",
-        "md_sba": "value of iframe allow-prop for markdown-sandbox",
-        "lg_sba": "value of iframe allow-prop for *logue-sandbox",
         "nohtml": "return html and markdown as text/html",
     },
-    "opengraph (discord embeds)": {
-        "og": "enable OG (disables hotlinking)",
-        "og_site": "sitename; defaults to --name, disable with '-'",
-        "og_desc": "description text for all files; disable with '-'",
-        "og_th=jf": "thumbnail format; j / jf / jf3 / w / w3 / ...",
-        "og_title_a": "audio title format; default: {{ artist }} - {{ title }}",
-        "og_title_v": "video title format; default: {{ title }}",
-        "og_title_i": "image title format; default: {{ title }}",
-        "og_title=foo": "fallback title if there's nothing in the db",
-        "og_s_title": "force default title; do not read from tags",
-        "og_tpl": "custom html; see --og-tpl in --help",
-        "og_no_head": "you want to add tags manually with og_tpl",
-        "og_ua": "if defined: only send OG html if useragent matches this regex",
-    },
-    "textfiles": {
-        "md_hist": "where to put markdown backups; s=subfolder, v=volHist, n=nope",
-        "exp": "enable textfile expansion; see --help-exp",
-        "exp_md": "placeholders to expand in markdown files; see --help",
-        "exp_lg": "placeholders to expand in prologue/epilogue; see --help",
-        "txt_eol=lf": "enable EOL conversion when writing docs (LF or CRLF)",
-    },
-    "tailing": {
-        "notail": "disable ?tail (download a growing file continuously)",
-        "tail_fd=1": "check if file was replaced (new fd) every 1 sec",
-        "tail_rate=0.2": "check for new data every 0.2 sec",
-        "tail_tmax=30": "kill connection after 30 sec",
-        "tail_who=2": "restrict ?tail access (1=admins,2=authed,3=everyone)",
-    },
     "others": {
         "dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
         "fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
         "fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
-        "dk=8": 'generates per-directory accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
-        "dks": "per-directory accesskeys allow browsing into subdirs",
-        "dky": 'allow seeing files (not folders) inside a specific folder\nwith "g" perm, and does not require a valid dirkey to do so',
-        "rss": "allow '?rss' URL suffix (experimental)",
-        "rmagic": "expensive analysis for mimetype accuracy",
-        "unp_who=2": "unpost only if same... 1=ip+name, 2=ip, 3=name",
-        "ups_who=2": "restrict viewing the list of recent uploads",
-        "zip_who=2": "restrict access to download-as-zip/tar",
-        "zipmaxn=9k": "reject download-as-zip if more than 9000 files",
-        "zipmaxs=2g": "reject download-as-zip if size over 2 GiB",
-        "zipmaxt=no": "reply with 'no' if download-as-zip exceeds max",
-        "zipmaxu": "zip-size-limit does not apply to authenticated users",
-        "nopipe": "disable race-the-beam (download unfinished uploads)",
         "mv_retry": "ms-windows: timeout for renaming busy files",
         "rm_retry": "ms-windows: timeout for deleting busy files",
         "davauth": "ask webdav clients to login for all folders",
@@ -363,10 +255,3 @@ flagcats = {
 
 
 flagdescs = {k.split("=")[0]: v for tab in flagcats.values() for k, v in tab.items()}
-
-
-if True:  # so it gets removed in release-builds
-    for fun in [vf_bmap, vf_cmap, vf_vmap]:
-        for k in fun().values():
-            if k not in flagdescs:
-                raise Exception("undocumented volflag: " + k)

copyparty/dxml.py

@@ -1,6 +1,3 @@
-# coding: utf-8
-from __future__ import print_function, unicode_literals
-
 import importlib
 import sys
 import xml.etree.ElementTree as ET
@@ -11,10 +8,6 @@ if True:  # pylint: disable=using-constant-test
     from typing import Any, Optional
 
 
-class BadXML(Exception):
-    pass
-
-
 def get_ET() -> ET.XMLParser:
     pn = "xml.etree.ElementTree"
     cn = "_elementtree"
@@ -41,7 +34,7 @@ def get_ET() -> ET.XMLParser:
 XMLParser: ET.XMLParser = get_ET()
 
 
-class _DXMLParser(XMLParser):  # type: ignore
+class DXMLParser(XMLParser):  # type: ignore
     def __init__(self) -> None:
         tb = ET.TreeBuilder()
         super(DXMLParser, self).__init__(target=tb)
@@ -56,57 +49,16 @@ class _DXMLParser(XMLParser):  # type: ignore
         raise BadXML("{}, {}".format(a, ka))
 
 
-class _NG(XMLParser):  # type: ignore
-    def __int__(self) -> None:
-        raise BadXML("dxml selftest failed")
-
-
-DXMLParser = _DXMLParser
+class BadXML(Exception):
+    pass
 
 
 def parse_xml(txt: str) -> ET.Element:
-    """
-    Parse XML into an xml.etree.ElementTree.Element while defusing some unsafe parts.
-    """
     parser = DXMLParser()
     parser.feed(txt)
     return parser.close()  # type: ignore
 
 
-def selftest() -> bool:
-    qbe = r"""<!DOCTYPE d [
-<!ENTITY a "nice_bakuretsu">
-]>
-<root>&a;&a;&a;</root>"""
-
-    emb = r"""<!DOCTYPE d [
-<!ENTITY a SYSTEM "file:///etc/hostname">
-]>
-<root>&a;</root>"""
-
-    # future-proofing; there's never been any known vulns
-    # regarding DTDs and ET.XMLParser, but might as well
-    # block them since webdav-clients don't use them
-    dtd = r"""<!DOCTYPE d SYSTEM "a.dtd">
-<root>a</root>"""
-
-    for txt in (qbe, emb, dtd):
-        try:
-            parse_xml(txt)
-            t = "WARNING: dxml selftest failed:\n%s\n"
-            print(t % (txt,), file=sys.stderr)
-            return False
-        except BadXML:
-            pass
-
-    return True
-
-
-DXML_OK = selftest()
-if not DXML_OK:
-    DXMLParser = _NG
-
-
 def mktnod(name: str, text: str) -> ET.Element:
     el = ET.Element(name)
     el.text = text

copyparty/fsutil.py

@@ -78,7 +78,7 @@ class Fstab(object):
             return vid
 
     def build_fallback(self) -> None:
-        self.tab = VFS(self.log_func, "idk", "/", "/", AXS(), {})
+        self.tab = VFS(self.log_func, "idk", "/", AXS(), {})
         self.trusted = False
 
     def build_tab(self) -> None:
@@ -111,10 +111,9 @@ class Fstab(object):
 
         tab1.sort(key=lambda x: (len(x[0]), x[0]))
         path1, fs1 = tab1[0]
-        tab = VFS(self.log_func, fs1, path1, path1, AXS(), {})
+        tab = VFS(self.log_func, fs1, path1, AXS(), {})
         for path, fs in tab1[1:]:
-            zs = path.lstrip("/")
-            tab.add(fs, zs, zs)
+            tab.add(fs, path.lstrip("/"))
 
         self.tab = tab
         self.srctab = srctab
@@ -131,10 +130,9 @@ class Fstab(object):
         if not self.trusted:
             # no mtab access; have to build as we go
             if "/" in rem:
-                zs = os.path.join(vn.vpath, rem.split("/")[0])
-                self.tab.add("idk", zs, zs)
+                self.tab.add("idk", os.path.join(vn.vpath, rem.split("/")[0]))
             if rem:
-                self.tab.add(nval, path, path)
+                self.tab.add(nval, path)
             else:
                 vn.realpath = nval
 

copyparty/ftpd.py

@@ -19,7 +19,6 @@ from .__init__ import PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
-    FN_EMB,
     VF_CAREFUL,
     Daemon,
     ODict,
@@ -31,7 +30,6 @@ from .util import (
     relchk,
     runhook,
     sanitize_fn,
-    set_fperms,
     vjoin,
     wunlink,
 )
@@ -83,12 +81,7 @@ class FtpAuth(DummyAuthorizer):
         uname = "*"
         if username != "anonymous":
             uname = ""
-            if args.usernames:
-                alts = ["%s:%s" % (username, password)]
-            else:
-                alts = password, username
-
-            for zs in alts:
+            for zs in (password, username):
                 zs = asrv.iacct.get(asrv.ah.hash(zs), "")
                 if zs:
                     uname = zs
@@ -96,10 +89,6 @@ class FtpAuth(DummyAuthorizer):
 
         if args.ipu and uname == "*":
             uname = args.ipu_iu[args.ipu_nm.map(ip)]
-        if args.ipr and uname in args.ipr_u:
-            if not args.ipr_u[uname].map(ip):
-                logging.warning("username [%s] rejected by --ipr", uname)
-                uname = "*"
 
         if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
             g = self.hub.gpwd
@@ -181,16 +170,6 @@ class FtpFs(AbstractedFS):
             fn = sanitize_fn(fn or "", "")
             vpath = vjoin(rd, fn)
             vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
-            if (
-                w
-                and fn.lower() in FN_EMB
-                and self.h.uname not in vfs.axs.uread
-                and "wo_up_readme" not in vfs.flags
-            ):
-                fn = "_wo_" + fn
-                vpath = vjoin(rd, fn)
-                vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
-
             if not vfs.realpath:
                 t = "No filesystem mounted at [{}]"
                 raise FSE(t.format(vpath))
@@ -239,7 +218,7 @@ class FtpFs(AbstractedFS):
         r = "r" in mode
         w = "w" in mode or "a" in mode or "+" in mode
 
-        ap, vfs, _ = self.rv2a(filename, r, w)
+        ap = self.rv2a(filename, r, w)[0]
         self.validpath(ap)
         if w:
             try:
@@ -271,11 +250,7 @@ class FtpFs(AbstractedFS):
 
             wunlink(self.log, ap, VF_CAREFUL)
 
-        ret = open(fsenc(ap), mode, self.args.iobuf)
-        if w and "fperms" in vfs.flags:
-            set_fperms(ret, vfs.flags)
-
-        return ret
+        return open(fsenc(ap), mode, self.args.iobuf)
 
     def chdir(self, path: str) -> None:
         nwd = join(self.cwd, path)
@@ -289,12 +264,9 @@ class FtpFs(AbstractedFS):
             # returning 550 is library-default and suitable
             raise FSE("No such file or directory")
 
-        if vfs.realpath:
-            avfs = vfs.chk_ap(ap, st)
-            if not avfs:
-                raise FSE("Permission denied", 1)
-        else:
-            avfs = vfs
+        avfs = vfs.chk_ap(ap, st)
+        if not avfs:
+            raise FSE("Permission denied", 1)
 
         self.cwd = nwd
         (
@@ -309,8 +281,8 @@ class FtpFs(AbstractedFS):
         ) = avfs.can_access("", self.h.uname)
 
     def mkdir(self, path: str) -> None:
-        ap, vfs, _ = self.rv2a(path, w=True)
-        bos.makedirs(ap, vf=vfs.flags)  # filezilla expects this
+        ap = self.rv2a(path, w=True)[0]
+        bos.makedirs(ap)  # filezilla expects this
 
     def listdir(self, path: str) -> list[str]:
         vpath = join(self.cwd, path)
@@ -409,12 +381,8 @@ class FtpFs(AbstractedFS):
             return st
 
     def utime(self, path: str, timeval: float) -> None:
-        try:
-            ap = self.rv2a(path, w=True)[0]
-            return bos.utime(ap, (int(time.time()), int(timeval)))
-        except Exception as ex:
-            logging.error("ftp.utime: %s, %r", ex, ex)
-            raise
+        ap = self.rv2a(path, w=True)[0]
+        return bos.utime(ap, (timeval, timeval))
 
     def lstat(self, path: str) -> os.stat_result:
         ap = self.rv2a(path)[0]
@@ -503,11 +471,7 @@ class FtpHandler(FTPHandler):
     def ftp_STOR(self, file: str, mode: str = "w") -> Any:
         # Optional[str]
         vp = join(self.fs.cwd, file).lstrip("/")
-        try:
-            ap, vfs, rem = self.fs.v2a(vp, w=True)
-        except Exception as ex:
-            self.respond("550 %s" % (ex,), logging.info)
-            return
+        ap, vfs, rem = self.fs.v2a(vp, w=True)
         self.vfs_map[ap] = vp
         xbu = vfs.flags.get("xbu")
         if xbu and not runhook(
@@ -627,7 +591,7 @@ class Ftpd(object):
         if "::" in ips:
             ips.append("0.0.0.0")
 
-        ips = [x for x in ips if not x.startswith(("unix:", "fd:"))]
+        ips = [x for x in ips if "unix:" not in x]
 
         if self.args.ftp4:
             ips = [x for x in ips if ":" not in x]

copyparty/httpconn.py

@@ -224,6 +224,3 @@ class HttpConn(object):
             if self.u2idx:
                 self.hsrv.put_u2idx(str(self.addr), self.u2idx)
                 self.u2idx = None
-
-            if self.rproxy:
-                self.set_rproxy()

copyparty/httpsrv.py

@@ -70,7 +70,6 @@ from .util import (
     build_netmap,
     has_resource,
     ipnorm,
-    load_ipr,
     load_ipu,
     load_resource,
     min_ex,
@@ -124,7 +123,6 @@ class HttpSrv(object):
         self.nm = NetMap([], [])
         self.ssdp: Optional["SSDPr"] = None
         self.gpwd = Garda(self.args.ban_pw)
-        self.gpwc = Garda(self.args.ban_pwc)
         self.g404 = Garda(self.args.ban_404)
         self.g403 = Garda(self.args.ban_403)
         self.g422 = Garda(self.args.ban_422, False)
@@ -177,7 +175,6 @@ class HttpSrv(object):
             "browser",
             "browser2",
             "cf",
-            "idp",
             "md",
             "mde",
             "msg",
@@ -194,15 +191,14 @@ class HttpSrv(object):
         else:
             self.ipu_iu = self.ipu_nm = None
 
-        if self.args.ipr:
-            self.ipr = load_ipr(self.log, self.args.ipr)
-        else:
-            self.ipr = None
-
         self.ipa_nm = build_netmap(self.args.ipa)
         self.xff_nm = build_netmap(self.args.xff_src)
         self.xff_lan = build_netmap("lan")
 
+        self.ptn_cc = re.compile(r"[\x00-\x1f]")
+        self.ptn_hsafe = re.compile(r"[\x00-\x1f<>\"'&]")
+        self.uparam_cc_ok = set("doc move tree".split())
+
         self.mallow = "GET HEAD POST PUT DELETE OPTIONS".split()
         if not self.args.no_dav:
             zs = "PROPFIND PROPPATCH LOCK UNLOCK MKCOL COPY MOVE"
@@ -321,8 +317,6 @@ class HttpSrv(object):
 
         Daemon(self.broker.say, "sig-hsrv-up1", ("cb_httpsrv_up",))
 
-        saddr = ("", 0)  # fwd-decl for `except TypeError as ex:`
-
         while not self.stopping:
             if self.args.log_conn:
                 self.log(self.name, "|%sC-ncli" % ("-" * 1,), c="90")
@@ -330,8 +324,7 @@ class HttpSrv(object):
             spins = 0
             while self.ncli >= self.nclimax:
                 if not spins:
-                    t = "at connection limit (global-option 'nc'); waiting"
-                    self.log(self.name, t, 3)
+                    self.log(self.name, "at connection limit; waiting", 3)
 
                 spins += 1
                 time.sleep(0.1)
@@ -405,19 +398,6 @@ class HttpSrv(object):
                 self.log(self.name, "accept({}): {}".format(fno, ex), c=6)
                 time.sleep(0.02)
                 continue
-            except TypeError as ex:
-                # on macOS, accept() may return a None saddr if blocked by LittleSnitch;
-                # unicode(saddr[0]) ==> TypeError: 'NoneType' object is not subscriptable
-                if tcp and not saddr:
-                    t = "accept(%s): failed to accept connection from client due to firewall or network issue"
-                    self.log(self.name, t % (fno,), c=3)
-                    try:
-                        sck.close()  # type: ignore
-                    except:
-                        pass
-                    time.sleep(0.02)
-                    continue
-                raise
 
             if self.args.log_conn:
                 t = "|{}C-acc2 \033[0;36m{} \033[3{}m{}".format(

copyparty/ico.py

@@ -94,21 +94,10 @@ class Ico(object):
 <?xml version="1.0" encoding="UTF-8"?>
 <svg version="1.1" viewBox="0 0 100 {}" xmlns="http://www.w3.org/2000/svg"><g>
 <rect width="100%" height="100%" fill="#{}" />
-<text x="50%" y="{}" dominant-baseline="middle" text-anchor="middle" xml:space="preserve"
+<text x="50%" y="50%" dominant-baseline="middle" text-anchor="middle" xml:space="preserve"
   fill="#{}" font-family="monospace" font-size="14px" style="letter-spacing:.5px">{}</text>
 </g></svg>
 """
-
-        txt = html_escape(ext, True)
-        if "\n" in txt:
-            lines = txt.split("\n")
-            n = len(lines)
-            y = "20%" if n == 2 else "10%" if n == 3 else "0"
-            zs = '<tspan x="50%%" dy="1.2em">%s</tspan>'
-            txt = "".join([zs % (x,) for x in lines])
-        else:
-            y = "50%"
-
-        svg = svg.format(h, c[:6], y, c[6:], txt)
+        svg = svg.format(h, c[:6], c[6:], html_escape(ext, True))
 
         return "image/svg+xml", svg.encode("utf-8")

copyparty/mdns.py

@@ -76,8 +76,7 @@ class MDNS(MCast):
         if not self.args.zm_nwa_1:
             set_avahi_379()
 
-        zs = self.args.zm_fqdn or (self.args.name + ".local")
-        zs = zs.replace("--name", self.args.name).rstrip(".") + "."
+        zs = self.args.name + ".local."
         zs = zs.encode("ascii", "replace").decode("ascii", "replace")
         self.hn = "-".join(x for x in zs.split("?") if x) or (
             "vault-{}".format(random.randint(1, 255))

copyparty/metrics.py

@@ -18,7 +18,7 @@ class Metrics(object):
 
     def tx(self, cli: "HttpCli") -> bool:
         if not cli.avol:
-            raise Pebkac(403, "'stats' not allowed for user " + cli.uname)
+            raise Pebkac(403, "not allowed for user " + cli.uname)
 
         args = cli.args
         if not args.stats:

copyparty/mtag.py

@@ -18,7 +18,6 @@ from .util import (
     REKOBO_LKEY,
     VF_CAREFUL,
     fsenc,
-    gzip,
     min_ex,
     pybin,
     retchk,
@@ -29,7 +28,7 @@ from .util import (
 )
 
 if True:  # pylint: disable=using-constant-test
-    from typing import IO, Any, Optional, Union
+    from typing import Any, Optional, Union
 
     from .util import NamedLogger, RootLogger
 
@@ -67,8 +66,6 @@ HAVE_FFPROBE = not os.environ.get("PRTY_NO_FFPROBE") and have_ff("ffprobe")
 CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif".split())
 CBZ_01 = re.compile(r"(^|[^0-9v])0+[01]\b")
 
-FMT_AU = set("mp3 ogg flac wav".split())
-
 
 class MParser(object):
     def __init__(self, cmdline: str) -> None:
@@ -141,6 +138,8 @@ def au_unpk(
         fd, ret = tempfile.mkstemp("." + au)
 
         if pk == "gz":
+            import gzip
+
             fi = gzip.GzipFile(abspath, mode="rb")
 
         elif pk == "xz":
@@ -168,16 +167,12 @@ def au_unpk(
             znil = [x for x in znil if "cover" in x[0]] or znil
             znil = [x for x in znil if CBZ_01.search(x[0])] or znil
             t = "cbz: %d files, %d hits" % (nf, len(znil))
-            using = sorted(znil)[0][1].filename
             if znil:
-                t += ", using " + using
+                t += ", using " + znil[0][1].filename
             log(t)
             if not znil:
                 raise Exception("no images inside cbz")
-            fi = zf.open(using)
-
-        elif pk == "epub":
-            fi = get_cover_from_epub(log, abspath)
+            fi = zf.open(znil[0][1])
 
         else:
             raise Exception("unknown compression %s" % (pk,))
@@ -208,7 +203,7 @@ def au_unpk(
 
 def ffprobe(
     abspath: str, timeout: int = 60
-) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]], list[Any], dict[str, Any]]:
+) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]]]:
     cmd = [
         b"ffprobe",
         b"-hide_banner",
@@ -222,17 +217,8 @@ def ffprobe(
     return parse_ffprobe(so)
 
 
-def parse_ffprobe(
-    txt: str,
-) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]], list[Any], dict[str, Any]]:
-    """
-    txt: output from ffprobe -show_format -show_streams
-    returns:
-     * normalized tags
-     * original/raw tags
-     * list of streams
-     * format props
-    """
+def parse_ffprobe(txt: str) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]]]:
+    """ffprobe -show_format -show_streams"""
     streams = []
     fmt = {}
     g = {}
@@ -256,7 +242,7 @@ def parse_ffprobe(
     ret: dict[str, Any] = {}  # processed
     md: dict[str, list[Any]] = {}  # raw tags
 
-    is_audio = fmt.get("format_name") in FMT_AU
+    is_audio = fmt.get("format_name") in ["mp3", "ogg", "flac", "wav"]
     if fmt.get("filename", "").split(".")[-1].lower() in ["m4a", "aac"]:
         is_audio = True
 
@@ -284,8 +270,6 @@ def parse_ffprobe(
                 ["channel_layout", "chs"],
                 ["sample_rate", ".hz"],
                 ["bit_rate", ".aq"],
-                ["bits_per_sample", ".bps"],
-                ["bits_per_raw_sample", ".bprs"],
                 ["duration", ".dur"],
             ]
 
@@ -325,7 +309,7 @@ def parse_ffprobe(
                 ret[rk] = v1
 
     if ret.get("vc") == "ansi":  # shellscript
-        return {}, {}, [], {}
+        return {}, {}
 
     for strm in streams:
         for sk, sv in strm.items():
@@ -374,77 +358,7 @@ def parse_ffprobe(
     zero = int("0")
     zd = {k: (zero, v) for k, v in ret.items()}
 
-    return zd, md, streams, fmt
-
-
-def get_cover_from_epub(log: "NamedLogger", abspath: str) -> Optional[IO[bytes]]:
-    import zipfile
-
-    from .dxml import parse_xml
-
-    try:
-        from urlparse import urljoin  # Python2
-    except ImportError:
-        from urllib.parse import urljoin  # Python3
-
-    with zipfile.ZipFile(abspath, "r") as z:
-        # First open the container file to find the package document (.opf file)
-        try:
-            container_root = parse_xml(z.read("META-INF/container.xml").decode())
-        except KeyError:
-            log("epub: no container file found in %s" % (abspath,))
-            return None
-
-        # https://www.w3.org/TR/epub-33/#sec-container.xml-rootfile-elem
-        container_ns = {"": "urn:oasis:names:tc:opendocument:xmlns:container"}
-        # One file could contain multiple package documents, default to the first one
-        rootfile_path = container_root.find("./rootfiles/rootfile", container_ns).get(
-            "full-path"
-        )
-
-        # Then open the first package document to find the path of the cover image
-        try:
-            package_root = parse_xml(z.read(rootfile_path).decode())
-        except KeyError:
-            log("epub: no package document found in %s" % (abspath,))
-            return None
-
-        # https://www.w3.org/TR/epub-33/#sec-package-doc
-        package_ns = {"": "http://www.idpf.org/2007/opf"}
-        # https://www.w3.org/TR/epub-33/#sec-cover-image
-        coverimage_path_node = package_root.find(
-            "./manifest/item[@properties='cover-image']", package_ns
-        )
-        if coverimage_path_node is not None:
-            coverimage_path = coverimage_path_node.get("href")
-        else:
-            # This might be an EPUB2 file, try the legacy way of specifying covers
-            coverimage_path = _get_cover_from_epub2(log, package_root, package_ns)
-
-        # This url is either absolute (in the .epub) or relative to the package document
-        adjusted_cover_path = urljoin(rootfile_path, coverimage_path)
-
-        return z.open(adjusted_cover_path)
-
-
-def _get_cover_from_epub2(
-    log: "NamedLogger", package_root, package_ns
-) -> Optional[str]:
-    # <meta name="cover" content="id-to-cover-image"> in <metadata>, then
-    # <item> in <manifest>
-    cover_id = package_root.find("./metadata/meta[@name='cover']", package_ns).get(
-        "content"
-    )
-
-    if not cover_id:
-        return None
-
-    for node in package_root.iterfind("./manifest/item", package_ns):
-        if node.get("id") == cover_id:
-            cover_path = node.get("href")
-            return cover_path
-
-    return None
+    return zd, md
 
 
 class MTag(object):
@@ -715,7 +629,7 @@ class MTag(object):
         if not bos.path.isfile(abspath):
             return {}
 
-        ret, md, _, _ = ffprobe(abspath, self.args.mtag_to)
+        ret, md = ffprobe(abspath, self.args.mtag_to)
 
         if self.args.mtag_vv:
             for zd in (ret, dict(md)):

copyparty/multicast.py

@@ -163,7 +163,6 @@ class MCast(object):
             sck.settimeout(None)
             sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
             try:
-                # safe for this purpose; https://lwn.net/Articles/853637/
                 sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
             except:
                 pass
@@ -183,7 +182,11 @@ class MCast(object):
                 srv.ips[oth_ip.split("/")[0]] = ipaddress.ip_network(oth_ip, False)
 
             # gvfs breaks if a linklocal ip appears in a dns reply
-            ll = {k: v for k, v in srv.ips.items() if k.startswith(("169.254", "fe80"))}
+            ll = {
+                k: v
+                for k, v in srv.ips.items()
+                if k.startswith("169.254") or k.startswith("fe80")
+            }
             rt = {k: v for k, v in srv.ips.items() if k not in ll}
 
             if self.args.ll or not rt:

copyparty/pwhash.py

@@ -15,7 +15,7 @@ try:
         raise Exception()
 
     HAVE_ARGON2 = True
-    from argon2 import exceptions as argon2ex
+    from argon2 import __version__ as argon2ver
 except:
     HAVE_ARGON2 = False
 
@@ -147,10 +147,6 @@ class PWHash(object):
     def cli(self) -> None:
         import getpass
 
-        if self.args.usernames:
-            t = "since you have enabled --usernames, please provide username:password"
-            print(t)
-
         while True:
             try:
                 p1 = getpass.getpass("password> ")

copyparty/smbd.py

@@ -320,7 +320,7 @@ class SMB(object):
 
         self.hub.up2k.handle_mv(uname, "1.7.6.2", vp1, vp2)
         try:
-            bos.makedirs(ap2, vf=vfs2.flags)
+            bos.makedirs(ap2)
         except:
             pass
 
@@ -334,7 +334,7 @@ class SMB(object):
             t = "blocked mkdir (no-write-acc %s): /%s @%s"
             yeet(t % (vfs.axs.uwrite, vpath, uname))
 
-        return bos.mkdir(ap, vfs.flags["chmod_d"])
+        return bos.mkdir(ap)
 
     def _stat(self, vpath: str, *a: Any, **ka: Any) -> os.stat_result:
         try:

copyparty/sutil.py

@@ -17,9 +17,6 @@ if True:  # pylint: disable=using-constant-test
     from .util import NamedLogger
 
 
-TAR_NO_OPUS = set("aac|m4a|mp3|oga|ogg|opus|wma".split("|"))
-
-
 class StreamArc(object):
     def __init__(
         self,
@@ -85,7 +82,9 @@ def enthumb(
 ) -> dict[str, Any]:
     rem = f["vp"]
     ext = rem.rsplit(".", 1)[-1].lower()
-    if (fmt == "mp3" and ext == "mp3") or (fmt == "opus" and ext in TAR_NO_OPUS):
+    if (fmt == "mp3" and ext == "mp3") or (
+        fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|")
+    ):
         raise Exception()
 
     vp = vjoin(vtop, rem.split("/", 1)[1])

copyparty/svchub.py

@@ -2,8 +2,8 @@
 from __future__ import print_function, unicode_literals
 
 import argparse
-import atexit
 import errno
+import gzip
 import logging
 import os
 import re
@@ -28,7 +28,6 @@ if True:  # pylint: disable=using-constant-test
 
 from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, E, EnvParams, unicode
 from .authsrv import BAD_CFG, AuthSrv
-from .bos import bos
 from .cert import ensure_cert
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, HAVE_MUTAGEN
 from .pwhash import HAVE_ARGON2
@@ -39,7 +38,6 @@ from .th_srv import (
     HAVE_FFPROBE,
     HAVE_HEIF,
     HAVE_PIL,
-    HAVE_RAW,
     HAVE_VIPS,
     HAVE_WEBP,
     ThumbSrv,
@@ -52,9 +50,6 @@ from .util import (
     FFMPEG_URL,
     HAVE_PSUTIL,
     HAVE_SQLITE3,
-    HAVE_ZMQ,
-    RE_ANSI,
-    URL_BUG,
     UTC,
     VERSIONS,
     Daemon,
@@ -63,25 +58,18 @@ from .util import (
     HMaccas,
     ODict,
     alltrace,
+    ansi_re,
     build_netmap,
-    expat_ver,
-    gzip,
-    load_ipr,
     load_ipu,
-    lock_file,
     min_ex,
     mp,
     odfusion,
     pybin,
     start_log_thrs,
     start_stackmon,
-    termsize,
     ub64enc,
 )
 
-if HAVE_SQLITE3:
-    import sqlite3
-
 if TYPE_CHECKING:
     try:
         from .mdns import MDNS
@@ -93,11 +81,6 @@ if PY2:
     range = xrange  # type: ignore
 
 
-VER_IDP_DB = 1
-VER_SESSION_DB = 1
-VER_SHARES_DB = 2
-
-
 class SvcHub(object):
     """
     Hosts all services which cannot be parallelized due to reliance on monolithic resources.
@@ -156,7 +139,6 @@ class SvcHub(object):
             args.no_del = True
             args.no_mv = True
             args.hardlink = True
-            args.dav_auth = True
             args.vague_403 = True
             args.nih = True
 
@@ -173,7 +155,6 @@ class SvcHub(object):
         # for non-http clients (ftp, tftp)
         self.bans: dict[str, int] = {}
         self.gpwd = Garda(self.args.ban_pw)
-        self.gpwc = Garda(self.args.ban_pwc)
         self.g404 = Garda(self.args.ban_404)
         self.g403 = Garda(self.args.ban_403)
         self.g422 = Garda(self.args.ban_422, False)
@@ -202,14 +183,8 @@ class SvcHub(object):
 
         if not args.use_fpool and args.j != 1:
             args.no_fpool = True
-            t = "multithreading enabled with -j {}, so disabling fpool -- this can reduce upload performance on some filesystems, and make some antivirus-softwares "
-            c = 0
-            if ANYWIN:
-                t += "(especially Microsoft Defender) stress your CPU and HDD severely during big uploads"
-                c = 3
-            else:
-                t += "consume more resources (CPU/HDD) than normal"
-            self.log("root", t.format(args.j), c)
+            t = "multithreading enabled with -j {}, so disabling fpool -- this can reduce upload performance on some filesystems"
+            self.log("root", t.format(args.j))
 
         if not args.no_fpool and args.j != 1:
             t = "WARNING: ignoring --use-fpool because multithreading (-j{}) is enabled"
@@ -245,7 +220,7 @@ class SvcHub(object):
             t = "WARNING: --th-ram-max is very small (%.2f GiB); will not be able to %s"
             self.log("root", t % (args.th_ram_max, zs), 3)
 
-        if args.chpw and args.have_idp_hdrs:
+        if args.chpw and args.idp_h_usr:
             t = "ERROR: user-changeable passwords is incompatible with IdP/identity-providers; you must disable either --chpw or --idp-h-usr"
             self.log("root", t, 1)
             raise Exception(t)
@@ -261,24 +236,8 @@ class SvcHub(object):
             setattr(args, "ipu_iu", iu)
             setattr(args, "ipu_nm", nm)
 
-        if args.ipr:
-            ipr = load_ipr(self.log, args.ipr, True)
-            setattr(args, "ipr_u", ipr)
-
-        for zs in "ah_salt fk_salt dk_salt".split():
-            if getattr(args, "show_%s" % (zs,)):
-                self.log("root", "effective %s is %s" % (zs, getattr(args, zs)))
-
-        if args.ah_cli or args.ah_gen:
-            args.idp_store = 0
-            args.no_ses = True
-            args.shr = ""
-
-        if args.idp_store and args.have_idp_hdrs:
-            self.setup_db("idp")
-
         if not self.args.no_ses:
-            self.setup_db("ses")
+            self.setup_session_db()
 
         args.shr1 = ""
         if args.shr:
@@ -330,8 +289,6 @@ class SvcHub(object):
             decs.pop("vips", None)
         if not HAVE_PIL:
             decs.pop("pil", None)
-        if not HAVE_RAW:
-            decs.pop("raw", None)
         if not HAVE_FFMPEG or not HAVE_FFPROBE:
             decs.pop("ff", None)
 
@@ -438,95 +395,34 @@ class SvcHub(object):
                 getattr(args, zs).mutex = threading.Lock()
             except:
                 pass
-        if args.ipr:
-            for nm in args.ipr_u.values():
-                nm.mutex = threading.Lock()
-
-    def _db_onfail_ses(self) -> None:
-        self.args.no_ses = True
-
-    def _db_onfail_idp(self) -> None:
-        self.args.idp_store = 0
-
-    def setup_db(self, which: str) -> None:
-        """
-        the "non-mission-critical" databases; if something looks broken then just nuke it
-        """
-        if which == "ses":
-            native_ver = VER_SESSION_DB
-            db_path = self.args.ses_db
-            desc = "sessions-db"
-            pathopt = "ses-db"
-            sanchk_q = "select count(*) from us"
-            createfun = self._create_session_db
-            failfun = self._db_onfail_ses
-        elif which == "idp":
-            native_ver = VER_IDP_DB
-            db_path = self.args.idp_db
-            desc = "idp-db"
-            pathopt = "idp-db"
-            sanchk_q = "select count(*) from us"
-            createfun = self._create_idp_db
-            failfun = self._db_onfail_idp
-        else:
-            raise Exception("unknown cachetype")
-
-        if not db_path.endswith(".db"):
-            zs = "config option --%s (the %s) was configured to [%s] which is invalid; must be a filepath ending with .db"
-            self.log("root", zs % (pathopt, desc, db_path), 1)
-            raise Exception(BAD_CFG)
 
+    def setup_session_db(self) -> None:
         if not HAVE_SQLITE3:
-            failfun()
-            if which == "ses":
-                zs = "disabling sessions, will use plaintext passwords in cookies"
-            elif which == "idp":
-                zs = "disabling idp-db, will be unable to remember IdP-volumes after a restart"
-            self.log("root", "WARNING: sqlite3 not available; %s" % (zs,), 3)
+            self.args.no_ses = True
+            t = "WARNING: sqlite3 not available; disabling sessions, will use plaintext passwords in cookies"
+            self.log("root", t, 3)
             return
 
-        assert sqlite3  # type: ignore  # !rm
+        import sqlite3
 
-        db_lock = db_path + ".lock"
-        try:
-            create = not os.path.getsize(db_path)
-        except:
-            create = True
-        zs = "creating new" if create else "opening"
-        self.log("root", "%s %s %s" % (zs, desc, db_path))
-
-        for tries in range(2):
-            sver = 0
+        create = True
+        db_path = self.args.ses_db
+        self.log("root", "opening sessions-db %s" % (db_path,))
+        for n in range(2):
             try:
                 db = sqlite3.connect(db_path)
                 cur = db.cursor()
                 try:
-                    zs = "select v from kv where k='sver'"
-                    sver = cur.execute(zs).fetchall()[0][0]
-                    if sver > native_ver:
-                        zs = "this version of copyparty only understands %s v%d and older; the db is v%d"
-                        raise Exception(zs % (desc, native_ver, sver))
-
-                    cur.execute(sanchk_q).fetchone()
+                    cur.execute("select count(*) from us").fetchone()
+                    create = False
+                    break
                 except:
-                    if sver:
-                        raise
-                    sver = createfun(cur)
-
-                err = self._verify_db(
-                    cur, which, pathopt, db_path, desc, sver, native_ver
-                )
-                if err:
-                    tries = 99
-                    self.args.no_ses = True
-                    self.log("root", err, 3)
-                break
-
+                    pass
             except Exception as ex:
-                if tries or sver > native_ver:
+                if n:
                     raise
-                t = "%s is unusable; deleting and recreating: %r"
-                self.log("root", t % (desc, ex), 3)
+                t = "sessions-db corrupt; deleting and recreating: %r"
+                self.log("root", t % (ex,), 3)
                 try:
                     cur.close()  # type: ignore
                 except:
@@ -535,13 +431,8 @@ class SvcHub(object):
                     db.close()  # type: ignore
                 except:
                     pass
-                try:
-                    os.unlink(db_lock)
-                except:
-                    pass
                 os.unlink(db_path)
 
-    def _create_session_db(self, cur: "sqlite3.Cursor") -> int:
         sch = [
             r"create table kv (k text, v int)",
             r"create table us (un text, si text, t0 int)",
@@ -551,74 +442,17 @@ class SvcHub(object):
             r"create index us_t0 on us(t0)",
             r"insert into kv values ('sver', 1)",
         ]
-        for cmd in sch:
-            cur.execute(cmd)
-        self.log("root", "created new sessions-db")
-        return 1
 
-    def _create_idp_db(self, cur: "sqlite3.Cursor") -> int:
-        sch = [
-            r"create table kv (k text, v int)",
-            r"create table us (un text, gs text)",
-            # username, groups
-            r"create index us_un on us(un)",
-            r"insert into kv values ('sver', 1)",
-        ]
-        for cmd in sch:
-            cur.execute(cmd)
-        self.log("root", "created new idp-db")
-        return 1
+        assert db  # type: ignore  # !rm
+        assert cur  # type: ignore  # !rm
+        if create:
+            for cmd in sch:
+                cur.execute(cmd)
+            self.log("root", "created new sessions-db")
+            db.commit()
 
-    def _verify_db(
-        self,
-        cur: "sqlite3.Cursor",
-        which: str,
-        pathopt: str,
-        db_path: str,
-        desc: str,
-        sver: int,
-        native_ver: int,
-    ) -> str:
-        # ensure writable (maybe owned by other user)
-        db = cur.connection
-
-        try:
-            zil = cur.execute("select v from kv where k='pid'").fetchall()
-            if len(zil) > 1:
-                raise Exception()
-            owner = zil[0][0]
-        except:
-            owner = 0
-
-        if which == "ses":
-            cons = "Will now disable sessions and instead use plaintext passwords in cookies."
-        elif which == "idp":
-            cons = "Each IdP-volume will not become available until its associated user sends their first request."
-        else:
-            raise Exception()
-
-        if not lock_file(db_path + ".lock"):
-            t = "the %s [%s] is already in use by another copyparty instance (pid:%d). This is not supported; please provide another database with --%s or give this copyparty-instance its entirely separate config-folder by setting another path in the XDG_CONFIG_HOME env-var. You can also disable this safeguard by setting env-var PRTY_NO_DB_LOCK=1. %s"
-            return t % (desc, db_path, owner, pathopt, cons)
-
-        vars = (("pid", os.getpid()), ("ts", int(time.time() * 1000)))
-        if owner:
-            # wear-estimate: 2 cells; offsets 0x10, 0x50, 0x19720
-            for k, v in vars:
-                cur.execute("update kv set v=? where k=?", (v, k))
-        else:
-            # wear-estimate: 3~4 cells; offsets 0x10, 0x50, 0x19180, 0x19710, 0x36000, 0x360b0, 0x36b90
-            for k, v in vars:
-                cur.execute("insert into kv values(?, ?)", (k, v))
-
-        if sver < native_ver:
-            cur.execute("delete from kv where k='sver'")
-            cur.execute("insert into kv values('sver',?)", (native_ver,))
-
-        db.commit()
         cur.close()
         db.close()
-        return ""
 
     def setup_share_db(self) -> None:
         al = self.args
@@ -627,7 +461,7 @@ class SvcHub(object):
             al.shr = ""
             return
 
-        assert sqlite3  # type: ignore  # !rm
+        import sqlite3
 
         al.shr = al.shr.strip("/")
         if "/" in al.shr or not al.shr:
@@ -638,48 +472,34 @@ class SvcHub(object):
         al.shr = "/%s/" % (al.shr,)
         al.shr1 = al.shr[1:]
 
-        # policy:
-        # the shares-db is important, so panic if something is wrong
-
+        create = True
+        modified = False
         db_path = self.args.shr_db
-        db_lock = db_path + ".lock"
-        try:
-            create = not os.path.getsize(db_path)
-        except:
-            create = True
-        zs = "creating new" if create else "opening"
-        self.log("root", "%s shares-db %s" % (zs, db_path))
-
-        sver = 0
-        try:
-            db = sqlite3.connect(db_path)
-            cur = db.cursor()
-            if not create:
-                zs = "select v from kv where k='sver'"
-                sver = cur.execute(zs).fetchall()[0][0]
-                if sver > VER_SHARES_DB:
-                    zs = "this version of copyparty only understands shares-db v%d and older; the db is v%d"
-                    raise Exception(zs % (VER_SHARES_DB, sver))
-
-                cur.execute("select count(*) from sh").fetchone()
-        except Exception as ex:
-            t = "could not open shares-db; will now panic...\nthe following database must be repaired or deleted before you can launch copyparty:\n%s\n\nERROR: %s\n\nadditional details:\n%s\n"
-            self.log("root", t % (db_path, ex, min_ex()), 1)
-            raise
-
-        try:
-            zil = cur.execute("select v from kv where k='pid'").fetchall()
-            if len(zil) > 1:
-                raise Exception()
-            owner = zil[0][0]
-        except:
-            owner = 0
-
-        if not lock_file(db_lock):
-            t = "the shares-db [%s] is already in use by another copyparty instance (pid:%d). This is not supported; please provide another database with --shr-db or give this copyparty-instance its entirely separate config-folder by setting another path in the XDG_CONFIG_HOME env-var. You can also disable this safeguard by setting env-var PRTY_NO_DB_LOCK=1. Will now panic."
-            t = t % (db_path, owner)
-            self.log("root", t, 1)
-            raise Exception(t)
+        self.log("root", "opening shares-db %s" % (db_path,))
+        for n in range(2):
+            try:
+                db = sqlite3.connect(db_path)
+                cur = db.cursor()
+                try:
+                    cur.execute("select count(*) from sh").fetchone()
+                    create = False
+                    break
+                except:
+                    pass
+            except Exception as ex:
+                if n:
+                    raise
+                t = "shares-db corrupt; deleting and recreating: %r"
+                self.log("root", t % (ex,), 3)
+                try:
+                    cur.close()  # type: ignore
+                except:
+                    pass
+                try:
+                    db.close()  # type: ignore
+                except:
+                    pass
+                os.unlink(db_path)
 
         sch1 = [
             r"create table kv (k text, v int)",
@@ -691,37 +511,34 @@ class SvcHub(object):
             r"create index sf_k on sf(k)",
             r"create index sh_k on sh(k)",
             r"create index sh_t1 on sh(t1)",
-            r"insert into kv values ('sver', 2)",
         ]
 
         assert db  # type: ignore  # !rm
         assert cur  # type: ignore  # !rm
-        if not sver:
-            sver = VER_SHARES_DB
+        if create:
+            dver = 2
+            modified = True
             for cmd in sch1 + sch2:
                 cur.execute(cmd)
             self.log("root", "created new shares-db")
+        else:
+            (dver,) = cur.execute("select v from kv where k = 'sver'").fetchall()[0]
 
-        if sver == 1:
+        if dver == 1:
+            modified = True
             for cmd in sch2:
                 cur.execute(cmd)
             cur.execute("update sh set st = 0")
             self.log("root", "shares-db schema upgrade ok")
 
-        if sver < VER_SHARES_DB:
-            cur.execute("delete from kv where k='sver'")
-            cur.execute("insert into kv values('sver',?)", (VER_SHARES_DB,))
+        if modified:
+            for cmd in [
+                r"delete from kv where k = 'sver'",
+                r"insert into kv values ('sver', %d)" % (2,),
+            ]:
+                cur.execute(cmd)
+            db.commit()
 
-        vars = (("pid", os.getpid()), ("ts", int(time.time() * 1000)))
-        if owner:
-            # wear-estimate: same as sessions-db
-            for k, v in vars:
-                cur.execute("update kv set v=? where k=?", (v, k))
-        else:
-            for k, v in vars:
-                cur.execute("insert into kv values(?, ?)", (k, v))
-
-        db.commit()
         cur.close()
         db.close()
 
@@ -786,39 +603,6 @@ class SvcHub(object):
     def sigterm(self) -> None:
         self.signal_handler(signal.SIGTERM, None)
 
-    def sticky_qr(self) -> None:
-        tw, th = termsize()
-        zs1, qr = self.tcpsrv.qr.split("\n", 1)
-        url, colr = zs1.split(" ", 1)
-        nl = len(qr.split("\n"))  # numlines
-        lp = 3 if nl * 2 + 4 < tw else 0  # leftpad
-        lp0 = lp
-        if self.args.qr_pin == 2:
-            url = ""
-        else:
-            while lp and (nl + lp) * 2 + len(url) + 1 > tw:
-                lp -= 1
-            if (nl + lp) * 2 + len(url) + 1 > tw:
-                qr = url + "\n" + qr
-                url = ""
-                nl += 1
-                lp = lp0
-        sh = 1 + th - nl
-        if lp:
-            zs = " " * lp
-            qr = zs + qr.replace("\n", "\n" + zs)
-        if url:
-            url = "%s\033[%d;%dH%s\033[0m" % (colr, sh + 1, (nl + lp) * 2, url)
-        qr = colr + qr
-
-        def unlock():
-            print("\033[s\033[r\033[u", file=sys.stderr)
-
-        atexit.register(unlock)
-        t = "%s\033[%dA" % ("\n" * nl, nl)
-        t = "%s\033[s\033[1;%dr\033[%dH%s%s\033[u" % (t, sh - 1, sh, qr, url)
-        self.pr(t, file=sys.stderr)
-
     def cb_httpsrv_up(self) -> None:
         self.httpsrv_up += 1
         if self.httpsrv_up != self.broker.num_workers:
@@ -831,10 +615,7 @@ class SvcHub(object):
                 break
 
         if self.tcpsrv.qr:
-            if self.args.qr_pin:
-                self.sticky_qr()
-            else:
-                self.log("qr-code", self.tcpsrv.qr)
+            self.log("qr-code", self.tcpsrv.qr)
         else:
             self.log("root", "workers OK\n")
 
@@ -858,10 +639,8 @@ class SvcHub(object):
             (HAVE_FFPROBE, "ffprobe", t_ff + ", read audio/media tags"),
             (HAVE_MUTAGEN, "mutagen", "read audio tags (ffprobe is better but slower)"),
             (HAVE_ARGON2, "argon2", "secure password hashing (advanced users only)"),
-            (HAVE_ZMQ, "pyzmq", "send zeromq messages from event-hooks"),
             (HAVE_HEIF, "pillow-heif", "read .heif images with pillow (rarely useful)"),
             (HAVE_AVIF, "pillow-avif", "read .avif images with pillow (rarely useful)"),
-            (HAVE_RAW, "rawpy", "read RAW images"),
         ]
         if ANYWIN:
             to_check += [
@@ -896,11 +675,19 @@ class SvcHub(object):
                 t += ", "
             t += "\033[0mNG: \033[35m" + sng
 
-        t += "\033[0m, see --deps (this is fine btw)"
-        self.log("optional-dependencies", t, 6)
+        t += "\033[0m, see --deps"
+        self.log("dependencies", t, 6)
 
     def _check_env(self) -> None:
-        al = self.args
+        try:
+            files = os.listdir(E.cfg)
+        except:
+            files = []
+
+        hits = [x for x in files if x.lower().endswith(".conf")]
+        if hits:
+            t = "WARNING: found config files in [%s]: %s\n  config files are not expected here, and will NOT be loaded (unless your setup is intentionally hella funky)"
+            self.log("root", t % (E.cfg, ", ".join(hits)), 3)
 
         if self.args.no_bauth:
             t = "WARNING: --no-bauth disables support for the Android app; you may want to use --bauth-last instead"
@@ -908,30 +695,6 @@ class SvcHub(object):
             if self.args.bauth_last:
                 self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
 
-        have_tcp = False
-        for zs in al.i:
-            if not zs.startswith(("unix:", "fd:")):
-                have_tcp = True
-        if not have_tcp:
-            zb = False
-            zs = "z zm zm4 zm6 zmv zmvv zs zsv zv"
-            for zs in zs.split():
-                if getattr(al, zs, False):
-                    setattr(al, zs, False)
-                    zb = True
-            if zb:
-                t = "not listening on any ip-addresses (only unix-sockets and/or FDs); cannot enable zeroconf/mdns/ssdp as requested"
-                self.log("root", t, 3)
-
-        if not self.args.no_dav:
-            from .dxml import DXML_OK
-
-            if not DXML_OK:
-                if not self.args.no_dav:
-                    self.args.no_dav = True
-                    t = "WARNING:\nDisabling WebDAV support because dxml selftest failed. Please report this bug;\n%s\n...and include the following information in the bug-report:\n%s | expat %s\n"
-                    self.log("root", t % (URL_BUG, VERSIONS, expat_ver()), 1)
-
     def _process_config(self) -> bool:
         al = self.args
 
@@ -987,20 +750,13 @@ class SvcHub(object):
             vl = [os.path.expandvars(os.path.expanduser(x)) for x in vl]
             setattr(al, k, vl)
 
-        for k in "lo hist dbpath ssl_log".split(" "):
+        for k in "lo hist ssl_log".split(" "):
             vs = getattr(al, k)
             if vs:
                 vs = os.path.expandvars(os.path.expanduser(vs))
                 setattr(al, k, vs)
 
-        for k in "idp_adm".split(" "):
-            vs = getattr(al, k)
-            vsa = [x.strip() for x in vs.split(",")]
-            vsa = [x.lower() for x in vsa if x]
-            setattr(al, k + "_set", set(vsa))
-
-        zs = "dav_ua1 sus_urls nonsus_urls ua_nodoc ua_nozip"
-        for k in zs.split(" "):
+        for k in "sus_urls nonsus_urls".split(" "):
             vs = getattr(al, k)
             if not vs or vs == "no":
                 setattr(al, k, None)
@@ -1020,23 +776,10 @@ class SvcHub(object):
             al.sus_urls = None
 
         al.xff_hdr = al.xff_hdr.lower()
-        al.idp_h_usr = [x.lower() for x in al.idp_h_usr or []]
+        al.idp_h_usr = al.idp_h_usr.lower()
         al.idp_h_grp = al.idp_h_grp.lower()
         al.idp_h_key = al.idp_h_key.lower()
 
-        al.idp_hm_usr_p = {}
-        for zs0 in al.idp_hm_usr or []:
-            try:
-                sep = zs0[:1]
-                hn, zs1, zs2 = zs0[1:].split(sep)
-                hn = hn.lower()
-                if hn in al.idp_hm_usr_p:
-                    al.idp_hm_usr_p[hn][zs1] = zs2
-                else:
-                    al.idp_hm_usr_p[hn] = {zs1: zs2}
-            except:
-                raise Exception("invalid --idp-hm-usr [%s]" % (zs0,))
-
         al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa, True)
         al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa, True)
 
@@ -1081,8 +824,6 @@ class SvcHub(object):
         except:
             raise Exception("invalid --mv-retry [%s]" % (self.args.mv_retry,))
 
-        al.js_utc = "false" if al.localtime else "true"
-
         al.tcolor = al.tcolor.lstrip("#")
         if len(al.tcolor) == 3:  # fc5 => ffcc55
             al.tcolor = "".join([x * 2 for x in al.tcolor])
@@ -1177,7 +918,7 @@ class SvcHub(object):
 
         fn = sel_fn
         try:
-            bos.makedirs(os.path.dirname(fn))
+            os.makedirs(os.path.dirname(fn))
         except:
             pass
 
@@ -1194,9 +935,6 @@ class SvcHub(object):
 
             lh = codecs.open(fn, "w", encoding="utf-8", errors="replace")
 
-        if getattr(self.args, "free_umask", False):
-            os.fchmod(lh.fileno(), 0o644)
-
         argv = [pybin] + self.argv
         if hasattr(shlex, "quote"):
             argv = [shlex.quote(x) for x in argv]
@@ -1464,18 +1202,11 @@ class SvcHub(object):
 
             fmt = "\033[36m%s \033[33m%-21s \033[0m%s\n"
             if self.no_ansi:
-                if c == 1:
-                    fmt = "%s %-21s CRIT: %s\n"
-                elif c == 3:
-                    fmt = "%s %-21s WARN: %s\n"
-                elif c == 6:
-                    fmt = "%s %-21s  BTW: %s\n"
-                else:
-                    fmt = "%s %-21s  LOG: %s\n"
+                fmt = "%s %-21s %s\n"
                 if "\033" in msg:
-                    msg = RE_ANSI.sub("", msg)
+                    msg = ansi_re.sub("", msg)
                 if "\033" in src:
-                    src = RE_ANSI.sub("", src)
+                    src = ansi_re.sub("", src)
             elif c:
                 if isinstance(c, int):
                     msg = "\033[3%sm%s\033[0m" % (c, msg)
@@ -1516,7 +1247,7 @@ class SvcHub(object):
                 raise
 
     def check_mp_support(self) -> str:
-        if MACOS and not os.environ.get("PRTY_FORCE_MP"):
+        if MACOS:
             return "multiprocessing is wonky on mac osx;"
         elif sys.version_info < (3, 3):
             return "need python 3.3 or newer for multiprocessing;"
@@ -1536,7 +1267,7 @@ class SvcHub(object):
             return False
 
         try:
-            if mp.cpu_count() <= 1 and not os.environ.get("PRTY_FORCE_MP"):
+            if mp.cpu_count() <= 1:
                 raise Exception()
         except:
             self.log("svchub", "only one CPU detected; multiprocessing disabled")

copyparty/szip.py

@@ -4,11 +4,12 @@ from __future__ import print_function, unicode_literals
 import calendar
 import stat
 import time
+import zlib
 
 from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
-from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile, zlib
+from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Generator, Optional
@@ -54,7 +55,6 @@ def gen_fdesc(sz: int, crc32: int, z64: bool) -> bytes:
 
 def gen_hdr(
     h_pos: Optional[int],
-    z64: bool,
     fn: str,
     sz: int,
     lastmod: int,
@@ -71,6 +71,7 @@ def gen_hdr(
     # appnote 4.5 / zip 3.0 (2008) / unzip 6.0 (2009) says to add z64
     # extinfo for values which exceed H, but that becomes an off-by-one
     # (can't tell if it was clamped or exactly maxval), make it obvious
+    z64 = sz >= 0xFFFFFFFF
     z64v = [sz, sz] if z64 else []
     if h_pos and h_pos >= 0xFFFFFFFF:
         # central, also consider ptr to original header
@@ -244,7 +245,6 @@ class StreamZip(StreamArc):
 
         sz = st.st_size
         ts = st.st_mtime
-        h_pos = self.pos
 
         crc = 0
         if self.pre_crc:
@@ -253,12 +253,8 @@ class StreamZip(StreamArc):
 
             crc &= 0xFFFFFFFF
 
-        # some unzip-programs expect a 64bit data-descriptor
-        # even if the only 32bit-exceeding value is the offset,
-        # so force that by placeholdering the filesize too
-        z64 = h_pos >= 0xFFFFFFFF or sz >= 0xFFFFFFFF
-
-        buf = gen_hdr(None, z64, name, sz, ts, self.utf8, crc, self.pre_crc)
+        h_pos = self.pos
+        buf = gen_hdr(None, name, sz, ts, self.utf8, crc, self.pre_crc)
         yield self._ct(buf)
 
         for buf in yieldfile(src, self.args.iobuf):
@@ -271,6 +267,8 @@ class StreamZip(StreamArc):
 
         self.items.append((name, sz, ts, crc, h_pos))
 
+        z64 = sz >= 4 * 1024 * 1024 * 1024
+
         if z64 or not self.pre_crc:
             buf = gen_fdesc(sz, crc, z64)
             yield self._ct(buf)
@@ -309,8 +307,7 @@ class StreamZip(StreamArc):
 
             cdir_pos = self.pos
             for name, sz, ts, crc, h_pos in self.items:
-                z64 = h_pos >= 0xFFFFFFFF or sz >= 0xFFFFFFFF
-                buf = gen_hdr(h_pos, z64, name, sz, ts, self.utf8, crc, self.pre_crc)
+                buf = gen_hdr(h_pos, name, sz, ts, self.utf8, crc, self.pre_crc)
                 mbuf += self._ct(buf)
                 if len(mbuf) >= 16384:
                     yield mbuf

copyparty/tcpsrv.py

@@ -25,8 +25,8 @@ from .util import (
     termsize,
 )
 
-if True:  # pylint: disable=using-constant-test
-    from typing import Generator, Optional, Union
+if True:
+    from typing import Generator, Union
 
 if TYPE_CHECKING:
     from .svchub import SvcHub
@@ -151,15 +151,9 @@ class TcpSrv(object):
             if just_ll or self.args.ll:
                 ll_ok.add(ip.split("/")[0])
 
-        listening_on = []
-        for ip, ports in sorted(ok.items()):
-            for port in sorted(ports):
-                listening_on.append("%s %s" % (ip, port))
-
         qr1: dict[str, list[int]] = {}
         qr2: dict[str, list[int]] = {}
         msgs = []
-        accessible_on = []
         title_tab: dict[str, dict[str, int]] = {}
         title_vars = [x[1:] for x in self.args.wintitle.split(" ") if x.startswith("$")]
         t = "available @ {}://{}:{}/  (\033[33m{}\033[0m)"
@@ -175,10 +169,6 @@ class TcpSrv(object):
                 ):
                     continue
 
-                zs = "%s %s" % (ip, port)
-                if zs not in accessible_on:
-                    accessible_on.append(zs)
-
                 proto = " http"
                 if self.args.http_only:
                     pass
@@ -229,14 +219,6 @@ class TcpSrv(object):
         else:
             print("\n", end="")
 
-        for fn, ls in (
-            (self.args.wr_h_eps, listening_on),
-            (self.args.wr_h_aon, accessible_on),
-        ):
-            if fn:
-                with open(fn, "wb") as f:
-                    f.write(("\n".join(ls)).encode("utf-8"))
-
         if self.args.qr or self.args.qrs:
             self.qr = self._qr(qr1, qr2)
 
@@ -245,10 +227,8 @@ class TcpSrv(object):
 
     def _listen(self, ip: str, port: int) -> None:
         uds_perm = uds_gid = -1
-        bound: Optional[socket.socket] = None
-        tcp = False
-
         if "unix:" in ip:
+            tcp = False
             ipv = socket.AF_UNIX
             uds = ip.split(":")
             ip = uds[-1]
@@ -261,12 +241,7 @@ class TcpSrv(object):
                     import grp
 
                     uds_gid = grp.getgrnam(uds[2]).gr_gid
-        elif "fd:" in ip:
-            fd = ip[3:]
-            bound = socket.socket(fileno=int(fd))
 
-            tcp = bound.proto == socket.IPPROTO_TCP
-            ipv = bound.family
         elif ":" in ip:
             tcp = True
             ipv = socket.AF_INET6
@@ -274,7 +249,7 @@ class TcpSrv(object):
             tcp = True
             ipv = socket.AF_INET
 
-        srv = bound or socket.socket(ipv, socket.SOCK_STREAM)
+        srv = socket.socket(ipv, socket.SOCK_STREAM)
 
         if not ANYWIN or self.args.reuseaddr:
             srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
@@ -289,13 +264,9 @@ class TcpSrv(object):
         except:
             pass  # will create another ipv4 socket instead
 
-        if getattr(self.args, "freebind", False):
+        if not ANYWIN and self.args.freebind:
             srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
 
-        if bound:
-            self.srv.append(srv)
-            return
-
         try:
             if tcp:
                 srv.bind((ip, port))
@@ -435,12 +406,12 @@ class TcpSrv(object):
             rem = []
             for k, v in netdevs.items():
                 if k not in self.netdevs:
-                    add.append("\n\033[32m  added %s = %s" % (k, v))
+                    add.append("\n  added %s = %s" % (k, v))
             for k, v in self.netdevs.items():
                 if k not in netdevs:
-                    rem.append("\n\033[33mremoved %s = %s" % (k, v))
+                    rem.append("\nremoved %s = %s" % (k, v))
 
-            t = "network change detected:%s%s"
+            t = "network change detected:\033[32m%s\033[33m%s"
             self.log("tcpsrv", t % ("".join(add), "".join(rem)), 3)
             self.netdevs = netdevs
             self._distribute_netdevs()
@@ -448,7 +419,7 @@ class TcpSrv(object):
     def detect_interfaces(self, listen_ips: list[str]) -> dict[str, Netdev]:
         from .stolen.ifaddr import get_adapters
 
-        listen_ips = [x for x in listen_ips if not x.startswith(("unix:", "fd:"))]
+        listen_ips = [x for x in listen_ips if "unix:" not in x]
 
         nics = get_adapters(True)
         eps: dict[str, Netdev] = {}
@@ -577,7 +548,7 @@ class TcpSrv(object):
         ip = None
         ips = list(t1) + list(t2)
         qri = self.args.qri
-        if self.args.zm and not qri and ips:
+        if self.args.zm and not qri:
             name = self.args.name + ".local"
             t1[name] = next(v for v in (t1 or t2).values())
             ips = [name] + ips
@@ -594,7 +565,8 @@ class TcpSrv(object):
         if not ip:
             return ""
 
-        hip = "[%s]" % (ip,) if ":" in ip else ip
+        if ":" in ip:
+            ip = "[{}]".format(ip)
 
         if self.args.http_only:
             https = ""
@@ -606,7 +578,7 @@ class TcpSrv(object):
         ports = t1.get(ip, t2.get(ip, []))
         dport = 443 if https else 80
         port = "" if dport in ports or not ports else ":{}".format(ports[0])
-        txt = "http{}://{}{}/{}".format(https, hip, port, self.args.qrl)
+        txt = "http{}://{}{}/{}".format(https, ip, port, self.args.qrl)
 
         btxt = txt.encode("utf-8")
         if PY2:
@@ -614,10 +586,6 @@ class TcpSrv(object):
 
         fg = self.args.qr_fg
         bg = self.args.qr_bg
-        nocolor = fg == -1
-        if nocolor:
-            fg = 0
-
         pad = self.args.qrp
         zoom = self.args.qrz
         qrc = QrCode.encode_binary(btxt)
@@ -645,8 +613,6 @@ class TcpSrv(object):
 
         qr = qr.replace("\n", "\033[K\n") + "\033[K"  # win10do
         cc = " \033[0;38;5;{0};47;48;5;{1}m" if fg else " \033[0;30;47m"
-        if nocolor:
-            cc = " \033[0m"
         t = cc + "\n{2}\033[999G\033[0m\033[J"
         t = t.format(fg, bg, qr)
         if ANYWIN:

copyparty/tftpd.py

@@ -36,20 +36,7 @@ from partftpy.TftpShared import TftpException
 from .__init__ import EXE, PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
-from .util import (
-    FN_EMB,
-    UTC,
-    BytesIO,
-    Daemon,
-    ODict,
-    exclude_dotfiles,
-    min_ex,
-    runhook,
-    set_fperms,
-    undot,
-    vjoin,
-    vsplit,
-)
+from .util import UTC, BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Union
@@ -179,7 +166,7 @@ class Tftpd(object):
         if "::" in ips:
             ips.append("0.0.0.0")
 
-        ips = [x for x in ips if not x.startswith(("unix:", "fd:"))]
+        ips = [x for x in ips if "unix:" not in x]
 
         if self.args.tftp4:
             ips = [x for x in ips if ":" not in x]
@@ -257,25 +244,16 @@ class Tftpd(object):
         for srv in srvs:
             srv.stop()
 
-    def _v2a(
-        self, caller: str, vpath: str, perms: list, *a: Any
-    ) -> tuple[VFS, str, str]:
+    def _v2a(self, caller: str, vpath: str, perms: list, *a: Any) -> tuple[VFS, str]:
         vpath = vpath.replace("\\", "/").lstrip("/")
         if not perms:
             perms = [True, True]
 
         debug('%s("%s", %s) %s\033[K\033[0m', caller, vpath, str(a), perms)
         vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
-        if perms[1] and "*" not in vfs.axs.uread and "wo_up_readme" not in vfs.flags:
-            zs, fn = vsplit(vpath)
-            if fn.lower() in FN_EMB:
-                vpath = vjoin(zs, "_wo_" + fn)
-                vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
-
         if not vfs.realpath:
             raise Exception("unmapped vfs")
-
-        return vfs, vpath, vfs.canonical(rem)
+        return vfs, vfs.canonical(rem)
 
     def _ls(self, vpath: str, raddress: str, rport: int, force=False) -> Any:
         # generate file listing if vpath is dir.txt and return as file object
@@ -285,7 +263,6 @@ class Tftpd(object):
             if not ptn or not ptn.match(fn.lower()):
                 return None
 
-        tsdt = datetime.fromtimestamp
         vn, rem = self.asrv.vfs.get(vpath, "*", True, False)
         fsroot, vfs_ls, vfs_virt = vn.ls(
             rem,
@@ -298,7 +275,7 @@ class Tftpd(object):
         dirs1 = [(v.st_mtime, v.st_size, k + "/") for k, v in vfs_ls if k in dnames]
         fils1 = [(v.st_mtime, v.st_size, k) for k, v in vfs_ls if k not in dnames]
         real1 = dirs1 + fils1
-        realt = [(tsdt(max(0, mt), UTC), sz, fn) for mt, sz, fn in real1]
+        realt = [(datetime.fromtimestamp(mt, UTC), sz, fn) for mt, sz, fn in real1]
         reals = [
             (
                 "%04d-%02d-%02d %02d:%02d:%02d"
@@ -354,7 +331,7 @@ class Tftpd(object):
         else:
             raise Exception("bad mode %s" % (mode,))
 
-        vfs, vpath, ap = self._v2a("open", vpath, [rd, wr])
+        vfs, ap = self._v2a("open", vpath, [rd, wr])
         if wr:
             if "*" not in vfs.axs.uwrite:
                 yeet("blocked write; folder not world-writable: /%s" % (vpath,))
@@ -388,24 +365,18 @@ class Tftpd(object):
         if not a:
             a = (self.args.iobuf,)
 
-        ret = open(ap, mode, *a, **ka)
-        if wr and "fperms" in vfs.flags:
-            set_fperms(ret, vfs.flags)
-
-        return ret
+        return open(ap, mode, *a, **ka)
 
     def _mkdir(self, vpath: str, *a) -> None:
-        vfs, _, ap = self._v2a("mkdir", vpath, [False, True])
+        vfs, ap = self._v2a("mkdir", vpath, [])
         if "*" not in vfs.axs.uwrite:
             yeet("blocked mkdir; folder not world-writable: /%s" % (vpath,))
 
-        bos.mkdir(ap, vfs.flags["chmod_d"])
-        if "chown" in vfs.flags:
-            bos.chown(ap, vfs.flags["uid"], vfs.flags["gid"])
+        return bos.mkdir(ap)
 
     def _unlink(self, vpath: str) -> None:
         # return bos.unlink(self._v2a("stat", vpath, *a)[1])
-        vfs, _, ap = self._v2a("delete", vpath, [True, False, False, True])
+        vfs, ap = self._v2a("delete", vpath, [True, False, False, True])
 
         try:
             inf = bos.stat(ap)
@@ -429,7 +400,7 @@ class Tftpd(object):
 
     def _p_exists(self, vpath: str) -> bool:
         try:
-            ap = self._v2a("p.exists", vpath, [False, False])[2]
+            ap = self._v2a("p.exists", vpath, [False, False])[1]
             bos.stat(ap)
             return True
         except:
@@ -437,7 +408,7 @@ class Tftpd(object):
 
     def _p_isdir(self, vpath: str) -> bool:
         try:
-            st = bos.stat(self._v2a("p.isdir", vpath, [False, False])[2])
+            st = bos.stat(self._v2a("p.isdir", vpath, [False, False])[1])
             ret = stat.S_ISDIR(st.st_mode)
             return ret
         except:

copyparty/th_cli.py

@@ -1,15 +1,13 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
-import errno
 import os
-import stat
 
 from .__init__ import TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
-from .th_srv import EXTS_AC, HAVE_WEBP, thumb_path
-from .util import Cooldown, Pebkac
+from .th_srv import HAVE_WEBP, thumb_path
+from .util import Cooldown
 
 if True:  # pylint: disable=using-constant-test
     from typing import Optional, Union
@@ -18,9 +16,6 @@ if TYPE_CHECKING:
     from .httpsrv import HttpSrv
 
 
-IOERROR = "reading the file was denied by the server os; either due to filesystem permissions, selinux, apparmor, or similar:\n%r"
-
-
 class ThumbCli(object):
     def __init__(self, hsrv: "HttpSrv") -> None:
         self.broker = hsrv.broker
@@ -36,15 +31,11 @@ class ThumbCli(object):
             if not c:
                 raise Exception()
         except:
-            c = {
-                k: set()
-                for k in ["thumbable", "pil", "vips", "raw", "ffi", "ffv", "ffa"]
-            }
+            c = {k: set() for k in ["thumbable", "pil", "vips", "ffi", "ffv", "ffa"]}
 
         self.thumbable = c["thumbable"]
         self.fmt_pil = c["pil"]
         self.fmt_vips = c["vips"]
-        self.fmt_raw = c["raw"]
         self.fmt_ffi = c["ffi"]
         self.fmt_ffv = c["ffv"]
         self.fmt_ffa = c["ffa"]
@@ -66,17 +57,13 @@ class ThumbCli(object):
         if is_vid and "dvthumb" in dbv.flags:
             return None
 
-        want_opus = fmt in EXTS_AC
+        want_opus = fmt in ("opus", "caf", "mp3")
         is_au = ext in self.fmt_ffa
         is_vau = want_opus and ext in self.fmt_ffv
         if is_au or is_vau:
             if want_opus:
                 if self.args.no_acode:
                     return None
-                elif fmt == "caf" and self.args.no_caf:
-                    fmt = "mp3"
-                elif fmt == "owa" and self.args.no_owa:
-                    fmt = "mp3"
             else:
                 if "dathumb" in dbv.flags:
                     return None
@@ -92,7 +79,7 @@ class ThumbCli(object):
         if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg", "png"]:
             return os.path.join(ptop, rem)
 
-        if fmt[:1] in "jw" and fmt != "wav":
+        if fmt[:1] in "jw":
             sfmt = fmt[:1]
 
             if sfmt == "j" and self.args.th_no_jpg:
@@ -133,7 +120,7 @@ class ThumbCli(object):
 
         tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
         tpaths = [tpath]
-        if fmt[:1] == "w" and fmt != "wav":
+        if fmt == "w":
             # also check for jpg (maybe webp is unavailable)
             tpaths.append(tpath.rsplit(".", 1)[0] + ".jpg")
 
@@ -166,22 +153,8 @@ class ThumbCli(object):
         if abort:
             return None
 
-        ap = os.path.join(ptop, rem)
-        try:
-            st = bos.stat(ap)
-            if not st.st_size or not stat.S_ISREG(st.st_mode):
-                return None
-
-            with open(ap, "rb", 4) as f:
-                if not f.read(4):
-                    raise Exception()
-        except OSError as ex:
-            if ex.errno == errno.ENOENT:
-                raise Pebkac(404)
-            else:
-                raise Pebkac(500, IOERROR % (ex,))
-        except Exception as ex:
-            raise Pebkac(500, IOERROR % (ex,))
+        if not bos.path.getsize(os.path.join(ptop, rem)):
+            return None
 
         x = self.broker.ask("thumbsrv.get", ptop, rem, mtime, fmt)
         return x.get()  # type: ignore

copyparty/th_srv.py

@@ -2,13 +2,10 @@
 from __future__ import print_function, unicode_literals
 
 import hashlib
-import io
 import logging
 import os
-import re
 import shutil
 import subprocess as sp
-import tempfile
 import threading
 import time
 
@@ -21,22 +18,21 @@ from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, au_unpk, ffprobe
 from .util import BytesIO  # type: ignore
 from .util import (
     FFMPEG_URL,
-    VF_CAREFUL,
     Cooldown,
     Daemon,
     afsenc,
-    atomic_move,
     fsenc,
     min_ex,
     runcmd,
     statdir,
     ub64enc,
     vsplit,
+    wrename,
     wunlink,
 )
 
 if True:  # pylint: disable=using-constant-test
-    from typing import Any, Optional, Union
+    from typing import Optional, Union
 
 if TYPE_CHECKING:
     from .svchub import SvcHub
@@ -50,13 +46,6 @@ HAVE_HEIF = False
 HAVE_AVIF = False
 HAVE_WEBP = False
 
-EXTS_TH = set(["jpg", "webp", "png"])
-EXTS_AC = set(["opus", "owa", "caf", "mp3", "flac", "wav"])
-EXTS_SPEC_SAFE = set("aif aiff flac mp3 opus wav".split())
-
-PTN_TS = re.compile("^-?[0-9a-f]{8,10}$")
-
-
 try:
     if os.environ.get("PRTY_NO_PIL"):
         raise Exception()
@@ -86,10 +75,7 @@ try:
         if os.environ.get("PRTY_NO_PIL_HEIF"):
             raise Exception()
 
-        try:
-            from pillow_heif import register_heif_opener
-        except ImportError:
-            from pyheif_pillow_opener import register_heif_opener
+        from pyheif_pillow_opener import register_heif_opener
 
         register_heif_opener()
         HAVE_HEIF = True
@@ -100,10 +86,6 @@ try:
         if os.environ.get("PRTY_NO_PIL_AVIF"):
             raise Exception()
 
-        if ".avif" in Image.registered_extensions():
-            HAVE_AVIF = True
-            raise Exception()
-
         import pillow_avif  # noqa: F401  # pylint: disable=unused-import
 
         HAVE_AVIF = True
@@ -116,28 +98,14 @@ except:
 
 try:
     if os.environ.get("PRTY_NO_VIPS"):
-        raise ImportError()
+        raise Exception()
 
     HAVE_VIPS = True
     import pyvips
 
     logging.getLogger("pyvips").setLevel(logging.WARNING)
-except Exception as e:
-    HAVE_VIPS = False
-    if not isinstance(e, ImportError):
-        logging.warning("libvips found, but failed to load: " + str(e))
-
-
-try:
-    if os.environ.get("PRTY_NO_RAW"):
-        raise Exception()
-
-    HAVE_RAW = True
-    import rawpy
-
-    logging.getLogger("rawpy").setLevel(logging.WARNING)
 except:
-    HAVE_RAW = False
+    HAVE_VIPS = False
 
 
 th_dir_cache = {}
@@ -171,7 +139,7 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
     h = hashlib.sha512(afsenc(fn)).digest()
     fn = ub64enc(h).decode("ascii")[:24]
 
-    if fmt in EXTS_AC:
+    if fmt in ("opus", "caf", "mp3"):
         cat = "ac"
     else:
         fc = fmt[:1]
@@ -192,15 +160,12 @@ class ThumbSrv(object):
 
         self.mutex = threading.Lock()
         self.busy: dict[str, list[threading.Condition]] = {}
-        self.untemp: dict[str, list[str]] = {}
         self.ram: dict[str, float] = {}
         self.memcond = threading.Condition(self.mutex)
         self.stopping = False
         self.rm_nullthumbs = True  # forget failed conversions on startup
         self.nthr = max(1, self.args.th_mt)
 
-        self.exts_spec_unsafe = set(self.args.th_spec_cnv.split(","))
-
         self.q: Queue[Optional[tuple[str, str, str, VFS]]] = Queue(self.nthr * 4)
         for n in range(self.nthr):
             Daemon(self.worker, "thumb-{}-{}".format(n, self.nthr))
@@ -223,19 +188,11 @@ class ThumbSrv(object):
         if self.args.th_clean:
             Daemon(self.cleaner, "thumb.cln")
 
-        (
-            self.fmt_pil,
-            self.fmt_vips,
-            self.fmt_raw,
-            self.fmt_ffi,
-            self.fmt_ffv,
-            self.fmt_ffa,
-        ) = [
+        self.fmt_pil, self.fmt_vips, self.fmt_ffi, self.fmt_ffv, self.fmt_ffa = [
             set(y.split(","))
             for y in [
                 self.args.th_r_pil,
                 self.args.th_r_vips,
-                self.args.th_r_raw,
                 self.args.th_r_ffi,
                 self.args.th_r_ffv,
                 self.args.th_r_ffa,
@@ -258,9 +215,6 @@ class ThumbSrv(object):
         if "vips" in self.args.th_dec:
             self.thumbable |= self.fmt_vips
 
-        if "raw" in self.args.th_dec:
-            self.thumbable |= self.fmt_raw
-
         if "ff" in self.args.th_dec:
             for zss in [self.fmt_ffi, self.fmt_ffv, self.fmt_ffa]:
                 self.thumbable |= zss
@@ -298,8 +252,7 @@ class ThumbSrv(object):
                 self.log("joined waiting room for %r" % (tpath,))
             except:
                 thdir = os.path.dirname(tpath)
-                chmod = bos.MKD_700 if self.args.free_umask else bos.MKD_755
-                bos.makedirs(os.path.join(thdir, "w"), vf=chmod)
+                bos.makedirs(os.path.join(thdir, "w"))
 
                 inf_path = os.path.join(thdir, "dir.txt")
                 if not bos.path.exists(inf_path):
@@ -314,7 +267,7 @@ class ThumbSrv(object):
             vn = next((x for x in allvols if x.realpath == ptop), None)
             if not vn:
                 self.log("ptop %r not in %s" % (ptop, allvols), 3)
-                vn = self.asrv.vfs.all_aps[0][1][0]
+                vn = self.asrv.vfs.all_aps[0][1]
 
             self.q.put((abspath, tpath, fmt, vn))
             self.log("conv %r :%s \033[0m%r" % (tpath, fmt, abspath), 6)
@@ -342,7 +295,6 @@ class ThumbSrv(object):
             "thumbable": self.thumbable,
             "pil": self.fmt_pil,
             "vips": self.fmt_vips,
-            "raw": self.fmt_raw,
             "ffi": self.fmt_ffi,
             "ffv": self.fmt_ffv,
             "ffa": self.fmt_ffa,
@@ -382,13 +334,10 @@ class ThumbSrv(object):
                 ap_unpk = abspath
 
             if not bos.path.exists(tpath):
-                tex = tpath.rsplit(".", 1)[-1]
-                want_mp3 = tex == "mp3"
-                want_opus = tex in ("opus", "owa", "caf")
-                want_flac = tex == "flac"
-                want_wav = tex == "wav"
-                want_png = tex == "png"
-                want_au = want_mp3 or want_opus or want_flac or want_wav
+                want_mp3 = tpath.endswith(".mp3")
+                want_opus = tpath.endswith(".opus") or tpath.endswith(".caf")
+                want_png = tpath.endswith(".png")
+                want_au = want_mp3 or want_opus
                 for lib in self.args.th_dec:
                     can_au = lib == "ff" and (
                         ext in self.fmt_ffa or ext in self.fmt_ffv
@@ -398,17 +347,11 @@ class ThumbSrv(object):
                         funs.append(self.conv_pil)
                     elif lib == "vips" and ext in self.fmt_vips:
                         funs.append(self.conv_vips)
-                    elif lib == "raw" and ext in self.fmt_raw:
-                        funs.append(self.conv_raw)
                     elif can_au and (want_png or want_au):
                         if want_opus:
                             funs.append(self.conv_opus)
                         elif want_mp3:
                             funs.append(self.conv_mp3)
-                        elif want_flac:
-                            funs.append(self.conv_flac)
-                        elif want_wav:
-                            funs.append(self.conv_wav)
                         elif want_png:
                             funs.append(self.conv_waves)
                             png_ok = True
@@ -438,12 +381,8 @@ class ThumbSrv(object):
                     self.log(msg, c)
                     if getattr(ex, "returncode", 0) != 321:
                         if fun == funs[-1]:
-                            try:
-                                with open(ttpath, "wb") as _:
-                                    pass
-                            except Exception as ex:
-                                t = "failed to create the file [%s]: %r"
-                                self.log(t % (ttpath, ex), 3)
+                            with open(ttpath, "wb") as _:
+                                pass
                     else:
                         # ffmpeg may spawn empty files on windows
                         try:
@@ -455,25 +394,14 @@ class ThumbSrv(object):
                 wunlink(self.log, ap_unpk, vn.flags)
 
             try:
-                atomic_move(self.log, ttpath, tpath, vn.flags)
-            except Exception as ex:
-                if not os.path.exists(tpath):
-                    t = "failed to move  [%s]  to  [%s]:  %r"
-                    self.log(t % (ttpath, tpath, ex), 3)
+                wrename(self.log, ttpath, tpath, vn.flags)
+            except:
                 pass
 
-            untemp = []
             with self.mutex:
                 subs = self.busy[tpath]
                 del self.busy[tpath]
                 self.ram.pop(ttpath, None)
-                untemp = self.untemp.pop(ttpath, None) or []
-
-            for ap in untemp:
-                try:
-                    wunlink(self.log, ap, VF_CAREFUL)
-                except:
-                    pass
 
             for x in subs:
                 with x:
@@ -512,38 +440,35 @@ class ThumbSrv(object):
 
         return im
 
-    def conv_image_pil(self, im: "Image.Image", tpath: str, fmt: str, vn: VFS) -> None:
-        try:
-            im = self.fancy_pillow(im, fmt, vn)
-        except Exception as ex:
-            self.log("fancy_pillow {}".format(ex), "90")
-            im.thumbnail(self.getres(vn, fmt))
-
-        fmts = ["RGB", "L"]
-        args = {"quality": 40}
-
-        if tpath.endswith(".webp"):
-            # quality 80 = pillow-default
-            # quality 75 = ffmpeg-default
-            # method 0 = pillow-default, fast
-            # method 4 = ffmpeg-default
-            # method 6 = max, slow
-            fmts.extend(("RGBA", "LA"))
-            args["method"] = 6
-        else:
-            # default q = 75
-            args["progressive"] = True
-
-        if im.mode not in fmts:
-            # print("conv {}".format(im.mode))
-            im = im.convert("RGB")
-
-        im.save(tpath, **args)
-
     def conv_pil(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
         self.wait4ram(0.2, tpath)
         with Image.open(fsenc(abspath)) as im:
-            self.conv_image_pil(im, tpath, fmt, vn)
+            try:
+                im = self.fancy_pillow(im, fmt, vn)
+            except Exception as ex:
+                self.log("fancy_pillow {}".format(ex), "90")
+                im.thumbnail(self.getres(vn, fmt))
+
+            fmts = ["RGB", "L"]
+            args = {"quality": 40}
+
+            if tpath.endswith(".webp"):
+                # quality 80 = pillow-default
+                # quality 75 = ffmpeg-default
+                # method 0 = pillow-default, fast
+                # method 4 = ffmpeg-default
+                # method 6 = max, slow
+                fmts.extend(("RGBA", "LA"))
+                args["method"] = 6
+            else:
+                # default q = 75
+                args["progressive"] = True
+
+            if im.mode not in fmts:
+                # print("conv {}".format(im.mode))
+                im = im.convert("RGB")
+
+            im.save(tpath, **args)
 
     def conv_vips(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
         self.wait4ram(0.2, tpath)
@@ -566,53 +491,9 @@ class ThumbSrv(object):
         assert img  # type: ignore  # !rm
         img.write_to_file(tpath, Q=40)
 
-    def conv_raw(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        self.wait4ram(0.2, tpath)
-        with rawpy.imread(abspath) as raw:
-            thumb = raw.extract_thumb()
-        if thumb.format == rawpy.ThumbFormat.JPEG and tpath.endswith(".jpg"):
-            # if we have a jpg thumbnail and no webp output is available,
-            # just write the jpg directly (it'll be the wrong size, but it's fast)
-            with open(tpath, "wb") as f:
-                f.write(thumb.data)
-        if HAVE_VIPS:
-            crops = ["centre", "none"]
-            if "f" in fmt:
-                crops = ["none"]
-            w, h = self.getres(vn, fmt)
-            kw = {"height": h, "size": "down", "intent": "relative"}
-
-            for c in crops:
-                try:
-                    kw["crop"] = c
-                    if thumb.format == rawpy.ThumbFormat.BITMAP:
-                        img = pyvips.Image.new_from_array(
-                            thumb.data, interpretation="rgb"
-                        )
-                        img = img.thumbnail_image(w, **kw)
-                    else:
-                        img = pyvips.Image.thumbnail_buffer(thumb.data, w, **kw)
-                    break
-                except:
-                    if c == crops[-1]:
-                        raise
-
-            assert img  # type: ignore  # !rm
-            img.write_to_file(tpath, Q=40)
-        elif HAVE_PIL:
-            if thumb.format == rawpy.ThumbFormat.BITMAP:
-                im = Image.fromarray(thumb.data, "RGB")
-            else:
-                im = Image.open(io.BytesIO(thumb.data))
-            self.conv_image_pil(im, tpath, fmt, vn)
-        else:
-            raise Exception(
-                "either pil or vips is needed to process embedded bitmap thumbnails in raw files"
-            )
-
     def conv_ffmpeg(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
         self.wait4ram(0.2, tpath)
-        ret, _, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
         if not ret:
             return
 
@@ -623,17 +504,6 @@ class ThumbSrv(object):
             dur = ret[".dur"][1] if ".dur" in ret else 4
             seek = [b"-ss", "{:.0f}".format(dur / 3).encode("utf-8")]
 
-        self._ffmpeg_im(abspath, tpath, fmt, vn, seek, b"0:v:0")
-
-    def _ffmpeg_im(
-        self,
-        abspath: str,
-        tpath: str,
-        fmt: str,
-        vn: VFS,
-        seek: list[bytes],
-        imap: bytes,
-    ) -> None:
         scale = "scale={0}:{1}:force_original_aspect_ratio="
         if "f" in fmt:
             scale += "decrease,setsar=1:1"
@@ -652,7 +522,7 @@ class ThumbSrv(object):
         cmd += seek
         cmd += [
             b"-i", fsenc(abspath),
-            b"-map", imap,
+            b"-map", b"0:v:0",
             b"-vf", bscale,
             b"-frames:v", b"1",
             b"-metadata:s:v:0", b"rotate=0",
@@ -673,11 +543,11 @@ class ThumbSrv(object):
             ]
 
         cmd += [fsenc(tpath)]
-        self._run_ff(cmd, vn, "convt")
+        self._run_ff(cmd, vn)
 
-    def _run_ff(self, cmd: list[bytes], vn: VFS, kto: str, oom: int = 400) -> None:
+    def _run_ff(self, cmd: list[bytes], vn: VFS, oom: int = 400) -> None:
         # self.log((b" ".join(cmd)).decode("utf-8"))
-        ret, _, serr = runcmd(cmd, timeout=vn.flags[kto], nice=True, oom=oom)
+        ret, _, serr = runcmd(cmd, timeout=vn.flags["convt"], nice=True, oom=oom)
         if not ret:
             return
 
@@ -721,7 +591,7 @@ class ThumbSrv(object):
         raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
 
     def conv_waves(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        ret, _, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
         if "ac" not in ret:
             raise Exception("not audio")
 
@@ -759,7 +629,7 @@ class ThumbSrv(object):
         # fmt: on
 
         cmd += [fsenc(tpath)]
-        self._run_ff(cmd, vn, "convt")
+        self._run_ff(cmd, vn)
 
         if "pngquant" in vn.flags:
             wtpath = tpath + ".png"
@@ -778,70 +648,22 @@ class ThumbSrv(object):
                 except:
                     pass
             else:
-                atomic_move(self.log, wtpath, tpath, vn.flags)
-
-    def conv_emb_cv(
-        self, abspath: str, tpath: str, fmt: str, vn: VFS, strm: dict[str, Any]
-    ) -> None:
-        self.wait4ram(0.2, tpath)
-        self._ffmpeg_im(
-            abspath, tpath, fmt, vn, [], b"0:" + strm["index"].encode("ascii")
-        )
+                wrename(self.log, wtpath, tpath, vn.flags)
 
     def conv_spec(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        ret, raw, strms, ctnr = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
         if "ac" not in ret:
             raise Exception("not audio")
 
-        want_spec = vn.flags.get("th_spec_p", 1)
-        if want_spec < 2:
-            for strm in strms:
-                if (
-                    strm.get("codec_type") == "video"
-                    and strm.get("DISPOSITION:attached_pic") == "1"
-                ):
-                    return self.conv_emb_cv(abspath, tpath, fmt, vn, strm)
-
-        if not want_spec:
-            raise Exception("spectrograms forbidden by volflag")
-
-        fext = abspath.split(".")[-1].lower()
-
         # https://trac.ffmpeg.org/ticket/10797
         # expect 1 GiB every 600 seconds when duration is tricky;
         # simple filetypes are generally safer so let's special-case those
-        coeff = 1800 if fext in EXTS_SPEC_SAFE else 600
-        dur = ret[".dur"][1] if ".dur" in ret else 900
+        safe = ("flac", "wav", "aif", "aiff", "opus")
+        coeff = 1800 if abspath.split(".")[-1].lower() in safe else 600
+        dur = ret[".dur"][1] if ".dur" in ret else 300
         need = 0.2 + dur / coeff
         self.wait4ram(need, tpath)
 
-        infile = abspath
-        if dur >= 900 or fext in self.exts_spec_unsafe:
-            with tempfile.NamedTemporaryFile(suffix=".spec.flac", delete=False) as f:
-                f.write(b"h")
-                infile = f.name
-                try:
-                    self.untemp[tpath].append(infile)
-                except:
-                    self.untemp[tpath] = [infile]
-
-            # fmt: off
-            cmd = [
-                b"ffmpeg",
-                b"-nostdin",
-                b"-v", b"error",
-                b"-hide_banner",
-                b"-i", fsenc(abspath),
-                b"-map", b"0:a:0",
-                b"-ac", b"1",
-                b"-ar", b"48000",
-                b"-sample_fmt", b"s16",
-                b"-t", b"900",
-                b"-y", fsenc(infile),
-            ]
-            # fmt: on
-            self._run_ff(cmd, vn, "convt")
-
         fc = "[0:a:0]aresample=48000{},showspectrumpic=s="
         if "3" in fmt:
             fc += "1280x1024,crop=1420:1056:70:48[o]"
@@ -861,7 +683,7 @@ class ThumbSrv(object):
             b"-nostdin",
             b"-v", b"error",
             b"-hide_banner",
-            b"-i", fsenc(infile),
+            b"-i", fsenc(abspath),
             b"-filter_complex", fc.encode("utf-8"),
             b"-map", b"[o]",
             b"-frames:v", b"1",
@@ -882,7 +704,7 @@ class ThumbSrv(object):
             ]
 
         cmd += [fsenc(tpath)]
-        self._run_ff(cmd, vn, "convt")
+        self._run_ff(cmd, vn)
 
     def conv_mp3(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
         quality = self.args.q_mp3.lower()
@@ -890,7 +712,7 @@ class ThumbSrv(object):
             raise Exception("disabled in server config")
 
         self.wait4ram(0.2, tpath)
-        tags, rawtags, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
         if "ac" not in tags:
             raise Exception("not audio")
 
@@ -921,173 +743,58 @@ class ThumbSrv(object):
             fsenc(tpath)
         ]
         # fmt: on
-        self._run_ff(cmd, vn, "aconvt", oom=300)
-
-    def conv_flac(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        if self.args.no_acode or not self.args.allow_flac:
-            raise Exception("flac not permitted in server config")
-
-        self.wait4ram(0.2, tpath)
-        tags, _, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
-        if "ac" not in tags:
-            raise Exception("not audio")
-
-        self.log("conv2 flac", 6)
-
-        # fmt: off
-        cmd = [
-            b"ffmpeg",
-            b"-nostdin",
-            b"-v", b"error",
-            b"-hide_banner",
-            b"-i", fsenc(abspath),
-            b"-map", b"0:a:0",
-            b"-c:a", b"flac",
-            fsenc(tpath)
-        ]
-        # fmt: on
-        self._run_ff(cmd, vn, "aconvt", oom=300)
-
-    def conv_wav(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        if self.args.no_acode or not self.args.allow_wav:
-            raise Exception("wav not permitted in server config")
-
-        self.wait4ram(0.2, tpath)
-        tags, _, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
-        if "ac" not in tags:
-            raise Exception("not audio")
-
-        bits = tags[".bps"][1]
-        if bits == 0.0:
-            bits = tags[".bprs"][1]
-
-        codec = b"pcm_s32le"
-        if bits <= 16.0:
-            codec = b"pcm_s16le"
-        elif bits <= 24.0:
-            codec = b"pcm_s24le"
-
-        self.log("conv2 wav", 6)
-
-        # fmt: off
-        cmd = [
-            b"ffmpeg",
-            b"-nostdin",
-            b"-v", b"error",
-            b"-hide_banner",
-            b"-i", fsenc(abspath),
-            b"-map", b"0:a:0",
-            b"-c:a", codec,
-            fsenc(tpath)
-        ]
-        # fmt: on
-        self._run_ff(cmd, vn, "aconvt", oom=300)
+        self._run_ff(cmd, vn, oom=300)
 
     def conv_opus(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
         if self.args.no_acode or not self.args.q_opus:
             raise Exception("disabled in server config")
 
         self.wait4ram(0.2, tpath)
-        tags, rawtags, _, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
         if "ac" not in tags:
             raise Exception("not audio")
 
-        sq = "%dk" % (self.args.q_opus,)
-        bq = sq.encode("ascii")
-        if tags["ac"][1] == "opus":
-            enc = "-c:a copy"
-        else:
-            enc = "-c:a libopus -b:a " + sq
-
-        fun = self._conv_caf if fmt == "caf" else self._conv_owa
-
-        fun(abspath, tpath, tags, rawtags, enc, bq, vn)
-
-    def _conv_owa(
-        self,
-        abspath: str,
-        tpath: str,
-        tags: dict[str, tuple[int, Any]],
-        rawtags: dict[str, list[Any]],
-        enc: str,
-        bq: bytes,
-        vn: VFS,
-    ) -> None:
-        if tpath.endswith(".owa"):
-            container = b"webm"
-            tagset = [b"-map_metadata", b"-1"]
-        else:
-            container = b"opus"
-            tagset = self.big_tags(rawtags)
-
-        self.log("conv2 %s [%s]" % (container, enc), 6)
-        benc = enc.encode("ascii").split(b" ")
-
-        # fmt: off
-        cmd = [
-            b"ffmpeg",
-            b"-nostdin",
-            b"-v", b"error",
-            b"-hide_banner",
-            b"-i", fsenc(abspath),
-        ] + tagset + [
-            b"-map", b"0:a:0",
-        ] + benc + [
-            b"-f", container,
-            fsenc(tpath)
-        ]
-        # fmt: on
-        self._run_ff(cmd, vn, "aconvt", oom=300)
-
-    def _conv_caf(
-        self,
-        abspath: str,
-        tpath: str,
-        tags: dict[str, tuple[int, Any]],
-        rawtags: dict[str, list[Any]],
-        enc: str,
-        bq: bytes,
-        vn: VFS,
-    ) -> None:
-        tmp_opus = tpath + ".opus"
-        try:
-            wunlink(self.log, tmp_opus, vn.flags)
-        except:
-            pass
-
         try:
             dur = tags[".dur"][1]
         except:
             dur = 0
 
-        self.log("conv2 caf-tmp [%s]" % (enc,), 6)
-        benc = enc.encode("ascii").split(b" ")
+        src_opus = abspath.lower().endswith(".opus") or tags["ac"][1] == "opus"
+        want_caf = tpath.endswith(".caf")
+        tmp_opus = tpath
+        if want_caf:
+            tmp_opus = tpath + ".opus"
+            try:
+                wunlink(self.log, tmp_opus, vn.flags)
+            except:
+                pass
 
-        # fmt: off
-        cmd = [
-            b"ffmpeg",
-            b"-nostdin",
-            b"-v", b"error",
-            b"-hide_banner",
-            b"-i", fsenc(abspath),
-            b"-map_metadata", b"-1",
-            b"-map", b"0:a:0",
-        ] + benc + [
-            b"-f", b"opus",
-            fsenc(tmp_opus)
-        ]
-        # fmt: on
-        self._run_ff(cmd, vn, "aconvt", oom=300)
+        caf_src = abspath if src_opus else tmp_opus
+        bq = ("%dk" % (self.args.q_opus,)).encode("ascii")
+
+        if not want_caf or not src_opus:
+            # fmt: off
+            cmd = [
+                b"ffmpeg",
+                b"-nostdin",
+                b"-v", b"error",
+                b"-hide_banner",
+                b"-i", fsenc(abspath),
+            ] + self.big_tags(rawtags) + [
+                b"-map", b"0:a:0",
+                b"-c:a", b"libopus",
+                b"-b:a", bq,
+                fsenc(tmp_opus)
+            ]
+            # fmt: on
+            self._run_ff(cmd, vn, oom=300)
 
         # iOS fails to play some "insufficiently complex" files
         # (average file shorter than 8 seconds), so of course we
         # fix that by mixing in some inaudible pink noise :^)
         # 6.3 sec seems like the cutoff so lets do 7, and
         # 7 sec of psyqui-musou.opus @ 3:50 is 174 KiB
-        sz = bos.path.getsize(tmp_opus)
-        if dur < 20 or sz < 256 * 1024:
-            zs = bq.decode("ascii")
-            self.log("conv2 caf-transcode; dur=%d sz=%d q=%s" % (dur, sz, zs), 6)
+        if want_caf and (dur < 20 or bos.path.getsize(caf_src) < 256 * 1024):
             # fmt: off
             cmd = [
                 b"ffmpeg",
@@ -1104,18 +811,17 @@ class ThumbSrv(object):
                 fsenc(tpath)
             ]
             # fmt: on
-            self._run_ff(cmd, vn, "aconvt", oom=300)
+            self._run_ff(cmd, vn, oom=300)
 
-        else:
+        elif want_caf:
             # simple remux should be safe
-            self.log("conv2 caf-remux; dur=%d sz=%d" % (dur, sz), 6)
             # fmt: off
             cmd = [
                 b"ffmpeg",
                 b"-nostdin",
                 b"-v", b"error",
                 b"-hide_banner",
-                b"-i", fsenc(tmp_opus),
+                b"-i", fsenc(abspath if src_opus else tmp_opus),
                 b"-map_metadata", b"-1",
                 b"-map", b"0:a:0",
                 b"-c:a", b"copy",
@@ -1123,12 +829,13 @@ class ThumbSrv(object):
                 fsenc(tpath)
             ]
             # fmt: on
-            self._run_ff(cmd, vn, "aconvt", oom=300)
+            self._run_ff(cmd, vn, oom=300)
 
-        try:
-            wunlink(self.log, tmp_opus, vn.flags)
-        except:
-            pass
+        if tmp_opus != tpath:
+            try:
+                wunlink(self.log, tmp_opus, vn.flags)
+            except:
+                pass
 
     def big_tags(self, raw_tags: dict[str, list[str]]) -> list[bytes]:
         ret = []
@@ -1184,7 +891,7 @@ class ThumbSrv(object):
 
     def _clean(self, cat: str, thumbpath: str) -> int:
         # self.log("cln {}".format(thumbpath))
-        exts = EXTS_TH if cat == "th" else EXTS_AC
+        exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf", "mp3"]
         maxage = getattr(self.args, cat + "_maxage")
         now = time.time()
         prev_b64 = None
@@ -1225,8 +932,6 @@ class ThumbSrv(object):
             # thumb file
             try:
                 b64, ts, ext = f.split(".")
-                if len(ts) > 8 and PTN_TS.match(ts):
-                    ts = "yeahokay"
                 if len(b64) != 24 or len(ts) != 8 or ext not in exts:
                     raise Exception()
             except:

copyparty/u2idx.py

@@ -134,9 +134,9 @@ class U2idx(object):
         assert sqlite3  # type: ignore  # !rm
 
         ptop = vn.realpath
-        histpath = self.asrv.vfs.dbpaths.get(ptop)
+        histpath = self.asrv.vfs.histtab.get(ptop)
         if not histpath:
-            self.log("no dbpath for %r" % (ptop,))
+            self.log("no histpath for %r" % (ptop,))
             return None
 
         db_path = os.path.join(histpath, "up2k.db")
@@ -391,7 +391,7 @@ class U2idx(object):
             fk_alg = 2 if "fka" in flags else 1
             c = cur.execute(uq, tuple(vuv))
             for hit in c:
-                w, ts, sz, rd, fn = hit[:5]
+                w, ts, sz, rd, fn, ip, at = hit[:7]
 
                 if rd.startswith("//") or fn.startswith("//"):
                     rd, fn = s3dec(rd, fn)

copyparty/up2k.py

@@ -2,6 +2,7 @@
 from __future__ import print_function, unicode_literals
 
 import errno
+import gzip
 import hashlib
 import json
 import math
@@ -41,7 +42,6 @@ from .util import (
     fsenc,
     gen_filekey,
     gen_filekey_dbg,
-    gzip,
     hidedir,
     humansize,
     min_ex,
@@ -77,7 +77,7 @@ except:
 if HAVE_SQLITE3:
     import sqlite3
 
-DB_VER = 6
+DB_VER = 5
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Optional, Pattern, Union
@@ -86,10 +86,7 @@ if TYPE_CHECKING:
     from .svchub import SvcHub
 
 zsg = "avif,avifs,bmp,gif,heic,heics,heif,heifs,ico,j2p,j2k,jp2,jpeg,jpg,jpx,png,tga,tif,tiff,webp"
-ICV_EXTS = set(zsg.split(","))
-
-zsg = "3gp,asf,av1,avc,avi,flv,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,vob,webm,wmv"
-VCV_EXTS = set(zsg.split(","))
+CV_EXTS = set(zsg.split(","))
 
 zsg = "nohash noidx xdev xvol"
 VF_AFFECTS_INDEXING = set(zsg.split(" "))
@@ -97,7 +94,7 @@ VF_AFFECTS_INDEXING = set(zsg.split(" "))
 
 SBUSY = "cannot receive uploads right now;\nserver busy with %s.\nPlease wait; the client will retry..."
 
-HINT_HISTPATH = "you could try moving the database to another location (preferably an SSD or NVME drive) using either the --hist argument (global option for all volumes), or the hist volflag (just for this volume), or, if you want to keep the thumbnails in the current location and only move the database itself, then use --dbpath or volflag dbpath"
+HINT_HISTPATH = "you could try moving the database to another location (preferably an SSD or NVME drive) using either the --hist argument (global option for all volumes), or the hist volflag (just for this volume)"
 
 
 NULLSTAT = os.stat_result((0, -1, -1, 0, 0, 0, 0, 0, 0, 0))
@@ -144,7 +141,6 @@ class Up2k(object):
 
         self.salt = self.args.warksalt
         self.r_hash = re.compile("^[0-9a-zA-Z_-]{44}$")
-        self.abrt_key = ""
 
         self.gid = 0
         self.gt0 = 0
@@ -376,12 +372,11 @@ class Up2k(object):
                 if ineed == ihash or not ineed:
                     continue
 
-                poke = job["poke"]
                 zt = (
                     ineed / ihash,
                     job["size"],
-                    int(job.get("t0c", poke)),
-                    int(poke),
+                    int(job["t0c"]),
+                    int(job["poke"]),
                     djoin(vtop, job["prel"], job["name"]),
                 )
                 ret.append(zt)
@@ -404,14 +399,12 @@ class Up2k(object):
 
         return "{}"
 
-    def get_unfinished_by_user(self, uname, ip) -> dict[str, Any]:
-        # returns dict due to ExceptionalQueue
+    def get_unfinished_by_user(self, uname, ip) -> str:
         if PY2 or not self.reg_mutex.acquire(timeout=2):
-            return {"timeout": 1}
+            return '[{"timeout":1}]'
 
         ret: list[tuple[int, str, int, int, int]] = []
         userset = set([(uname or "\n"), "*"])
-        n = 1000
         try:
             for ptop, tab2 in self.registry.items():
                 cfg = self.flags.get(ptop, {}).get("u2abort", 1)
@@ -426,6 +419,7 @@ class Up2k(object):
                         or (addr and addr != job["addr"])
                     ):
                         continue
+
                     zt5 = (
                         int(job["t0"]),
                         djoin(job["vtop"], job["prel"], job["name"]),
@@ -434,9 +428,6 @@ class Up2k(object):
                         len(job["hash"]),
                     )
                     ret.append(zt5)
-                    n -= 1
-                    if not n:
-                        break
         finally:
             self.reg_mutex.release()
 
@@ -453,7 +444,7 @@ class Up2k(object):
             }
             for (at, vp, sz, nn, nh) in ret
         ]
-        return {"f": ret2}
+        return json.dumps(ret2, separators=(",\n", ": "))
 
     def get_unfinished(self) -> str:
         if PY2 or not self.reg_mutex.acquire(timeout=0.5):
@@ -566,7 +557,6 @@ class Up2k(object):
             else:
                 # important; not deferred by db_act
                 timeout = self._check_lifetimes()
-                timeout = min(self._check_forget_ip(), timeout)
                 try:
                     if self.args.shr:
                         timeout = min(self._check_shares(), timeout)
@@ -627,43 +617,6 @@ class Up2k(object):
                 for v in vols:
                     volage[v] = now
 
-    def _check_forget_ip(self) -> float:
-        now = time.time()
-        timeout = now + 9001
-        for vp, vol in sorted(self.vfs.all_vols.items()):
-            maxage = vol.flags["forget_ip"]
-            if not maxage:
-                continue
-
-            cur = self.cur.get(vol.realpath)
-            if not cur:
-                continue
-
-            cutoff = now - maxage * 60
-
-            for _ in range(2):
-                q = "select ip, at from up where ip > '' order by +at limit 1"
-                hits = cur.execute(q).fetchall()
-                if not hits:
-                    break
-
-                remains = hits[0][1] - cutoff
-                if remains > 0:
-                    timeout = min(timeout, now + remains)
-                    break
-
-                q = "update up set ip = '' where ip > '' and at <= %d"
-                cur.execute(q % (cutoff,))
-                zi = cur.rowcount
-                cur.connection.commit()
-
-                t = "forget-ip(%d) removed %d IPs from db [/%s]"
-                self.log(t % (maxage, zi, vol.vpath))
-
-            timeout = min(timeout, now + 900)
-
-        return timeout
-
     def _check_lifetimes(self) -> float:
         now = time.time()
         timeout = now + 9001
@@ -842,7 +795,7 @@ class Up2k(object):
                 continue
 
             self.log("xiu: %d# %r" % (len(wrfs), cmd))
-            runihook(self.log, self.args.hook_v, cmd, vol, ups)
+            runihook(self.log, cmd, vol, ups)
 
     def _vis_job_progress(self, job: dict[str, Any]) -> str:
         perc = 100 - (len(job["need"]) * 100.0 / (len(job["hash"]) or 1))
@@ -903,9 +856,9 @@ class Up2k(object):
         self.iacct = self.asrv.iacct
         self.grps = self.asrv.grps
 
-        have_e2d = self.args.have_idp_hdrs or self.args.chpw or self.args.shr
         vols = list(all_vols.values())
         t0 = time.time()
+        have_e2d = False
 
         if self.no_expr_idx:
             modified = False
@@ -924,8 +877,7 @@ class Up2k(object):
             # only need to protect register_vpath but all in one go feels right
             for vol in vols:
                 try:
-                    # mkdir gonna happen at snap anyways;
-                    bos.makedirs(vol.realpath, vf=vol.flags)
+                    bos.makedirs(vol.realpath)  # gonna happen at snap anyways
                     dir_is_empty(self.log_func, not self.args.no_scandir, vol.realpath)
                 except Exception as ex:
                     self.volstate[vol.vpath] = "OFFLINE (cannot access folder)"
@@ -1106,9 +1058,9 @@ class Up2k(object):
         self, ptop: str, flags: dict[str, Any]
     ) -> Optional[tuple["sqlite3.Cursor", str]]:
         """mutex(main,reg) me"""
-        histpath = self.vfs.dbpaths.get(ptop)
+        histpath = self.vfs.histtab.get(ptop)
         if not histpath:
-            self.log("no dbpath for %r" % (ptop,))
+            self.log("no histpath for %r" % (ptop,))
             return None
 
         db_path = os.path.join(histpath, "up2k.db")
@@ -1129,7 +1081,7 @@ class Up2k(object):
         ft = "\033[0;32m{}{:.0}"
         ff = "\033[0;35m{}{:.0}"
         fv = "\033[0;36m{}:\033[90m{}"
-        zs = "ext_th_d html_head put_name2 mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot zipmax zipmaxn_v zipmaxs_v"
+        zs = "html_head mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot"
         fx = set(zs.split())
         fd = vf_bmap()
         fd.update(vf_cmap())
@@ -1151,20 +1103,6 @@ class Up2k(object):
                 del fl[k1]
             else:
                 fl[k1] = ",".join(x for x in fl[k1])
-
-        if fl["chmod_d"] == int(self.args.chmod_d, 8):
-            fl.pop("chmod_d")
-        try:
-            if fl["chmod_f"] == int(self.args.chmod_f or "-1", 8):
-                fl.pop("chmod_f")
-        except:
-            pass
-        for k in ("chmod_f", "chmod_d"):
-            try:
-                fl[k] = "%o" % (fl[k])
-            except:
-                pass
-
         a = [
             (ft if v is True else ff if v is False else fv).format(k, str(v))
             for k, v in fl.items()
@@ -1181,7 +1119,6 @@ class Up2k(object):
         reg = {}
         drp = None
         emptylist = []
-        dotpart = "." if self.args.dotpart else ""
         snap = os.path.join(histpath, "up2k.snap")
         if bos.path.exists(snap):
             with gzip.GzipFile(snap, "rb") as f:
@@ -1194,8 +1131,6 @@ class Up2k(object):
             except:
                 pass
 
-            reg = reg2  # diff-golf
-
             if reg2 and "dwrk" not in reg2[next(iter(reg2))]:
                 for job in reg2.values():
                     job["dwrk"] = job["wark"]
@@ -1203,8 +1138,7 @@ class Up2k(object):
             rm = []
             for k, job in reg2.items():
                 job["ptop"] = ptop
-                is_done = "done" in job
-                if is_done:
+                if "done" in job:
                     job["need"] = job["hash"] = emptylist
                 else:
                     if "need" not in job:
@@ -1212,13 +1146,10 @@ class Up2k(object):
                     if "hash" not in job:
                         job["hash"] = []
 
-                if is_done:
-                    fp = djoin(ptop, job["prel"], job["name"])
-                else:
-                    fp = djoin(ptop, job["prel"], dotpart + job["name"] + ".PARTIAL")
-
+                fp = djoin(ptop, job["prel"], job["name"])
                 if bos.path.exists(fp):
-                    if is_done:
+                    reg[k] = job
+                    if "done" in job:
                         continue
                     job["poke"] = time.time()
                     job["busy"] = {}
@@ -1226,18 +1157,11 @@ class Up2k(object):
                     self.log("ign deleted file in snap: %r" % (fp,))
                     if not n4g:
                         rm.append(k)
+                        continue
 
             for x in rm:
                 del reg2[x]
 
-            # optimize pre-1.15.4 entries
-            if next((x for x in reg.values() if "done" in x and "poke" in x), None):
-                zsl = "host tnam busy sprs poke t0c".split()
-                for job in reg.values():
-                    if "done" in job:
-                        for k in zsl:
-                            job.pop(k, None)
-
             if drp is None:
                 drp = [k for k, v in reg.items() if not v["need"]]
             else:
@@ -1368,15 +1292,12 @@ class Up2k(object):
             ]
             excl += [absreal(x) for x in excl]
             excl += list(self.vfs.histtab.values())
-            excl += list(self.vfs.dbpaths.values())
             if WINDOWS:
                 excl = [x.replace("/", "\\") for x in excl]
             else:
                 # ~/.wine/dosdevices/z:/ and such
                 excl.extend(("/dev", "/proc", "/run", "/sys"))
 
-            excl = list({k: 1 for k in excl})
-
             if self.args.re_dirsz:
                 db.c.execute("delete from ds")
                 db.n += 1
@@ -1388,10 +1309,6 @@ class Up2k(object):
                     t = "volume /%s at [%s] is empty; will not be indexed as this could be due to an offline filesystem"
                     self.log(t % (vol.vpath, rtop), 6)
                     return True, False
-                if not vol.check_landmarks():
-                    t = "volume /%s at [%s] will not be indexed due to bad landmarks"
-                    self.log(t % (vol.vpath, rtop), 6)
-                    return True, False
 
                 n_add, _, _ = self._build_dir(
                     db,
@@ -1483,7 +1400,7 @@ class Up2k(object):
         unreg: list[str] = []
         files: list[tuple[int, int, str]] = []
         fat32 = True
-        cv = vcv = ""
+        cv = ""
 
         th_cvd = self.args.th_coversd
         th_cvds = self.args.th_coversd_set
@@ -1578,24 +1495,25 @@ class Up2k(object):
 
                 rsz += sz
                 files.append((sz, lmod, iname))
-                if sz:
-                    liname = iname.lower()
-                    ext = liname.rsplit(".", 1)[-1]
-                    if (
+                liname = iname.lower()
+                if (
+                    sz
+                    and (
                         liname in th_cvds
-                        or (not cv and ext in ICV_EXTS and not iname.startswith("."))
-                    ) and (
+                        or (
+                            not cv
+                            and liname.rsplit(".", 1)[-1] in CV_EXTS
+                            and not iname.startswith(".")
+                        )
+                    )
+                    and (
                         not cv
                         or liname not in th_cvds
                         or cv.lower() not in th_cvds
                         or th_cvd.index(liname) < th_cvd.index(cv.lower())
-                    ):
-                        cv = iname
-                    elif not vcv and ext in VCV_EXTS and not iname.startswith("."):
-                        vcv = iname
-
-        if not cv:
-            cv = vcv
+                    )
+                ):
+                    cv = iname
 
         if not self.args.no_dirsz:
             tnf += len(files)
@@ -1655,7 +1573,7 @@ class Up2k(object):
             abspath = cdirs + fn
             nohash = reh.search(abspath) if reh else False
 
-            sql = "select w, mt, sz, ip, at, un from up where rd = ? and fn = ?"
+            sql = "select w, mt, sz, ip, at from up where rd = ? and fn = ?"
             try:
                 c = db.c.execute(sql, (rd, fn))
             except:
@@ -1664,7 +1582,7 @@ class Up2k(object):
             in_db = list(c.fetchall())
             if in_db:
                 self.pp.n -= 1
-                dw, dts, dsz, ip, at, un = in_db[0]
+                dw, dts, dsz, ip, at = in_db[0]
                 if len(in_db) > 1:
                     t = "WARN: multiple entries: %r => %r |%d|\n%r"
                     rep_db = "\n".join([repr(x) for x in in_db])
@@ -1677,9 +1595,6 @@ class Up2k(object):
                 if dts == lmod and dsz == sz and (nohash or dw[0] != "#" or not sz):
                     continue
 
-                if un is None:
-                    un = ""
-
                 t = "reindex %r => %r mtime(%s/%s) size(%s/%s)"
                 self.log(t % (top, rp, dts, lmod, dsz, sz))
                 self.db_rm(db.c, rd, fn, 0)
@@ -1690,7 +1605,6 @@ class Up2k(object):
                 dw = ""
                 ip = ""
                 at = 0
-                un = ""
 
             self.pp.msg = "a%d %s" % (self.pp.n, abspath)
 
@@ -1716,10 +1630,9 @@ class Up2k(object):
             if dw and dw != wark:
                 ip = ""
                 at = 0
-                un = ""
 
             # skip upload hooks by not providing vflags
-            self.db_add(db.c, {}, rd, fn, lmod, sz, "", "", wark, wark, "", un, ip, at)
+            self.db_add(db.c, {}, rd, fn, lmod, sz, "", "", wark, wark, "", "", ip, at)
             db.n += 1
             db.nf += 1
             tfa += 1
@@ -2152,12 +2065,11 @@ class Up2k(object):
                 return -1
 
             w = bw[:-1].decode("ascii")
-            w16 = w[:16]
 
             with self.mutex:
                 try:
-                    q = "select rd, fn, ip, at, un from up where substr(w,1,16)=? and +w=?"
-                    rd, fn, ip, at, un = cur.execute(q, (w16, w)).fetchone()
+                    q = "select rd, fn, ip, at from up where substr(w,1,16)=? and +w=?"
+                    rd, fn, ip, at = cur.execute(q, (w[:16], w)).fetchone()
                 except:
                     # file modified/deleted since spooling
                     continue
@@ -2166,25 +2078,18 @@ class Up2k(object):
                     rd, fn = s3dec(rd, fn)
 
                 if "mtp" in flags:
-                    q = "select 1 from mt where w=? and +k='t:mtp' limit 1"
-                    if cur.execute(q, (w16,)).fetchone():
-                        continue
-
                     q = "insert into mt values (?,'t:mtp','a')"
-                    cur.execute(q, (w16,))
+                    cur.execute(q, (w[:16],))
 
             abspath = djoin(ptop, rd, fn)
             self.pp.msg = "c%d %s" % (nq, abspath)
             if not mpool:
-                n_tags = self._tagscan_file(cur, entags, w, abspath, ip, at, un)
+                n_tags = self._tagscan_file(cur, entags, w, abspath, ip, at)
             else:
-                oth_tags = {}
                 if ip:
-                    oth_tags["up_ip"] = ip
-                if at:
-                    oth_tags["up_at"] = at
-                if un:
-                    oth_tags["up_by"] = un
+                    oth_tags = {"up_ip": ip, "up_at": at}
+                else:
+                    oth_tags = {}
 
                 mpool.put(Mpqe({}, entags, w, abspath, oth_tags))
                 with self.mutex:
@@ -2230,7 +2135,7 @@ class Up2k(object):
                     return tf, -1
 
                 if flt == 1:
-                    q = "select 1 from mt where w=? and +k != 't:mtp'"
+                    q = "select w from mt where w = ?"
                     if c2.execute(q, (row[0][:16],)).fetchone():
                         continue
 
@@ -2340,8 +2245,8 @@ class Up2k(object):
                     if w in in_progress:
                         continue
 
-                    q = "select rd, fn, ip, at, un from up where substr(w,1,16)=? limit 1"
-                    rd, fn, ip, at, un = cur.execute(q, (w,)).fetchone()
+                    q = "select rd, fn, ip, at from up where substr(w,1,16)=? limit 1"
+                    rd, fn, ip, at = cur.execute(q, (w,)).fetchone()
                     rd, fn = s3dec(rd, fn)
                     abspath = djoin(ptop, rd, fn)
 
@@ -2365,10 +2270,7 @@ class Up2k(object):
 
                     if ip:
                         oth_tags["up_ip"] = ip
-                    if at:
                         oth_tags["up_at"] = at
-                    if un:
-                        oth_tags["up_by"] = un
 
                     jobs.append(Mpqe(parsers, set(), w, abspath, oth_tags))
                     in_progress[w] = True
@@ -2557,7 +2459,6 @@ class Up2k(object):
         abspath: str,
         ip: str,
         at: float,
-        un: Optional[str],
     ) -> int:
         """will mutex(main)"""
         assert self.mtag  # !rm
@@ -2578,10 +2479,7 @@ class Up2k(object):
 
         if ip:
             tags["up_ip"] = ip
-        if at:
             tags["up_at"] = at
-        if un:
-            tags["up_by"] = un
 
         with self.mutex:
             return self._tag_file(write_cur, entags, wark, abspath, tags)
@@ -2685,19 +2583,16 @@ class Up2k(object):
         if not existed and ver is None:
             return self._try_create_db(db_path, cur)
 
-        for upver in (4, 5):
-            if ver != upver:
-                continue
+        if ver == 4:
             try:
                 t = "creating backup before upgrade: "
                 cur = self._backup_db(db_path, cur, ver, t)
-                getattr(self, "_upgrade_v%d" % (upver,))(cur)
-                ver += 1  # type: ignore
+                self._upgrade_v4(cur)
+                ver = 5
             except:
-                self.log("WARN: failed to upgrade from v%d" % (ver,), 3)
+                self.log("WARN: failed to upgrade from v4", 3)
 
         if ver == DB_VER:
-            # these no longer serve their intended purpose but they're great as additional sanchks
             self._add_dhash_tab(cur)
             self._add_xiu_tab(cur)
             self._add_cv_tab(cur)
@@ -2799,7 +2694,7 @@ class Up2k(object):
             idx = r"create index up_w on up(w)"
 
         for cmd in [
-            r"create table up (w text, mt int, sz int, rd text, fn text, ip text, at int, un text)",
+            r"create table up (w text, mt int, sz int, rd text, fn text, ip text, at int)",
             r"create index up_vp on up(rd, fn)",
             r"create index up_fn on up(fn)",
             r"create index up_ip on up(ip)",
@@ -2832,15 +2727,6 @@ class Up2k(object):
 
         cur.connection.commit()
 
-    def _upgrade_v5(self, cur: "sqlite3.Cursor") -> None:
-        for cmd in [
-            r"alter table up add column un text",
-            r"update kv set v=6 where k='sver'",
-        ]:
-            cur.execute(cmd)
-
-        cur.connection.commit()
-
     def _add_dhash_tab(self, cur: "sqlite3.Cursor") -> None:
         # v5 -> v5a
         try:
@@ -2862,7 +2748,7 @@ class Up2k(object):
         # v5a -> v5b
         # store rd+fn rather than warks to support nohash vols
         try:
-            cur.execute("select c, w, rd, fn from iu limit 1").fetchone()
+            cur.execute("select ws, rd, fn from iu limit 1").fetchone()
             return
         except:
             pass
@@ -2980,6 +2866,7 @@ class Up2k(object):
             if ptop not in self.registry:
                 raise Pebkac(410, "location unavailable")
 
+        cj["name"] = sanitize_fn(cj["name"], "")
         cj["poke"] = now = self.db_act = self.vol_act[ptop] = time.time()
         wark = dwark = self._get_wark(cj)
         job = None
@@ -3015,14 +2902,9 @@ class Up2k(object):
                     self.salt, cj["size"], cj["lmod"], cj["prel"], cj["name"]
                 )
 
-            zi = cj["lmod"]
-            bad_mt = zi <= 0 or zi > 0xAAAAAAAA
-            if bad_mt or vfs.flags.get("up_ts", "") == "fu":
+            if vfs.flags.get("up_ts", "") == "fu" or not cj["lmod"]:
                 # force upload time rather than last-modified
                 cj["lmod"] = int(time.time())
-                if zi and bad_mt:
-                    t = "ignoring impossible last-modified time from client: %s"
-                    self.log(t % (zi,), 6)
 
             alts: list[tuple[int, int, dict[str, Any], "sqlite3.Cursor", str, str]] = []
             for ptop, cur in vols:
@@ -3038,7 +2920,7 @@ class Up2k(object):
                     argv = [dwark[:16], dwark]
 
                 c2 = cur.execute(q, tuple(argv))
-                for _, dtime, dsize, dp_dir, dp_fn, ip, at, _ in c2:
+                for _, dtime, dsize, dp_dir, dp_fn, ip, at in c2:
                     if dp_dir.startswith("//") or dp_fn.startswith("//"):
                         dp_dir, dp_fn = s3dec(dp_dir, dp_fn)
 
@@ -3122,7 +3004,7 @@ class Up2k(object):
                 if wark in reg:
                     del reg[wark]
                 job["hash"] = job["need"] = []
-                job["done"] = 1
+                job["done"] = True
                 job["busy"] = {}
 
             if lost:
@@ -3290,16 +3172,14 @@ class Up2k(object):
                             if hr.get("reloc"):
                                 x = pathmod(self.vfs, dst, vp, hr["reloc"])
                                 if x:
-                                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                                    zvfs = vfs
                                     pdir, _, job["name"], (vfs, rem) = x
                                     dst = os.path.join(pdir, job["name"])
                                     job["vcfg"] = vfs.flags
                                     job["ptop"] = vfs.realpath
                                     job["vtop"] = vfs.vpath
                                     job["prel"] = rem
-                                    job["name"] = sanitize_fn(job["name"], "")
-                                    ud2 = (vfs.vpath, job["prel"], job["name"])
-                                    if ud1 != ud2:
+                                    if zvfs.vpath != vfs.vpath:
                                         # print(json.dumps(job, sort_keys=True, indent=4))
                                         job["hash"] = cj["hash"]
                                         self.log("xbu reloc1:%d..." % (depth,), 6)
@@ -3344,7 +3224,7 @@ class Up2k(object):
                         reg,
                         "up2k._get_volsize",
                     )
-                    bos.makedirs(ap2, vf=vfs.flags)
+                    bos.makedirs(ap2)
                     vfs.lim.nup(cj["addr"])
                     vfs.lim.bup(cj["addr"], cj["size"])
 
@@ -3441,26 +3321,16 @@ class Up2k(object):
             return fname
 
         fp = djoin(fdir, fname)
-
-        ow = job.get("replace") and bos.path.exists(fp)
-        if ow and "mt" in str(job["replace"]).lower():
-            mts = bos.stat(fp).st_mtime
-            mtc = job["lmod"]
-            if mtc < mts:
-                t = "will not overwrite; server %d sec newer than client; %d > %d %r"
-                self.log(t % (mts - mtc, mts, mtc, fp))
-                ow = False
-
-        ptop = job["ptop"]
-        vf = self.flags.get(ptop) or {}
-        if ow:
+        if job.get("replace") and bos.path.exists(fp):
             self.log("replacing existing file at %r" % (fp,))
             cur = None
+            ptop = job["ptop"]
+            vf = self.flags.get(ptop) or {}
             st = bos.stat(fp)
             try:
                 vrel = vjoin(job["prel"], fname)
                 xlink = bool(vf.get("xlink"))
-                cur, wark, _, _, _, _, _ = self._find_from_vpath(ptop, vrel)
+                cur, wark, _, _, _, _ = self._find_from_vpath(ptop, vrel)
                 self._forget_file(ptop, vrel, cur, wark, True, st.st_size, xlink)
             except Exception as ex:
                 self.log("skipping replace-relink: %r" % (ex,))
@@ -3475,13 +3345,8 @@ class Up2k(object):
         else:
             dip = self.hub.iphash.s(ip)
 
-        f, ret = ren_open(
-            fname,
-            "wb",
-            fdir=fdir,
-            suffix="-%.6f-%s" % (ts, dip),
-            vf=vf,
-        )
+        suffix = "-%.6f-%s" % (ts, dip)
+        f, ret = ren_open(fname, "wb", fdir=fdir, suffix=suffix)
         f.close()
         return ret
 
@@ -3494,7 +3359,6 @@ class Up2k(object):
         rm: bool = False,
         lmod: float = 0,
         fsrc: Optional[str] = None,
-        is_mv: bool = False,
     ) -> None:
         if src == dst or (fsrc and fsrc == dst):
             t = "symlinking a file to itself?? orig(%s) fsrc(%s) link(%s)"
@@ -3511,9 +3375,7 @@ class Up2k(object):
 
         linked = False
         try:
-            if "reflink" in flags:
-                raise Exception("reflink")
-            if not is_mv and not flags.get("dedup"):
+            if not flags.get("dedup"):
                 raise Exception("dedup is disabled in config")
 
             lsrc = src
@@ -3569,8 +3431,7 @@ class Up2k(object):
 
                 linked = True
         except Exception as ex:
-            if str(ex) != "reflink":
-                self.log("cannot link; creating copy: " + repr(ex))
+            self.log("cannot link; creating copy: " + repr(ex))
             if bos.path.isfile(src):
                 csrc = src
             elif fsrc and bos.path.isfile(fsrc):
@@ -3780,9 +3641,8 @@ class Up2k(object):
         if self.idx_wark(vflags, *z2):
             del self.registry[ptop][wark]
         else:
-            for k in "host tnam busy sprs poke".split():
+            for k in "host tnam busy sprs poke t0c".split():
                 del job[k]
-            job.pop("t0c", None)
             job["t0"] = int(job["t0"])
             job["hash"] = []
             job["done"] = 1
@@ -3915,16 +3775,16 @@ class Up2k(object):
             db_ip = ""
         else:
             # plugins may expect this to look like an actual IP
-            db_ip = "1.1.1.1" if "no_db_ip" in vflags else ip
+            db_ip = "1.1.1.1" if self.args.no_db_ip else ip
 
-        sql = "insert into up values (?,?,?,?,?,?,?,?)"
-        v = (dwark, int(ts), sz, rd, fn, db_ip, int(at or 0), usr)
+        sql = "insert into up values (?,?,?,?,?,?,?)"
+        v = (dwark, int(ts), sz, rd, fn, db_ip, int(at or 0))
         try:
             db.execute(sql, v)
         except:
             assert self.mem_cur  # !rm
             rd, fn = s3enc(self.mem_cur, rd, fn)
-            v = (dwark, int(ts), sz, rd, fn, db_ip, int(at or 0), usr)
+            v = (dwark, int(ts), sz, rd, fn, db_ip, int(at or 0))
             db.execute(sql, v)
 
         self.volsize[db] += sz
@@ -4016,9 +3876,6 @@ class Up2k(object):
             except:
                 pass
 
-    def handle_fs_abrt(self, akey: str) -> None:
-        self.abrt_key = akey
-
     def handle_rm(
         self,
         uname: str,
@@ -4065,7 +3922,7 @@ class Up2k(object):
             vn, rem = vn0.get_dbv(rem0)
             ptop = vn.realpath
             with self.mutex, self.reg_mutex:
-                abrt_cfg = vn.flags.get("u2abort", 1)
+                abrt_cfg = self.flags.get(ptop, {}).get("u2abort", 1)
                 addr = (ip or "\n") if abrt_cfg in (1, 2) else ""
                 user = ((uname or "\n"), "*") if abrt_cfg in (1, 3) else None
                 reg = self.registry.get(ptop, {}) if abrt_cfg else {}
@@ -4086,22 +3943,17 @@ class Up2k(object):
                 if partial:
                     dip = ip
                     dat = time.time()
-                    dun = uname
-                    un_cfg = 1
                 else:
-                    un_cfg = vn.flags["unp_who"]
-                    if not self.args.unpost or not un_cfg:
+                    if not self.args.unpost:
                         t = "the unpost feature is disabled in server config"
                         raise Pebkac(400, t)
 
-                    _, _, _, _, dip, dat, dun = self._find_from_vpath(ptop, rem)
+                    _, _, _, _, dip, dat = self._find_from_vpath(ptop, rem)
 
             t = "you cannot delete this: "
             if not dip:
                 t += "file not found"
-            elif dip != ip and un_cfg in (1, 2):
-                t += "not uploaded by (You)"
-            elif dun != uname and un_cfg in (1, 3):
+            elif dip != ip:
                 t += "not uploaded by (You)"
             elif dat < time.time() - self.args.unpost:
                 t += "uploaded too long ago"
@@ -4190,7 +4042,7 @@ class Up2k(object):
                     try:
                         ptop = dbv.realpath
                         xlink = bool(dbv.flags.get("xlink"))
-                        cur, wark, _, _, _, _, _ = self._find_from_vpath(ptop, volpath)
+                        cur, wark, _, _, _, _ = self._find_from_vpath(ptop, volpath)
                         self._forget_file(
                             ptop, volpath, cur, wark, True, st.st_size, xlink
                         )
@@ -4233,7 +4085,7 @@ class Up2k(object):
 
         return n_files, ok + ok2, ng + ng2
 
-    def handle_cp(self, abrt: str, uname: str, ip: str, svp: str, dvp: str) -> str:
+    def handle_cp(self, uname: str, ip: str, svp: str, dvp: str) -> str:
         if svp == dvp or dvp.startswith(svp + "/"):
             raise Pebkac(400, "cp: cannot copy parent into subfolder")
 
@@ -4280,8 +4132,6 @@ class Up2k(object):
 
                         dvpf = dvp + svpf[len(svp) :]
                         self._cp_file(uname, ip, svpf, dvpf, curs)
-                        if abrt and abrt == self.abrt_key:
-                            raise Pebkac(400, "filecopy aborted by http-api")
 
                     for v in curs:
                         v.connection.commit()
@@ -4349,9 +4199,9 @@ class Up2k(object):
                 self.log(t, 1)
                 raise Pebkac(405, t)
 
-        bos.makedirs(os.path.dirname(dabs), vf=dvn.flags)
+        bos.makedirs(os.path.dirname(dabs))
 
-        c1, w, ftime_, fsize_, ip, at, un = self._find_from_vpath(
+        c1, w, ftime_, fsize_, ip, at = self._find_from_vpath(
             svn_dbv.realpath, srem_dbv
         )
         c2 = self.cur.get(dvn.realpath)
@@ -4376,7 +4226,7 @@ class Up2k(object):
                     w,
                     w,
                     "",
-                    un or "",
+                    "",
                     ip or "",
                     at or 0,
                 )
@@ -4449,7 +4299,7 @@ class Up2k(object):
 
         return "k"
 
-    def handle_mv(self, abrt: str, uname: str, ip: str, svp: str, dvp: str) -> str:
+    def handle_mv(self, uname: str, ip: str, svp: str, dvp: str) -> str:
         if svp == dvp or dvp.startswith(svp + "/"):
             raise Pebkac(400, "mv: cannot move parent into subfolder")
 
@@ -4504,8 +4354,6 @@ class Up2k(object):
 
                         dvpf = dvp + svpf[len(svp) :]
                         self._mv_file(uname, ip, svpf, dvpf, curs)
-                        if abrt and abrt == self.abrt_key:
-                            raise Pebkac(400, "filemove aborted by http-api")
 
                     for v in curs:
                         v.connection.commit()
@@ -4527,10 +4375,7 @@ class Up2k(object):
                 vp = vjoin(dvp, rem)
                 try:
                     dvn, drem = self.vfs.get(vp, uname, False, True)
-                    dap = dvn.canonical(drem)
-                    bos.mkdir(dap, dvn.flags["chmod_d"])
-                    if "chown" in dvn.flags:
-                        bos.chown(dap, dvn.flags["uid"], dvn.flags["gid"])
+                    bos.mkdir(dvn.canonical(drem))
                 except:
                     pass
 
@@ -4600,7 +4445,7 @@ class Up2k(object):
 
         is_xvol = svn.realpath != dvn.realpath
 
-        bos.makedirs(os.path.dirname(dabs), vf=dvn.flags)
+        bos.makedirs(os.path.dirname(dabs))
 
         if is_dirlink:
             dlabs = absreal(sabs)
@@ -4637,7 +4482,7 @@ class Up2k(object):
 
             return "k"
 
-        c1, w, ftime_, fsize_, ip, at, un = self._find_from_vpath(svn.realpath, srem)
+        c1, w, ftime_, fsize_, ip, at = self._find_from_vpath(svn.realpath, srem)
         c2 = self.cur.get(dvn.realpath)
 
         has_dupes = False
@@ -4671,7 +4516,7 @@ class Up2k(object):
                     w,
                     w,
                     "",
-                    un or "",
+                    "",
                     ip or "",
                     at or 0,
                 )
@@ -4689,7 +4534,7 @@ class Up2k(object):
                 dlink = bos.readlink(sabs)
                 dlink = os.path.join(os.path.dirname(sabs), dlink)
                 dlink = bos.path.abspath(dlink)
-                self._symlink(dlink, dabs, dvn.flags, lmod=ftime, is_mv=True)
+                self._symlink(dlink, dabs, dvn.flags, lmod=ftime)
                 wunlink(self.log, sabs, svn.flags)
             else:
                 atomic_move(self.log, sabs, dabs, svn.flags)
@@ -4769,16 +4614,15 @@ class Up2k(object):
         Optional[str],
         Optional[int],
         Optional[int],
-        str,
+        Optional[str],
         Optional[int],
-        str,
     ]:
         cur = self.cur.get(ptop)
         if not cur:
-            return None, None, None, None, "", None, ""
+            return None, None, None, None, None, None
 
         rd, fn = vsplit(vrem)
-        q = "select w, mt, sz, ip, at, un from up where rd=? and fn=? limit 1"
+        q = "select w, mt, sz, ip, at from up where rd=? and fn=? limit 1"
         try:
             c = cur.execute(q, (rd, fn))
         except:
@@ -4787,9 +4631,9 @@ class Up2k(object):
 
         hit = c.fetchone()
         if hit:
-            wark, ftime, fsize, ip, at, un = hit
-            return cur, wark, ftime, fsize, ip, at, un
-        return cur, None, None, None, "", None, ""
+            wark, ftime, fsize, ip, at = hit
+            return cur, wark, ftime, fsize, ip, at
+        return cur, None, None, None, None, None
 
     def _forget_file(
         self,
@@ -4909,7 +4753,7 @@ class Up2k(object):
             flags = self.flags.get(ptop) or {}
             atomic_move(self.log, sabs, slabs, flags)
             bos.utime(slabs, (int(time.time()), int(mt)), False)
-            self._symlink(slabs, sabs, flags, False, is_mv=True)
+            self._symlink(slabs, sabs, flags, False)
             full[slabs] = (ptop, rem)
             sabs = slabs
 
@@ -4968,9 +4812,7 @@ class Up2k(object):
             # (for example a volume with symlinked dupes but no --dedup);
             # fsrc=sabs is then a source that currently resolves to copy
 
-            self._symlink(
-                dabs, alink, flags, False, lmod=lmod or 0, fsrc=sabs, is_mv=True
-            )
+            self._symlink(dabs, alink, flags, False, lmod=lmod or 0, fsrc=sabs)
 
         return len(full) + len(links)
 
@@ -5049,8 +4891,7 @@ class Up2k(object):
             except:
                 pass
 
-        vf = self.flags[job["ptop"]]
-        xbu = vf.get("xbu")
+        xbu = self.flags[job["ptop"]].get("xbu")
         ap_chk = djoin(pdir, job["name"])
         vp_chk = djoin(job["vtop"], job["prel"], job["name"])
         if xbu:
@@ -5078,15 +4919,13 @@ class Up2k(object):
             if hr.get("reloc"):
                 x = pathmod(self.vfs, ap_chk, vp_chk, hr["reloc"])
                 if x:
-                    ud1 = (vfs.vpath, job["prel"], job["name"])
+                    zvfs = vfs
                     pdir, _, job["name"], (vfs, rem) = x
-                    job["vcfg"] = vf = vfs.flags
+                    job["vcfg"] = vfs.flags
                     job["ptop"] = vfs.realpath
                     job["vtop"] = vfs.vpath
                     job["prel"] = rem
-                    job["name"] = sanitize_fn(job["name"], "")
-                    ud2 = (vfs.vpath, job["prel"], job["name"])
-                    if ud1 != ud2:
+                    if zvfs.vpath != vfs.vpath:
                         self.log("xbu reloc2:%d..." % (depth,), 6)
                         return self._handle_json(job, depth + 1)
 
@@ -5108,13 +4947,8 @@ class Up2k(object):
         else:
             dip = self.hub.iphash.s(job["addr"])
 
-        f, job["tnam"] = ren_open(
-            tnam,
-            "wb",
-            fdir=pdir,
-            suffix="-%.6f-%s" % (job["t0"], dip),
-            vf=vf,
-        )
+        suffix = "-%.6f-%s" % (job["t0"], dip)
+        f, job["tnam"] = ren_open(tnam, "wb", fdir=pdir, suffix=suffix)
         try:
             abspath = djoin(pdir, job["tnam"])
             sprs = job["sprs"]
@@ -5137,13 +4971,8 @@ class Up2k(object):
                 fs = self.fstab.get(pdir)
                 if fs == "ok":
                     pass
-                elif "nosparse" in vf:
-                    t = "volflag 'nosparse' is preventing creation of sparse files for uploads to [%s]"
-                    self.log(t % (job["ptop"],))
-                    relabel = True
-                    sprs = False
-                elif "sparse" in vf:
-                    t = "volflag 'sparse' is forcing creation of sparse files for uploads to [%s]"
+                elif "sparse" in self.flags[job["ptop"]]:
+                    t = "volflag 'sparse' is forcing use of sparse files for uploads to [%s]"
                     self.log(t % (job["ptop"],))
                     relabel = True
                 else:
@@ -5195,7 +5024,7 @@ class Up2k(object):
 
     def _snap_reg(self, ptop: str, reg: dict[str, dict[str, Any]]) -> None:
         now = time.time()
-        histpath = self.vfs.dbpaths.get(ptop)
+        histpath = self.vfs.histtab.get(ptop)
         if not histpath:
             return
 

copyparty/util.py

@@ -31,17 +31,6 @@ from collections import Counter
 from ipaddress import IPv4Address, IPv4Network, IPv6Address, IPv6Network
 from queue import Queue
 
-try:
-    from zlib_ng import gzip_ng as gzip
-    from zlib_ng import zlib_ng as zlib
-
-    sys.modules["gzip"] = gzip
-    # sys.modules["zlib"] = zlib
-    # `- somehow makes tarfile 3% slower with default malloc, and barely faster with mimalloc
-except:
-    import gzip
-    import zlib
-
 from .__init__ import (
     ANYWIN,
     EXE,
@@ -105,7 +94,6 @@ def _ens(want: str) -> tuple[int, ...]:
 # WSAENOTSOCK - no longer a socket
 # EUNATCH - can't assign requested address (wifi down)
 E_SCK = _ens("ENOTCONN EUNATCH EBADF WSAENOTSOCK WSAECONNRESET")
-E_SCK_WR = _ens("EPIPE ESHUTDOWN EBADFD")
 E_ADDR_NOT_AVAIL = _ens("EADDRNOTAVAIL WSAEADDRNOTAVAIL")
 E_ADDR_IN_USE = _ens("EADDRINUSE WSAEADDRINUSE")
 E_ACCESS = _ens("EACCES WSAEACCES")
@@ -114,15 +102,9 @@ E_UNREACH = _ens("EHOSTUNREACH WSAEHOSTUNREACH ENETUNREACH WSAENETUNREACH")
 IP6ALL = "0:0:0:0:0:0:0:0"
 
 
-try:
-    import fcntl
-
-    HAVE_FCNTL = True
-except:
-    HAVE_FCNTL = False
-
 try:
     import ctypes
+    import fcntl
     import termios
 except:
     pass
@@ -138,13 +120,6 @@ try:
 except:
     HAVE_SQLITE3 = False
 
-try:
-    import importlib.util
-
-    HAVE_ZMQ = bool(importlib.util.find_spec("zmq"))
-except:
-    HAVE_ZMQ = False
-
 try:
     if os.environ.get("PRTY_NO_PSUTIL"):
         raise Exception()
@@ -154,16 +129,6 @@ try:
 except:
     HAVE_PSUTIL = False
 
-try:
-    if os.environ.get("PRTY_NO_MAGIC") or (
-        ANYWIN and not os.environ.get("PRTY_FORCE_MAGIC")
-    ):
-        raise Exception()
-
-    import magic
-except:
-    pass
-
 if True:  # pylint: disable=using-constant-test
     import types
     from collections.abc import Callable, Iterable
@@ -186,6 +151,8 @@ if True:  # pylint: disable=using-constant-test
 
 
 if TYPE_CHECKING:
+    import magic
+
     from .authsrv import VFS
     from .broker_util import BrokerCli
     from .up2k import Up2k
@@ -243,18 +210,7 @@ except:
     BITNESS = struct.calcsize("P") * 8
 
 
-RE_ANSI = re.compile("\033\\[[^mK]*[mK]")
-RE_HTML_SH = re.compile(r"[<>&$?`\"';]")
-RE_CTYPE = re.compile(r"^content-type: *([^; ]+)", re.IGNORECASE)
-RE_CDISP = re.compile(r"^content-disposition: *([^; ]+)", re.IGNORECASE)
-RE_CDISP_FIELD = re.compile(
-    r'^content-disposition:(?: *|.*; *)name="([^"]+)"', re.IGNORECASE
-)
-RE_CDISP_FILE = re.compile(
-    r'^content-disposition:(?: *|.*; *)filename="(.*)"', re.IGNORECASE
-)
-RE_MEMTOTAL = re.compile("^MemTotal:.* kB")
-RE_MEMAVAIL = re.compile("^MemAvailable:.* kB")
+ansi_re = re.compile("\033\\[[^mK]*[mK]")
 
 
 BOS_SEP = ("%s" % (os.sep,)).encode("ascii")
@@ -271,19 +227,11 @@ SYMTIME = PY36 and os.utime in os.supports_follow_symlinks
 
 META_NOBOTS = '<meta name="robots" content="noindex, nofollow">\n'
 
-# smart enough to understand javascript while also ignoring rel="nofollow"
-BAD_BOTS = r"Barkrowler|bingbot|BLEXBot|Googlebot|GoogleOther|GPTBot|PetalBot|SeekportBot|SemrushBot|YandexBot"
-
 FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"
 
-URL_PRJ = "https://github.com/9001/copyparty"
-
-URL_BUG = URL_PRJ + "/issues/new?labels=bug&template=bug_report.md"
-
 HTTPCODE = {
     200: "OK",
     201: "Created",
-    202: "Accepted",
     204: "No Content",
     206: "Partial Content",
     207: "Multi-Status",
@@ -371,7 +319,6 @@ DAV_ALLPROPS = set(DAV_ALLPROP_L)
 
 MIMES = {
     "opus": "audio/ogg; codecs=opus",
-    "owa": "audio/webm; codecs=opus",
 }
 
 
@@ -399,9 +346,6 @@ application swf=x-shockwave-flash m3u=vnd.apple.mpegurl db3=vnd.sqlite3 sqlite=v
 text ass=plain ssa=plain
 image jpg=jpeg xpm=x-xpixmap psd=vnd.adobe.photoshop jpf=jpx tif=tiff ico=x-icon djvu=vnd.djvu
 image heic=heic-sequence heif=heif-sequence hdr=vnd.radiance svg=svg+xml
-image arw=x-sony-arw cr2=x-canon-cr2 crw=x-canon-crw dcr=x-kodak-dcr dng=x-adobe-dng erf=x-epson-erf
-image k25=x-kodak-k25 kdc=x-kodak-kdc mrw=x-minolta-mrw nef=x-nikon-nef orf=x-olympus-orf
-image pef=x-pentax-pef raf=x-fuji-raf raw=x-panasonic-raw sr2=x-sony-sr2 srf=x-sony-srf x3f=x-sigma-x3f
 audio caf=x-caf mp3=mpeg m4a=mp4 mid=midi mpc=musepack aif=aiff au=basic qcp=qcelp
 video mkv=x-matroska mov=quicktime avi=x-msvideo m4v=x-m4v ts=mp2t
 video asf=x-ms-asf flv=x-flv 3gp=3gpp 3g2=3gpp2 rmvb=vnd.rn-realmedia-vbr
@@ -491,22 +435,18 @@ UNHUMANIZE_UNITS = {
 
 VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
 
-FN_EMB = set([".prologue.html", ".epilogue.html", "readme.md", "preadme.md"])
-
 
 def read_ram() -> tuple[float, float]:
-    # NOTE: apparently no need to consider /sys/fs/cgroup/memory.max
-    #  (cgroups2) since the limit is synced to /proc/meminfo
     a = b = 0
     try:
         with open("/proc/meminfo", "rb", 0x10000) as f:
             zsl = f.read(0x10000).decode("ascii", "replace").split("\n")
 
-        p = RE_MEMTOTAL
+        p = re.compile("^MemTotal:.* kB")
         zs = next((x for x in zsl if p.match(x)))
         a = int((int(zs.split()[1]) / 0x100000) * 100) / 100
 
-        p = RE_MEMAVAIL
+        p = re.compile("^MemAvailable:.* kB")
         zs = next((x for x in zsl if p.match(x)))
         b = int((int(zs.split()[1]) / 0x100000) * 100) / 100
     except:
@@ -551,15 +491,6 @@ def py_desc() -> str:
     )
 
 
-def expat_ver() -> str:
-    try:
-        import pyexpat
-
-        return ".".join([str(x) for x in pyexpat.version_info])
-    except:
-        return "?"
-
-
 def _sqlite_ver() -> str:
     assert sqlite3  # type: ignore  # !rm
     try:
@@ -641,38 +572,6 @@ except Exception as ex:
         print("using fallback base64 codec due to %r" % (ex,))
 
 
-class NotUTF8(Exception):
-    pass
-
-
-def read_utf8(log: Optional["NamedLogger"], ap: Union[str, bytes], strict: bool) -> str:
-    with open(ap, "rb") as f:
-        buf = f.read()
-
-    try:
-        return buf.decode("utf-8", "strict")
-    except UnicodeDecodeError as ex:
-        eo = ex.start
-        eb = buf[eo : eo + 1]
-
-    if not strict:
-        t = "WARNING: The file [%s] is not using the UTF-8 character encoding; some characters in the file will be skipped/ignored. The first unreadable character was byte %r at offset %d. Please convert this file to UTF-8 by opening the file in your text-editor and saving it as UTF-8."
-        t = t % (ap, eb, eo)
-        if log:
-            log(t, 3)
-        else:
-            print(t)
-        return buf.decode("utf-8", "replace")
-
-    t = "ERROR: The file [%s] is not using the UTF-8 character encoding, and cannot be loaded. The first unreadable character was byte %r at offset %d. Please convert this file to UTF-8 by opening the file in your text-editor and saving it as UTF-8."
-    t = t % (ap, eb, eo)
-    if log:
-        log(t, 3)
-    else:
-        print(t)
-    raise NotUTF8(t)
-
-
 class Daemon(threading.Thread):
     def __init__(
         self,
@@ -1279,6 +1178,8 @@ class Magician(object):
         self.magic: Optional["magic.Magic"] = None
 
     def ext(self, fpath: str) -> str:
+        import magic
+
         try:
             if self.bad_magic:
                 raise Exception()
@@ -1496,6 +1397,8 @@ def stackmon(fp: str, ival: float, suffix: str) -> None:
         buf = st.encode("utf-8", "replace")
 
         if fp.endswith(".gz"):
+            import gzip
+
             # 2459b 2304b 2241b 2202b 2194b 2191b lv3..8
             # 0.06s 0.08s 0.11s 0.13s 0.16s 0.19s
             buf = gzip.compress(buf, compresslevel=6)
@@ -1575,12 +1478,6 @@ def vol_san(vols: list["VFS"], txt: bytes) -> bytes:
         txt = txt.replace(bap.replace(b"\\", b"\\\\"), bvp)
         txt = txt.replace(bhp.replace(b"\\", b"\\\\"), bvph)
 
-        if vol.histpath != vol.dbpath:
-            bdp = vol.dbpath.encode("utf-8")
-            bdph = b"$db(/" + bvp + b")"
-            txt = txt.replace(bdp, bdph)
-            txt = txt.replace(bdp.replace(b"\\", b"\\\\"), bdph)
-
     if txt != txt0:
         txt += b"\r\nNOTE: filepaths sanitized; see serverlog for correct values"
 
@@ -1601,8 +1498,6 @@ def ren_open(fname: str, *args: Any, **kwargs: Any) -> tuple[typing.IO[Any], str
     fun = kwargs.pop("fun", open)
     fdir = kwargs.pop("fdir", None)
     suffix = kwargs.pop("suffix", None)
-    vf = kwargs.pop("vf", None)
-    fperms = vf and "fperms" in vf
 
     if fname == os.devnull:
         return fun(fname, *args, **kwargs), fname
@@ -1646,11 +1541,6 @@ def ren_open(fname: str, *args: Any, **kwargs: Any) -> tuple[typing.IO[Any], str
                 fp2 = os.path.join(fdir, fp2)
                 with open(fsenc(fp2), "wb") as f2:
                     f2.write(orig_name.encode("utf-8"))
-                    if fperms:
-                        set_fperms(f2, vf)
-
-            if fperms:
-                set_fperms(f, vf)
 
             return f, fname
 
@@ -1712,10 +1602,14 @@ class MultipartParser(object):
         self.args = args
         self.headers = http_headers
 
-        self.re_ctype = RE_CTYPE
-        self.re_cdisp = RE_CDISP
-        self.re_cdisp_field = RE_CDISP_FIELD
-        self.re_cdisp_file = RE_CDISP_FILE
+        self.re_ctype = re.compile(r"^content-type: *([^; ]+)", re.IGNORECASE)
+        self.re_cdisp = re.compile(r"^content-disposition: *([^; ]+)", re.IGNORECASE)
+        self.re_cdisp_field = re.compile(
+            r'^content-disposition:(?: *|.*; *)name="([^"]+)"', re.IGNORECASE
+        )
+        self.re_cdisp_file = re.compile(
+            r'^content-disposition:(?: *|.*; *)filename="(.*)"', re.IGNORECASE
+        )
 
         self.boundary = b""
         self.gen: Optional[
@@ -1987,7 +1881,7 @@ def rand_name(fdir: str, fn: str, rnd: int) -> str:
     return fn
 
 
-def _gen_filekey(alg: int, salt: str, fspath: str, fsize: int, inode: int) -> str:
+def gen_filekey(alg: int, salt: str, fspath: str, fsize: int, inode: int) -> str:
     if alg == 1:
         zs = "%s %s %s %s" % (salt, fspath, fsize, inode)
     else:
@@ -1997,13 +1891,6 @@ def _gen_filekey(alg: int, salt: str, fspath: str, fsize: int, inode: int) -> st
     return ub64enc(hashlib.sha512(zb).digest()).decode("ascii")
 
 
-def _gen_filekey_w(alg: int, salt: str, fspath: str, fsize: int, inode: int) -> str:
-    return _gen_filekey(alg, salt, fspath.replace("/", "\\"), fsize, inode)
-
-
-gen_filekey = _gen_filekey_w if ANYWIN else _gen_filekey
-
-
 def gen_filekey_dbg(
     alg: int,
     salt: str,
@@ -2050,25 +1937,15 @@ def formatdate(ts: Optional[float] = None) -> str:
     return RFC2822 % (WKDAYS[wd], d, MONTHS[mo - 1], y, h, mi, s)
 
 
-def gencookie(
-    k: str, v: str, r: str, lax: bool, tls: bool, dur: int = 0, txt: str = ""
-) -> str:
+def gencookie(k: str, v: str, r: str, tls: bool, dur: int = 0, txt: str = "") -> str:
     v = v.replace("%", "%25").replace(";", "%3B")
     if dur:
         exp = formatdate(time.time() + dur)
     else:
         exp = "Fri, 15 Aug 1997 01:00:00 GMT"
 
-    t = "%s=%s; Path=/%s; Expires=%s%s%s; SameSite=%s"
-    return t % (
-        k,
-        v,
-        r,
-        exp,
-        "; Secure" if tls else "",
-        txt,
-        "Lax" if lax else "Strict",
-    )
+    t = "%s=%s; Path=/%s; Expires=%s%s%s; SameSite=Lax"
+    return t % (k, v, r, exp, "; Secure" if tls else "", txt)
 
 
 def humansize(sz: float, terse: bool = False) -> str:
@@ -2257,16 +2134,6 @@ def find_prefix(ips: list[str], cidrs: list[str]) -> list[str]:
     return ret
 
 
-def html_sh_esc(s: str) -> str:
-    s = re.sub(RE_HTML_SH, "_", s).replace(" ", "%20")
-    s = s.replace("\r", "_").replace("\n", "_")
-    return s
-
-
-def json_hesc(s: str) -> str:
-    return s.replace("<", "\\u003c").replace(">", "\\u003e").replace("&", "\\u0026")
-
-
 def html_escape(s: str, quot: bool = False, crlf: bool = False) -> str:
     """html.escape but also newlines"""
     s = s.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
@@ -2399,21 +2266,6 @@ def ujoin(rd: str, fn: str) -> str:
         return rd or fn
 
 
-def str_anchor(txt) -> tuple[int, str]:
-    if not txt:
-        return 0, ""
-    txt = txt.lower()
-    a = txt.startswith("^")
-    b = txt.endswith("$")
-    if not b:
-        if not a:
-            return 1, txt  # ~
-        return 2, txt[1:]  # ^
-    if not a:
-        return 3, txt[:-1]  # $
-    return 4, txt[1:-1]  # ^$
-
-
 def log_reloc(
     log: "NamedLogger",
     re: dict[str, str],
@@ -2462,11 +2314,11 @@ def pathmod(
 
         # try to map abspath to vpath
         np = np.replace("/", os.sep)
-        for vn_ap, vns in vfs.all_aps:
+        for vn_ap, vn in vfs.all_aps:
             if not np.startswith(vn_ap):
                 continue
             zs = np[len(vn_ap) :].replace(os.sep, "/")
-            nvp = vjoin(vns[0].vpath, zs)
+            nvp = vjoin(vn.vpath, zs)
             break
 
     if nvp == "\n":
@@ -2601,14 +2453,6 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
         log("lsof failed; " + min_ex(), 3)
 
 
-def set_fperms(f: Union[typing.BinaryIO, typing.IO[Any]], vf: dict[str, Any]) -> None:
-    fno = f.fileno()
-    if "chmod_f" in vf:
-        os.fchmod(fno, vf["chmod_f"])
-    if "chown" in vf:
-        os.fchown(fno, vf["uid"], vf["gid"])
-
-
 def _fs_mvrm(
     log: "NamedLogger", src: str, dst: str, atomic: bool, flags: dict[str, Any]
 ) -> bool:
@@ -2657,11 +2501,6 @@ def _fs_mvrm(
             now = time.time()
             if ex.errno == errno.ENOENT:
                 return False
-            if not attempt and ex.errno == errno.EXDEV:
-                t = "using copy+delete (%s)\n  %s\n  %s"
-                log(t % (ex.strerror, src, dst))
-                osfun = shutil.move
-                continue
             if now - t0 > maxtime or attempt == 90209:
                 raise
             if not attempt:
@@ -2686,18 +2525,15 @@ def atomic_move(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -
     elif flags.get("mv_re_t"):
         _fs_mvrm(log, src, dst, True, flags)
     else:
-        try:
-            os.replace(bsrc, bdst)
-        except OSError as ex:
-            if ex.errno != errno.EXDEV:
-                raise
-            t = "using copy+delete (%s);\n  %s\n  %s"
-            log(t % (ex.strerror, src, dst))
-            try:
-                os.unlink(bdst)
-            except:
-                pass
-            shutil.move(bsrc, bdst)
+        os.replace(bsrc, bdst)
+
+
+def wrename(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> bool:
+    if not flags.get("mv_re_t"):
+        os.rename(fsenc(src), fsenc(dst))
+        return True
+
+    return _fs_mvrm(log, src, dst, False, flags)
 
 
 def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
@@ -2708,7 +2544,7 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
     return _fs_mvrm(log, abspath, "", False, flags)
 
 
-def get_df(abspath: str, prune: bool) -> tuple[int, int, str]:
+def get_df(abspath: str, prune: bool) -> tuple[Optional[int], Optional[int], str]:
     try:
         ap = fsenc(abspath)
         while prune and not os.path.isdir(ap) and BOS_SEP in ap:
@@ -2719,22 +2555,17 @@ def get_df(abspath: str, prune: bool) -> tuple[int, int, str]:
             assert ctypes  # type: ignore  # !rm
             abspath = fsdec(ap)
             bfree = ctypes.c_ulonglong(0)
-            btotal = ctypes.c_ulonglong(0)
-            bavail = ctypes.c_ulonglong(0)
             ctypes.windll.kernel32.GetDiskFreeSpaceExW(  # type: ignore
-                ctypes.c_wchar_p(abspath),
-                ctypes.pointer(bavail),
-                ctypes.pointer(btotal),
-                ctypes.pointer(bfree),
+                ctypes.c_wchar_p(abspath), None, None, ctypes.pointer(bfree)
             )
-            return (bavail.value, btotal.value, "")
+            return (bfree.value, None, "")
         else:
             sv = os.statvfs(ap)
-            free = sv.f_frsize * sv.f_bavail
+            free = sv.f_frsize * sv.f_bfree
             total = sv.f_frsize * sv.f_blocks
             return (free, total, "")
     except Exception as ex:
-        return (0, 0, repr(ex))
+        return (None, None, repr(ex))
 
 
 if not ANYWIN and not MACOS:
@@ -2954,27 +2785,6 @@ def load_ipu(
     return ip_u, nm
 
 
-def load_ipr(
-    log: "RootLogger", iprs: list[str], defer_mutex: bool = False
-) -> dict[str, NetMap]:
-    ret = {}
-    for ipr in iprs:
-        try:
-            zs, uname = ipr.split("=")
-            cidrs = zs.split(",")
-        except:
-            t = "\n  invalid value %r for argument --ipr; must be CIDR[,CIDR[,...]]=UNAME (192.168.0.0/16=amelia)"
-            raise Exception(t % (ipr,))
-        try:
-            nm = NetMap(["::"], cidrs, True, True, defer_mutex)
-        except Exception as ex:
-            t = "failed to translate --ipr into netmap, probably due to invalid config: %r"
-            log("root", t % (ex,), 1)
-            raise
-        ret[uname] = nm
-    return ret
-
-
 def yieldfile(fn: str, bufsz: int) -> Generator[bytes, None, None]:
     readsz = min(bufsz, 128 * 1024)
     with open(fsenc(fn), "rb", bufsz) as f:
@@ -3006,17 +2816,6 @@ def justcopy(
     return tlen, "checksum-disabled", "checksum-disabled"
 
 
-def eol_conv(
-    fin: Generator[bytes, None, None], conv: str
-) -> Generator[bytes, None, None]:
-    crlf = conv.lower() == "crlf"
-    for buf in fin:
-        buf = buf.replace(b"\r", b"")
-        if crlf:
-            buf = buf.replace(b"\n", b"\r\n")
-        yield buf
-
-
 def hashcopy(
     fin: Generator[bytes, None, None],
     fout: Union[typing.BinaryIO, typing.IO[Any]],
@@ -3263,13 +3062,11 @@ def unescape_cookie(orig: str) -> str:
     return "".join(ret)
 
 
-def guess_mime(
-    url: str, path: str = "", fallback: str = "application/octet-stream"
-) -> str:
+def guess_mime(url: str, fallback: str = "application/octet-stream") -> str:
     try:
         ext = url.rsplit(".", 1)[1].lower()
     except:
-        ext = ""
+        return fallback
 
     ret = MIMES.get(ext)
 
@@ -3277,16 +3074,6 @@ def guess_mime(
         x = mimetypes.guess_type(url)
         ret = "application/{}".format(x[1]) if x[1] else x[0]
 
-    if not ret and path:
-        try:
-            with open(fsenc(path), "rb", 0) as f:
-                ret = magic.from_buffer(f.read(4096), mime=True)
-                if ret.startswith("text/htm"):
-                    # avoid serving up HTML content unless there was actually a .html extension
-                    ret = "text/plain"
-        except Exception as ex:
-            pass
-
     if not ret:
         ret = fallback
 
@@ -3599,10 +3386,9 @@ def _parsehook(
 
 def runihook(
     log: Optional["NamedLogger"],
-    verbose: bool,
     cmd: str,
     vol: "VFS",
-    ups: list[tuple[str, int, int, str, str, str, int, str]],
+    ups: list[tuple[str, int, int, str, str, str, int]],
 ) -> bool:
     _, chk, fork, jtxt, wait, sp_ka, acmd = _parsehook(log, cmd)
     bcmd = [sfsenc(x) for x in acmd]
@@ -3629,17 +3415,6 @@ def runihook(
     else:
         sp_ka["sin"] = b"\n".join(fsenc(x) for x in aps)
 
-    if acmd[0].startswith("zmq:"):
-        try:
-            msg = sp_ka["sin"].decode("utf-8", "replace")
-            _zmq_hook(log, verbose, "xiu", acmd[0][4:].lower(), msg, wait, sp_ka)
-            if verbose and log:
-                log("hook(xiu) %r OK" % (cmd,), 6)
-        except Exception as ex:
-            if log:
-                log("zeromq failed: %r" % (ex,))
-        return True
-
     t0 = time.time()
     if fork:
         Daemon(runcmd, cmd, bcmd, ka=sp_ka)
@@ -3649,126 +3424,15 @@ def runihook(
             retchk(rc, bcmd, err, log, 5)
             return False
 
-    if wait:
-        wait -= time.time() - t0
-        if wait > 0:
-            time.sleep(wait)
+    wait -= time.time() - t0
+    if wait > 0:
+        time.sleep(wait)
 
     return True
 
 
-ZMQ = {}
-ZMQ_DESC = {
-    "pub": "fire-and-forget to all/any connected SUB-clients",
-    "push": "fire-and-forget to one of the connected PULL-clients",
-    "req": "send messages to a REP-server and blocking-wait for ack",
-}
-
-
-def _zmq_hook(
-    log: Optional["NamedLogger"],
-    verbose: bool,
-    src: str,
-    cmd: str,
-    msg: str,
-    wait: float,
-    sp_ka: dict[str, Any],
-) -> tuple[int, str]:
-    import zmq
-
-    try:
-        mtx = ZMQ["mtx"]
-    except:
-        ZMQ["mtx"] = threading.Lock()
-        time.sleep(0.1)
-        mtx = ZMQ["mtx"]
-
-    ret = ""
-    nret = 0
-    t0 = time.time()
-    if verbose and log:
-        log("hook(%s) %r entering zmq-main-lock" % (src, cmd), 6)
-
-    with mtx:
-        try:
-            mode, sck, mtx = ZMQ[cmd]
-        except:
-            mode, uri = cmd.split(":", 1)
-            try:
-                desc = ZMQ_DESC[mode]
-                if log:
-                    t = "libzmq(%s) pyzmq(%s) init(%s); %s"
-                    log(t % (zmq.zmq_version(), zmq.__version__, cmd, desc))
-            except:
-                raise Exception("the only supported ZMQ modes are REQ PUB PUSH")
-
-            try:
-                ctx = ZMQ["ctx"]
-            except:
-                ctx = ZMQ["ctx"] = zmq.Context()
-
-            timeout = sp_ka["timeout"]
-
-            if mode == "pub":
-                sck = ctx.socket(zmq.PUB)
-                sck.setsockopt(zmq.LINGER, 0)
-                sck.bind(uri)
-                time.sleep(1)  # give clients time to connect; avoids losing first msg
-            elif mode == "push":
-                sck = ctx.socket(zmq.PUSH)
-                if timeout:
-                    sck.SNDTIMEO = int(timeout * 1000)
-                sck.setsockopt(zmq.LINGER, 0)
-                sck.bind(uri)
-            elif mode == "req":
-                sck = ctx.socket(zmq.REQ)
-                if timeout:
-                    sck.RCVTIMEO = int(timeout * 1000)
-                sck.setsockopt(zmq.LINGER, 0)
-                sck.connect(uri)
-            else:
-                raise Exception()
-
-            mtx = threading.Lock()
-            ZMQ[cmd] = (mode, sck, mtx)
-
-    if verbose and log:
-        log("hook(%s) %r entering socket-lock" % (src, cmd), 6)
-
-    with mtx:
-        if verbose and log:
-            log("hook(%s) %r sending |%d|" % (src, cmd, len(msg)), 6)
-
-        sck.send_string(msg)  # PUSH can safely timeout here
-
-        if mode == "req":
-            if verbose and log:
-                log("hook(%s) %r awaiting ack from req" % (src, cmd), 6)
-            try:
-                ret = sck.recv().decode("utf-8", "replace")
-                if ret.startswith("return "):
-                    m = re.search("^return ([0-9]+)", ret[:12])
-                    if m:
-                        nret = int(m.group(1))
-            except:
-                sck.close()
-                del ZMQ[cmd]  # bad state; must reset
-                raise Exception("ack timeout; zmq socket killed")
-
-    if ret and log:
-        log("hook(%s) %r ACK: %r" % (src, cmd, ret), 6)
-
-    if wait:
-        wait -= time.time() - t0
-        if wait > 0:
-            time.sleep(wait)
-
-    return nret, ret
-
-
 def _runhook(
     log: Optional["NamedLogger"],
-    verbose: bool,
     src: str,
     cmd: str,
     ap: str,
@@ -3809,12 +3473,6 @@ def _runhook(
     else:
         arg = txt or ap
 
-    if acmd[0].startswith("zmq:"):
-        zi, zs = _zmq_hook(log, verbose, src, acmd[0][4:].lower(), arg, wait, sp_ka)
-        if zi:
-            raise Exception("zmq says %d" % (zi,))
-        return {"rc": 0, "stdout": zs}
-
     acmd += [arg]
     if acmd[0].endswith(".py"):
         acmd = [pybin] + acmd
@@ -3843,10 +3501,9 @@ def _runhook(
             except:
                 ret = {"rc": rc, "stdout": v}
 
-    if wait:
-        wait -= time.time() - t0
-        if wait > 0:
-            time.sleep(wait)
+    wait -= time.time() - t0
+    if wait > 0:
+        time.sleep(wait)
 
     return ret
 
@@ -3870,15 +3527,14 @@ def runhook(
 ) -> dict[str, Any]:
     assert broker or up2k  # !rm
     args = (broker or up2k).args
-    verbose = args.hook_v
     vp = vp.replace("\\", "/")
     ret = {"rc": 0}
     for cmd in cmds:
         try:
             hr = _runhook(
-                log, verbose, src, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt
+                log, src, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt
             )
-            if verbose and log:
+            if log and args.hook_v:
                 log("hook(%s) %r => \033[32m%s" % (src, cmd, hr), 6)
             if not hr:
                 return {}
@@ -3894,8 +3550,6 @@ def runhook(
                 elif k in ret:
                     if k == "rc" and v:
                         ret[k] = v
-                    elif k == "stdout" and v and not ret[k]:
-                        ret[k] = v
                 else:
                     ret[k] = v
         except Exception as ex:
@@ -4079,75 +3733,8 @@ def hidedir(dp) -> None:
             pass
 
 
-_flocks = {}
-
-
-def _lock_file_noop(ap: str) -> bool:
-    return True
-
-
-def _lock_file_ioctl(ap: str) -> bool:
-    assert fcntl  # type: ignore  # !rm
-    try:
-        fd = _flocks.pop(ap)
-        os.close(fd)
-    except:
-        pass
-
-    fd = os.open(ap, os.O_RDWR | os.O_CREAT, 438)
-    # NOTE: the fcntl.lockf identifier is (pid,node);
-    #  the lock will be dropped if os.close(os.open(ap))
-    #  is performed anywhere else in this thread
-
-    try:
-        fcntl.lockf(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
-        _flocks[ap] = fd
-        return True
-    except Exception as ex:
-        eno = getattr(ex, "errno", -1)
-        try:
-            os.close(fd)
-        except:
-            pass
-        if eno in (errno.EAGAIN, errno.EACCES):
-            return False
-        print("WARNING: unexpected errno %d from fcntl.lockf; %r" % (eno, ex))
-        return True
-
-
-def _lock_file_windows(ap: str) -> bool:
-    try:
-        import msvcrt
-
-        try:
-            fd = _flocks.pop(ap)
-            os.close(fd)
-        except:
-            pass
-
-        fd = os.open(ap, os.O_RDWR | os.O_CREAT, 438)
-        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
-        return True
-    except Exception as ex:
-        eno = getattr(ex, "errno", -1)
-        if eno == errno.EACCES:
-            return False
-        print("WARNING: unexpected errno %d from msvcrt.locking; %r" % (eno, ex))
-        return True
-
-
-if os.environ.get("PRTY_NO_DB_LOCK"):
-    lock_file = _lock_file_noop
-elif ANYWIN:
-    lock_file = _lock_file_windows
-elif HAVE_FCNTL:
-    lock_file = _lock_file_ioctl
-else:
-    lock_file = _lock_file_noop
-
-
 try:
-    if sys.version_info < (3, 10) or os.environ.get("PRTY_NO_IMPRESO"):
+    if sys.version_info < (3, 10):
         # py3.8 doesn't have .files
         # py3.9 has broken .is_file
         raise ImportError()
@@ -4247,12 +3834,7 @@ def load_resource(E: EnvParams, name: str, mode="rb") -> IO[bytes]:
                 stream = codecs.getreader(enc)(stream)
             return stream
 
-    ap = os.path.join(E.mod, name)
-
-    if PY2:
-        return codecs.open(ap, "r", encoding=enc)  # type: ignore
-
-    return open(ap, mode, encoding=enc)
+    return open(os.path.join(E.mod, name), mode, encoding=enc)
 
 
 class Pebkac(Exception):
@@ -4284,22 +3866,9 @@ class WrongPostKey(Pebkac):
         self.datagen = datagen
 
 
-_: Any = (
-    gzip,
-    mp,
-    zlib,
-    BytesIO,
-    quote,
-    unquote,
-    SQLITE_VER,
-    JINJA_VER,
-    PYFTPD_VER,
-    PARTFTPY_VER,
-)
+_: Any = (mp, BytesIO, quote, unquote, SQLITE_VER, JINJA_VER, PYFTPD_VER, PARTFTPY_VER)
 __all__ = [
-    "gzip",
     "mp",
-    "zlib",
     "BytesIO",
     "quote",
     "unquote",

copyparty/web/baguettebox.js

@@ -48,7 +48,6 @@ window.baguetteBox = (function () {
 
     var onFSC = function (e) {
         isFullscreen = !!document.fullscreenElement;
-        clmod(document.documentElement, 'bb_fsc', isFullscreen);
     };
 
     var overlayClickHandler = function (e) {
@@ -403,7 +402,7 @@ window.baguetteBox = (function () {
             if (isFullscreen)
                 document.exitFullscreen();
             else
-                ebi('bbox-overlay').requestFullscreen();
+                (vid() || ebi('bbox-overlay')).requestFullscreen();
         }
         catch (ex) {
             if (IPHONE)
@@ -593,7 +592,9 @@ window.baguetteBox = (function () {
             preloadPrev(currentIndex);
         });
 
-        show_buttons(0);
+        clmod(ebi('bbox-btns'), 'off');
+        clmod(btnPrev, 'off');
+        clmod(btnNext, 'off');
 
         updateOffset();
         overlay.style.display = 'block';
@@ -632,9 +633,6 @@ window.baguetteBox = (function () {
         catch (ex) { }
         isFullscreen = false;
 
-        if (toast.tag == 'bb-ded')
-            toast.hide();
-
         if (dtor || overlay.style.display === 'none')
             return;
 
@@ -670,7 +668,6 @@ window.baguetteBox = (function () {
             if (v == keep)
                 continue;
 
-            unbind(v, 'error', lerr);
             v.src = '';
             v.load();
 
@@ -698,28 +695,6 @@ window.baguetteBox = (function () {
         }
     }
 
-    function lerr() {
-        var t;
-        try {
-            t = this.getAttribute('src');
-            t = uricom_dec(t.split('/').pop().split('?')[0]);
-        }
-        catch (ex) { }
-
-        t = 'Failed to open ' + (t?t:'file');
-        console.log('bb-ded', t);
-        t += '\n\nEither the file is corrupt, or your browser does not understand the file format or codec';
-
-        try {
-            t += "\n\nerr#" + this.error.code + ", " + this.error.message;
-        }
-        catch (ex) { }
- 
-        this.ded = esc(t);
-        if (this === vidimg())
-            toast.err(20, this.ded, 'bb-ded');
-    }
-
     function loadImage(index, callback) {
         var imageContainer = imagesElements[index];
         var galleryItem = currentGallery[index];
@@ -764,8 +739,7 @@ window.baguetteBox = (function () {
         var image = mknod(is_vid ? 'video' : 'img');
         clmod(imageContainer, 'vid', is_vid);
 
-        bind(image, 'error', lerr);
-        bind(image, is_vid ? 'loadedmetadata' : 'load', function () {
+        image.addEventListener(is_vid ? 'loadedmetadata' : 'load', function () {
             // Remove loader element
             qsr('#baguette-img-' + index + ' .bbox-spinner');
             if (!options.async && callback)
@@ -775,8 +749,6 @@ window.baguetteBox = (function () {
         if (is_vid) {
             image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
             image.setAttribute('controls', 'controls');
-            image.setAttribute('playsinline', '1');
-            // ios ignores poster
             image.onended = vidEnd;
             image.onplay = function () { show_buttons(1); };
             image.onpause = function () { show_buttons(); };
@@ -844,12 +816,6 @@ window.baguetteBox = (function () {
         });
         updateOffset();
 
-        var im = vidimg();
-        if (im && im.ded)
-            toast.err(20, im.ded, 'bb-ded');
-        else if (toast.tag == 'bb-ded')
-            toast.hide();
-
         if (options.animation == 'none')
             unvid(vid());
         else

copyparty/web/browser.html

@@ -109,7 +109,7 @@
 {%- for f in files %}
 <tr><td>{{ f.lead }}</td><td><a href="{{ f.href }}">{{ f.name|e }}</a></td><td>{{ f.sz }}</td>
 {%- if f.tags is defined %}
-	{%- for k in taglist %}<td>{{ f.tags[k]|e }}</td>{%- endfor %}
+	{%- for k in taglist %}<td>{{ f.tags[k] }}</td>{%- endfor %}
 {%- endif %}<td>{{ f.ext }}</td><td>{{ f.dt }}</td></tr>
 {%- endfor %}
 
@@ -124,12 +124,14 @@
 
 </div>
 
+	{%- if srv_info %}
 	<div id="srv_info"><span>{{ srv_info }}</span></div>
+	{%- endif %}
 
 	<div id="widget"></div>
 
 	<script>
-		var SR = "{{ r }}",
+		var SR = {{ r|tojson }},
 			CGV1 = {{ cgv1 }},
 			CGV = {{ cgv|tojson }},
 			TS = "{{ ts }}",

copyparty/web/idp.html

@@ -1,55 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-
-<head>
-	<meta charset="utf-8">
-	<title>{{ s_doctitle }}</title>
-	<meta http-equiv="X-UA-Compatible" content="IE=edge">
-	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <meta name="robots" content="noindex, nofollow">
-	<meta name="theme-color" content="#{{ tcolor }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/shares.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
-{{ html_head }}
-</head>
-
-<body>
-	<div id="wrap">
-		<a href="{{ r }}/?idp">refresh</a>
-		<a href="{{ r }}/?h">control-panel</a>
-
-        <table id="tab"><thead><tr>
-            <th>forget</th>
-            <th>user</th>
-            <th>groups</th>
-        </tr></thead><tbody>
-        {% for un, gn in rows %}
-        <tr>
-            <td><a href="{{ r }}/?idp=rm={{ un|e }}">forget</a></td>
-            <td>{{ un|e }}</td>
-            <td>{{ gn|e }}</td>
-        </tr>
-        {% endfor %}
-        </tbody></table>
-        {% if not rows %}
-        (there are no IdP users in the cache)
-        {% endif %}
-    </div>
-	<a href="#" id="repl">π</a>
-	<script>
-
-var SR="{{ r }}",
-	lang="{{ lang }}",
-	dfavico="{{ favico }}";
-
-var STG = window.localStorage;
-document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";
-
-</script>
-<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
-{%- if js %}
-<script src="{{ js }}_={{ ts }}"></script>
-{%- endif %}
-</body>
-</html>
-

copyparty/web/md.html

@@ -128,9 +128,9 @@ write markdown (most html is 🙆 too)
 
 	<script>
 
-var SR = "{{ r }}",
+var SR = {{ r|tojson }},
 	last_modified = {{ lastmod }},
-	have_emp = {{ "true" if have_emp else "false" }},
+	have_emp = {{ have_emp|tojson }},
 	dfavico = "{{ favico }}";
 
 var md_opt = {

copyparty/web/md.js

@@ -23,7 +23,8 @@ var dbg = function () { };
 
 // dodge browser issues
 (function () {
-    if (UA.indexOf(') Gecko/') !== -1 && /Linux| Mac /.exec(UA)) {
+    var ua = navigator.userAgent;
+    if (ua.indexOf(') Gecko/') !== -1 && /Linux| Mac /.exec(ua)) {
         // necessary on ff-68.7 at least
         var s = mknod('style');
         s.innerHTML = '@page { margin: .5in .6in .8in .6in; }';

copyparty/web/md2.js

@@ -255,7 +255,7 @@ function Modpoll() {
         }
 
         console.log('modpoll...');
-        var url = (location + '').split('?')[0] + '?_=' + Date.now();
+        var url = (document.location + '').split('?')[0] + '?_=' + Date.now();
         var xhr = new XHR();
         xhr.open('GET', url, true);
         xhr.responseType = 'text';
@@ -346,7 +346,7 @@ function save(e) {
         fd.append("lastmod", (force ? -1 : last_modified));
         fd.append("body", txt);
 
-        var url = (location + '').split('?')[0];
+        var url = (document.location + '').split('?')[0];
         var xhr = new XHR();
         xhr.open('POST', url, true);
         xhr.responseType = 'text';
@@ -404,7 +404,7 @@ function save_cb() {
 
 function run_savechk(lastmod, txt, btn, ntry) {
     // download the saved doc from the server and compare
-    var url = (location + '').split('?')[0] + '?_=' + Date.now();
+    var url = (document.location + '').split('?')[0] + '?_=' + Date.now();
     var xhr = new XHR();
     xhr.open('GET', url, true);
     xhr.responseType = 'text';
@@ -450,7 +450,7 @@ function savechk_cb() {
 
 // firefox bug: initial selection offset isn't cleared properly through js
 var ff_clearsel = (function () {
-    if (UA.indexOf(') Gecko/') === -1)
+    if (navigator.userAgent.indexOf(') Gecko/') === -1)
         return function () { }
 
     return function () {
@@ -1078,28 +1078,26 @@ action_stack = (function () {
         var p1 = from.length,
             p2 = to.length;
 
-        while (p1 --> 0 && p2 --> 0)
+        while (p1-- > 0 && p2-- > 0)
             if (from[p1] != to[p2])
                 break;
 
-        if (car > ++p1)
+        if (car > ++p1) {
             car = p1;
+        }
 
         var txt = from.substring(car, p1)
         return {
             car: car,
-            cdr: p2 + (car && 1),
+            cdr: ++p2,
             txt: txt,
             cpos: cpos
         };
     }
 
     var undiff = function (from, change) {
-        var t1 = from.substring(0, change.car),
-            t2 = from.substring(change.cdr);
-
         return {
-            txt: t1 + change.txt + t2,
+            txt: from.substring(0, change.car) + change.txt + from.substring(change.cdr),
             cpos: change.cpos
         };
     }

copyparty/web/mde.html

@@ -26,9 +26,9 @@
 	<a href="#" id="repl">π</a>
 	<script>
 
-var SR = "{{ r }}",
+var SR = {{ r|tojson }},
 	last_modified = {{ lastmod }},
-	have_emp = {{ "true" if have_emp else "false" }},
+	have_emp = {{ have_emp|tojson }},
 	dfavico = "{{ favico }}";
 
 var md_opt = {

copyparty/web/mde.js

@@ -6,7 +6,7 @@ var dom_doc = ebi('m');
 var dom_md = ebi('mt');
 
 (function () {
-    var n = location + '';
+    var n = document.location + '';
     n = (n.slice(n.indexOf('//') + 2).split('?')[0] + '?v').split('/');
     n[0] = 'top';
     var loc = [];
@@ -113,7 +113,7 @@ function save(mde) {
         fd.append("lastmod", (force ? -1 : last_modified));
         fd.append("body", txt);
 
-        var url = (location + '').split('?')[0];
+        var url = (document.location + '').split('?')[0];
         var xhr = new XHR();
         xhr.open('POST', url, true);
         xhr.responseType = 'text';
@@ -166,7 +166,7 @@ function save_cb() {
     //alert('save OK -- wrote ' + r.size + ' bytes.\n\nsha512: ' + r.sha512);
 
     // download the saved doc from the server and compare
-    var url = (location + '').split('?')[0] + '?_=' + Date.now();
+    var url = (document.location + '').split('?')[0] + '?_=' + Date.now();
     var xhr = new XHR();
     xhr.open('GET', url, true);
     xhr.responseType = 'text';