support hashed passwords; closes #39

This commit is contained in:
ed 2023-06-25 21:50:33 +00:00
parent cb75efa05d
commit e197895c10
14 changed files with 324 additions and 27 deletions

View file

@ -85,6 +85,7 @@ turn almost any device into a file server with resumable uploads/downloads using
* [security](#security) - some notes on hardening * [security](#security) - some notes on hardening
* [gotchas](#gotchas) - behavior that might be unexpected * [gotchas](#gotchas) - behavior that might be unexpected
* [cors](#cors) - cross-site request config * [cors](#cors) - cross-site request config
* [password hashing](#password-hashing) - you can hash passwords
* [https](#https) - both HTTP and HTTPS are accepted * [https](#https) - both HTTP and HTTPS are accepted
* [recovering from crashes](#recovering-from-crashes) * [recovering from crashes](#recovering-from-crashes)
* [client crashes](#client-crashes) * [client crashes](#client-crashes)
@ -1568,6 +1569,17 @@ by default, except for `GET` and `HEAD` operations, all requests must either:
cors can be configured with `--acao` and `--acam`, or the protections entirely disabled with `--allow-csrf` cors can be configured with `--acao` and `--acam`, or the protections entirely disabled with `--allow-csrf`
## password hashing
you can hash passwords before putting them into config files / providing them as arguments; see `--help-pwhash` for all the details
basically, specify `--ah-alg argon2` to enable the feature and it will print the hashed passwords on startup so you can replace the plaintext ones
optionally also specify `--ah-cli` to enter an interactive mode where it will hash passwords without ever writing the plaintext ones to disk
the default configs take about 0.4 sec to process a new password on a decent laptop
## https ## https
both HTTP and HTTPS are accepted by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default) both HTTP and HTTPS are accepted by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
@ -1615,6 +1627,8 @@ mandatory deps:
install these to enable bonus features install these to enable bonus features
enable hashed passwords in config: `argon2-cffi`
enable ftp-server: enable ftp-server:
* for just plaintext FTP, `pyftpdlib` (is built into the SFX) * for just plaintext FTP, `pyftpdlib` (is built into the SFX)
* with TLS encryption, `pyftpdlib pyopenssl` * with TLS encryption, `pyftpdlib pyopenssl`

View file

@ -15,6 +15,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
"libkeyfinder-git: detection of musical keys" "libkeyfinder-git: detection of musical keys"
"qm-vamp-plugins: BPM detection" "qm-vamp-plugins: BPM detection"
"python-pyopenssl: ftps functionality" "python-pyopenssl: ftps functionality"
"python-argon2_cffi: hashed passwords in config"
"python-impacket-git: smb support (bad idea)" "python-impacket-git: smb support (bad idea)"
) )
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz") source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")

View file

@ -1,4 +1,7 @@
{ lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, pillow, pyvips, ffmpeg, mutagen, { lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, argon2-cffi, pillow, pyvips, ffmpeg, mutagen,
# use argon2id-hashed passwords in config files (sha2 is always available)
withHashedPasswords ? true,
# create thumbnails with Pillow; faster than FFmpeg / MediaProcessing # create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
withThumbnails ? true, withThumbnails ? true,
@ -35,6 +38,7 @@ let
++ lib.optional withFastThumbnails pyvips ++ lib.optional withFastThumbnails pyvips
++ lib.optional withMediaProcessing ffmpeg ++ lib.optional withMediaProcessing ffmpeg
++ lib.optional withBasicAudioMetadata mutagen ++ lib.optional withBasicAudioMetadata mutagen
++ lib.optional withHashedPasswords argon2-cffi
); );
in stdenv.mkDerivation { in stdenv.mkDerivation {
pname = "copyparty"; pname = "copyparty";

View file

@ -258,6 +258,19 @@ def get_fk_salt(cert_path) -> str:
return ret.decode("utf-8") return ret.decode("utf-8")
def get_ah_salt() -> str:
fp = os.path.join(E.cfg, "ah-salt.txt")
try:
with open(fp, "rb") as f:
ret = f.read().strip()
except:
ret = base64.b64encode(os.urandom(18))
with open(fp, "wb") as f:
f.write(ret + b"\n")
return ret.decode("utf-8")
def ensure_locale() -> None: def ensure_locale() -> None:
safe = "en_US.UTF-8" safe = "en_US.UTF-8"
for x in [ for x in [
@ -622,6 +635,37 @@ def get_sects():
""" """
), ),
], ],
[
"pwhash",
"password hashing",
dedent(
"""
when \033[36m--ah-alg\033[0m is not the default [\033[32mnone\033[0m], all account passwords must be hashed
passwords can be hashed on the commandline with \033[36m--ah-gen\033[0m, but copyparty will also hash and print any passwords that are non-hashed (password which do not start with '+') and then terminate afterwards
\033[36m--ah-alg\033[0m specifies the hashing algorithm and a list of optional comma-separated arguments:
\033[36m--ah-alg argon2\033[0m # which is the same as:
\033[36m--ah-alg argon2,3,256,4,19\033[0m
use argon2id with timecost 3, 256 MiB, 4 threads, version 19 (0x13/v1.3)
\033[36m--ah-alg scrypt\033[0m # which is the same as:
\033[36m--ah-alg scrypt,13,2,8,4\033[0m
use scrypt with cost 2**13, 2 iterations, blocksize 8, 4 threads
\033[36m--ah-alg sha2\033[0m # which is the same as:
\033[36m--ah-alg sha2,424242\033[0m
use sha2-512 with 424242 iterations
recommended: \033[32m--ah-alg argon2\033[0m
argon2 needs python-package argon2-cffi,
scrypt needs openssl,
sha2 is always available
"""
),
],
] ]
@ -844,7 +888,7 @@ def add_optouts(ap):
ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (as specified by the 'lifetime' volflag)") ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (as specified by the 'lifetime' volflag)")
def add_safety(ap, fk_salt): def add_safety(ap):
ap2 = ap.add_argument_group('safety options') ap2 = ap.add_argument_group('safety options')
ap2.add_argument("-s", action="count", default=0, help="increase safety: Disable thumbnails / potentially dangerous software (ffmpeg/pillow/vips), hide partial uploads, avoid crawlers.\n └─Alias of\033[32m --dotpart --no-thumb --no-mtag-ff --no-robots --force-js") ap2.add_argument("-s", action="count", default=0, help="increase safety: Disable thumbnails / potentially dangerous software (ffmpeg/pillow/vips), hide partial uploads, avoid crawlers.\n └─Alias of\033[32m --dotpart --no-thumb --no-mtag-ff --no-robots --force-js")
ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --hardlink --vague-403 --ban-404=50,60,1440 -nih") ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --hardlink --vague-403 --ban-404=50,60,1440 -nih")
@ -852,8 +896,6 @@ def add_safety(ap, fk_salt):
ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m; example [\033[32m**,*,ln,p,r\033[0m]") ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m; example [\033[32m**,*,ln,p,r\033[0m]")
ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user has similar access (volflag=xvol)") ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user has similar access (volflag=xvol)")
ap2.add_argument("--xdev", action="store_true", help="stay within the filesystem of the volume root; do not descend into other devices (symlink or bind-mount to another HDD, ...) (volflag=xdev)") ap2.add_argument("--xdev", action="store_true", help="stay within the filesystem of the volume root; do not descend into other devices (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files -- this one DOES matter")
ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles") ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to make something a dotfile") ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to make something a dotfile")
ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings") ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings")
@ -870,6 +912,16 @@ def add_safety(ap, fk_salt):
ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like described in --acao)") ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like described in --acao)")
def add_salt(ap, fk_salt, ah_salt):
ap2 = ap.add_argument_group('salting options')
ap2.add_argument("--ah-alg", metavar="ALG", type=u, default="none", help="account-pw hashing algorithm; one of these, best to worst: argon2 scrypt sha2 none (each optionally followed by alg-specific comma-sep. config)")
ap2.add_argument("--ah-salt", metavar="SALT", type=u, default=ah_salt, help="account-pw salt; ignored if --ah-alg is none (default)")
ap2.add_argument("--ah-gen", metavar="PW", type=u, default="", help="generate hashed password for \033[33mPW\033[0m, or read passwords from STDIN if \033[33mPW\033[0m is [\033[32m-\033[0m]")
ap2.add_argument("--ah-cli", action="store_true", help="interactive shell which hashes passwords without ever storing or displaying the original passwords")
ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files")
ap2.add_argument("--warksalt", metavar="SALT", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
def add_shutdown(ap): def add_shutdown(ap):
ap2 = ap.add_argument_group('shutdown options') ap2 = ap.add_argument_group('shutdown options')
ap2.add_argument("--ign-ebind", action="store_true", help="continue running even if it's impossible to listen on some of the requested endpoints") ap2.add_argument("--ign-ebind", action="store_true", help="continue running even if it's impossible to listen on some of the requested endpoints")
@ -1030,6 +1082,7 @@ def run_argparse(
cert_path = os.path.join(E.cfg, "cert.pem") cert_path = os.path.join(E.cfg, "cert.pem")
fk_salt = get_fk_salt(cert_path) fk_salt = get_fk_salt(cert_path)
ah_salt = get_ah_salt()
hcores = min(CORES, 4) # optimal on py3.11 @ r5-4500U hcores = min(CORES, 4) # optimal on py3.11 @ r5-4500U
@ -1053,7 +1106,8 @@ def run_argparse(
add_ftp(ap) add_ftp(ap)
add_webdav(ap) add_webdav(ap)
add_smb(ap) add_smb(ap)
add_safety(ap, fk_salt) add_safety(ap)
add_salt(ap, fk_salt, ah_salt)
add_optouts(ap) add_optouts(ap)
add_shutdown(ap) add_shutdown(ap)
add_yolo(ap) add_yolo(ap)
@ -1138,16 +1192,22 @@ def main(argv: Optional[list[str]] = None) -> None:
supp = args_from_cfg(v) supp = args_from_cfg(v)
argv.extend(supp) argv.extend(supp)
deprecated: list[tuple[str, str]] = [] deprecated: list[tuple[str, str]] = [("--salt", "--warksalt")]
for dk, nk in deprecated: for dk, nk in deprecated:
try: idx = -1
idx = argv.index(dk) ov = ""
except: for n, k in enumerate(argv):
if k == dk or k.startswith(dk + "="):
idx = n
if "=" in k:
ov = "=" + k.split("=", 1)[1]
if idx < 0:
continue continue
msg = "\033[1;31mWARNING:\033[0;1m\n {} \033[0;33mwas replaced with\033[0;1m {} \033[0;33mand will be removed\n\033[0m" msg = "\033[1;31mWARNING:\033[0;1m\n {} \033[0;33mwas replaced with\033[0;1m {} \033[0;33mand will be removed\n\033[0m"
lprint(msg.format(dk, nk)) lprint(msg.format(dk, nk))
argv[idx] = nk argv[idx] = nk + ov
time.sleep(2) time.sleep(2)
da = len(argv) == 1 da = len(argv) == 1

View file

@ -15,6 +15,7 @@ from datetime import datetime
from .__init__ import ANYWIN, TYPE_CHECKING, WINDOWS from .__init__ import ANYWIN, TYPE_CHECKING, WINDOWS
from .bos import bos from .bos import bos
from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
from .pwhash import PWHash
from .util import ( from .util import (
IMPLICATIONS, IMPLICATIONS,
META_NOBOTS, META_NOBOTS,
@ -757,6 +758,7 @@ class AuthSrv(object):
warn_anonwrite: bool = True, warn_anonwrite: bool = True,
dargs: Optional[argparse.Namespace] = None, dargs: Optional[argparse.Namespace] = None,
) -> None: ) -> None:
self.ah = PWHash(args)
self.args = args self.args = args
self.dargs = dargs or args self.dargs = dargs or args
self.log_func = log_func self.log_func = log_func
@ -1134,6 +1136,8 @@ class AuthSrv(object):
self.log("\n{0}\n{1}{0}".format(t, "\n".join(slns))) self.log("\n{0}\n{1}{0}".format(t, "\n".join(slns)))
raise raise
self.setup_pwhash()
# case-insensitive; normalize # case-insensitive; normalize
if WINDOWS: if WINDOWS:
cased = {} cased = {}
@ -1574,6 +1578,10 @@ class AuthSrv(object):
self.log(t, 1) self.log(t, 1)
errors = True errors = True
if self.args.smb and self.ah.on and acct:
self.log("--smb can only be used when --ah-alg is none", 1)
errors = True
for vol in vfs.all_vols.values(): for vol in vfs.all_vols.values():
for k in list(vol.flags.keys()): for k in list(vol.flags.keys()):
if re.match("^-[^-]+$", k): if re.match("^-[^-]+$", k):
@ -1643,7 +1651,54 @@ class AuthSrv(object):
self.re_pwd = None self.re_pwd = None
pwds = [re.escape(x) for x in self.iacct.keys()] pwds = [re.escape(x) for x in self.iacct.keys()]
if pwds: if pwds:
self.re_pwd = re.compile("=(" + "|".join(pwds) + ")([]&; ]|$)") if self.ah.on:
zs = r"(\[H\] pw:.*|[?&]pw=)([^&]+)"
else:
zs = r"(\[H\] pw:.*|=)(" + "|".join(pwds) + r")([]&; ]|$)"
self.re_pwd = re.compile(zs)
def setup_pwhash(self) -> None:
self.ah = PWHash(self.args)
if self.ah.alg == "none":
return
if self.args.ah_cli:
self.ah.cli()
sys.exit()
elif self.args.ah_gen == "-":
self.ah.stdin()
sys.exit()
elif self.args.ah_gen:
print(self.ah.hash(self.args.ah_gen))
sys.exit()
if not self.args.a:
return
changed = False
for acct in self.args.a[:]:
uname, pw = acct.split(":", 1)
if pw.startswith("+") and len(pw) == 33:
continue
changed = True
hpw = self.ah.hash(pw)
self.args.a.remove(acct)
self.args.a.append("{}:{}".format(uname, hpw))
t = "hashed password for account {}: {}"
self.log(t.format(uname, hpw), 3)
if not changed:
return
lns = []
for acct in self.args.a:
uname, pw = acct.split(":", 1)
lns.append(" {}: {}".format(uname, pw))
t = "please use the following hashed passwords in your config:\n{}"
self.log(t.format("\n".join(lns)), 3)
def chk_sqlite_threadsafe(self) -> str: def chk_sqlite_threadsafe(self) -> str:
v = SQLITE_VER[-1:] v = SQLITE_VER[-1:]

View file

@ -79,10 +79,13 @@ class FtpAuth(DummyAuthorizer):
raise AuthenticationFailed("banned") raise AuthenticationFailed("banned")
asrv = self.hub.asrv asrv = self.hub.asrv
if username == "anonymous":
uname = "*" uname = "*"
else: if username != "anonymous":
uname = asrv.iacct.get(password, "") or asrv.iacct.get(username, "") or "*" for zs in (password, username):
zs = asrv.iacct.get(asrv.ah.hash(zs), "")
if zs:
uname = zs
break
if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)): if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
g = self.hub.gpwd g = self.hub.gpwd

View file

@ -173,13 +173,16 @@ class HttpCli(object):
def log(self, msg: str, c: Union[int, str] = 0) -> None: def log(self, msg: str, c: Union[int, str] = 0) -> None:
ptn = self.asrv.re_pwd ptn = self.asrv.re_pwd
if ptn and ptn.search(msg): if ptn and ptn.search(msg):
if self.asrv.ah.on:
msg = ptn.sub("\033[7m pw \033[27m", msg)
else:
msg = ptn.sub(self.unpwd, msg) msg = ptn.sub(self.unpwd, msg)
self.log_func(self.log_src, msg, c) self.log_func(self.log_src, msg, c)
def unpwd(self, m: Match[str]) -> str: def unpwd(self, m: Match[str]) -> str:
a, b = m.groups() a, b, c = m.groups()
return "=\033[7m {} \033[27m{}".format(self.asrv.iacct[a], b) return "{}\033[7m {} \033[27m{}".format(a, self.asrv.iacct[b], c)
def _check_nonfatal(self, ex: Pebkac, post: bool) -> bool: def _check_nonfatal(self, ex: Pebkac, post: bool) -> bool:
if post: if post:
@ -383,13 +386,14 @@ class HttpCli(object):
zs = base64.b64decode(zb).decode("utf-8") zs = base64.b64decode(zb).decode("utf-8")
# try "pwd", "x:pwd", "pwd:x" # try "pwd", "x:pwd", "pwd:x"
for bauth in [zs] + zs.split(":", 1)[::-1]: for bauth in [zs] + zs.split(":", 1)[::-1]:
if self.asrv.iacct.get(bauth): hpw = self.asrv.ah.hash(bauth)
if self.asrv.iacct.get(hpw):
break break
except: except:
pass pass
self.pw = uparam.get("pw") or self.headers.get("pw") or bauth or cookie_pw self.pw = uparam.get("pw") or self.headers.get("pw") or bauth or cookie_pw
self.uname = self.asrv.iacct.get(self.pw) or "*" self.uname = self.asrv.iacct.get(self.asrv.ah.hash(self.pw)) or "*"
self.rvol = self.asrv.vfs.aread[self.uname] self.rvol = self.asrv.vfs.aread[self.uname]
self.wvol = self.asrv.vfs.awrite[self.uname] self.wvol = self.asrv.vfs.awrite[self.uname]
self.mvol = self.asrv.vfs.amove[self.uname] self.mvol = self.asrv.vfs.amove[self.uname]
@ -1968,7 +1972,7 @@ class HttpCli(object):
return True return True
def get_pwd_cookie(self, pwd: str) -> str: def get_pwd_cookie(self, pwd: str) -> str:
if pwd in self.asrv.iacct: if self.asrv.ah.hash(pwd) in self.asrv.iacct:
msg = "login ok" msg = "login ok"
dur = int(60 * 60 * self.args.logout) dur = int(60 * 60 * self.args.logout)
else: else:

142
copyparty/pwhash.py Normal file
View file

@ -0,0 +1,142 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import base64
import hashlib
import sys
import threading
from .__init__ import unicode
class PWHash(object):
def __init__(self, args: argparse.Namespace):
self.args = args
try:
alg, ac = args.ah_alg.split(",")
except:
alg = args.ah_alg
ac = {}
self.alg = alg
self.ac = ac
if alg == "none":
self.on = False
self.hash = unicode
return
self.on = True
self.salt = args.ah_salt.encode("utf-8")
self.cache: dict[str, str] = {}
self.mutex = threading.Lock()
self.hash = self._cache_hash
if alg == "sha2":
self._hash = self._gen_sha2
elif alg == "scrypt":
self._hash = self._gen_scrypt
elif alg == "argon2":
self._hash = self._gen_argon2
else:
t = "unsupported password hashing algorithm [{}], must be one of these: argon2 scrypt sha2 none"
raise Exception(t.format(alg))
def _cache_hash(self, plain: str) -> str:
with self.mutex:
try:
return self.cache[plain]
except:
pass
if not plain:
return ""
if len(plain) > 255:
raise Exception("password too long")
if len(self.cache) > 9000:
self.cache = {}
ret = self._hash(plain)
self.cache[plain] = ret
return ret
def _gen_sha2(self, plain: str) -> str:
its = int(self.ac[0]) if self.ac else 424242
bplain = plain.encode("utf-8")
ret = b"\n"
for _ in range(its):
ret = hashlib.sha512(self.salt + bplain + ret).digest()
return "+" + base64.urlsafe_b64encode(ret[:24]).decode("utf-8")
def _gen_scrypt(self, plain: str) -> str:
cost = 2 << 13
its = 2
blksz = 8
para = 4
try:
cost = 2 << int(self.ac[0])
its = int(self.ac[1])
blksz = int(self.ac[2])
para = int(self.ac[3])
except:
pass
ret = plain.encode("utf-8")
for _ in range(its):
ret = hashlib.scrypt(ret, salt=self.salt, n=cost, r=blksz, p=para, dklen=24)
return "+" + base64.urlsafe_b64encode(ret).decode("utf-8")
def _gen_argon2(self, plain: str) -> str:
from argon2.low_level import Type as ArgonType
from argon2.low_level import hash_secret
time_cost = 3
mem_cost = 256
parallelism = 4
version = 19
try:
time_cost = int(self.ac[0])
mem_cost = int(self.ac[1])
parallelism = int(self.ac[2])
version = int(self.ac[3])
except:
pass
bplain = plain.encode("utf-8")
bret = hash_secret(
secret=bplain,
salt=self.salt,
time_cost=time_cost,
memory_cost=mem_cost * 1024,
parallelism=parallelism,
hash_len=24,
type=ArgonType.ID,
version=version,
)
ret = bret.split(b"$")[-1].decode("utf-8")
return "+" + ret.replace("/", "_").replace("+", "-")
def stdin(self) -> None:
while True:
ln = sys.stdin.readline().strip()
if not ln:
break
print(self.hash(ln))
def cli(self) -> None:
import getpass
while True:
p1 = getpass.getpass("password> ")
p2 = getpass.getpass("again or just hit ENTER> ")
if p2 and p1 != p2:
print("\033[31minputs don't match; try again\033[0m", file=sys.stderr)
continue
print(self.hash(p1))
print()

View file

@ -69,7 +69,7 @@ class U2idx(object):
fsize = body["size"] fsize = body["size"]
fhash = body["hash"] fhash = body["hash"]
wark = up2k_wark_from_hashlist(self.args.salt, fsize, fhash) wark = up2k_wark_from_hashlist(self.args.warksalt, fsize, fhash)
uq = "substr(w,1,16) = ? and w = ?" uq = "substr(w,1,16) = ? and w = ?"
uv: list[Union[str, int]] = [wark[:16], wark] uv: list[Union[str, int]] = [wark[:16], wark]

View file

@ -112,7 +112,7 @@ class Up2k(object):
self.args = hub.args self.args = hub.args
self.log_func = hub.log self.log_func = hub.log
self.salt = self.args.salt self.salt = self.args.warksalt
self.r_hash = re.compile("^[0-9a-zA-Z_-]{44}$") self.r_hash = re.compile("^[0-9a-zA-Z_-]{44}$")
self.gid = 0 self.gid = 0

View file

@ -6,6 +6,7 @@
* [up2k](#up2k) - quick outline of the up2k protocol * [up2k](#up2k) - quick outline of the up2k protocol
* [why not tus](#why-not-tus) - I didn't know about [tus](https://tus.io/) * [why not tus](#why-not-tus) - I didn't know about [tus](https://tus.io/)
* [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right? * [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
* [hashed passwords](#hashed-passwords) - regarding the curious decisions
* [http api](#http-api) * [http api](#http-api)
* [read](#read) * [read](#read)
* [write](#write) * [write](#write)
@ -68,14 +69,14 @@ regarding the frequent server log message during uploads;
* on this http connection, `2.77 GiB` transferred, `102.9 MiB/s` average, `948` chunks handled * on this http connection, `2.77 GiB` transferred, `102.9 MiB/s` average, `948` chunks handled
* client says `4` uploads OK, `0` failed, `3` busy, `1` queued, `10042 MiB` total size, `7198 MiB` and `00:01:09` left * client says `4` uploads OK, `0` failed, `3` busy, `1` queued, `10042 MiB` total size, `7198 MiB` and `00:01:09` left
## why not tus ### why not tus
I didn't know about [tus](https://tus.io/) when I made this, but: I didn't know about [tus](https://tus.io/) when I made this, but:
* up2k has the advantage that it supports parallel uploading of non-contiguous chunks straight into the final file -- [tus does a merge at the end](https://tus.io/protocols/resumable-upload.html#concatenation) which is slow and taxing on the server HDD / filesystem (unless i'm misunderstanding) * up2k has the advantage that it supports parallel uploading of non-contiguous chunks straight into the final file -- [tus does a merge at the end](https://tus.io/protocols/resumable-upload.html#concatenation) which is slow and taxing on the server HDD / filesystem (unless i'm misunderstanding)
* up2k has the slight disadvantage of requiring the client to hash the entire file before an upload can begin, but this has the benefit of immediately skipping duplicate files * up2k has the slight disadvantage of requiring the client to hash the entire file before an upload can begin, but this has the benefit of immediately skipping duplicate files
* and the hashing happens in a separate thread anyways so it's usually not a bottleneck * and the hashing happens in a separate thread anyways so it's usually not a bottleneck
## why chunk-hashes ### why chunk-hashes
a single sha512 would be better, right? a single sha512 would be better, right?
@ -92,6 +93,15 @@ hashwasm would solve the streaming issue but reduces hashing speed for sha512 (x
* blake2 might be a better choice since xxh is non-cryptographic, but that gets ~15 MiB/s on slower androids * blake2 might be a better choice since xxh is non-cryptographic, but that gets ~15 MiB/s on slower androids
# hashed passwords
regarding the curious decisions
there is a static salt for all passwords;
* because most copyparty APIs allow users to authenticate using only their password, making the username unknown, so impossible to do per-account salts
* the drawback of this is that an attacker can bruteforce all accounts in parallel, however most copyparty instances only have a handful of accounts in the first place, and it can be compensated by increasing the hashing cost anyways
# http api # http api
* table-column `params` = URL parameters; `?foo=bar&qux=...` * table-column `params` = URL parameters; `?foo=bar&qux=...`

View file

@ -48,6 +48,7 @@ thumbnails2 = ["pyvips"]
audiotags = ["mutagen"] audiotags = ["mutagen"]
ftpd = ["pyftpdlib"] ftpd = ["pyftpdlib"]
ftps = ["pyftpdlib", "pyopenssl"] ftps = ["pyftpdlib", "pyopenssl"]
pwhash = ["argon2-cffi"]
[project.scripts] [project.scripts]
copyparty = "copyparty.__main__:main" copyparty = "copyparty.__main__:main"

View file

@ -16,6 +16,8 @@ cat $f | awk '
h=0 h=0
}; };
}; };
/```/{o=!o}
o{next}
/^#/{s=1;rs=0;pr()} /^#/{s=1;rs=0;pr()}
/^#* *(nix package)/{rs=1} /^#* *(nix package)/{rs=1}
/^#* *(install on android|dev env setup|just the sfx|complete release|optional gpl stuff|nixos module)|`$/{s=rs} /^#* *(install on android|dev env setup|just the sfx|complete release|optional gpl stuff|nixos module)|`$/{s=rs}

View file

@ -140,6 +140,7 @@ args = {
"audiotags": ["mutagen"], "audiotags": ["mutagen"],
"ftpd": ["pyftpdlib"], "ftpd": ["pyftpdlib"],
"ftps": ["pyftpdlib", "pyopenssl"], "ftps": ["pyftpdlib", "pyopenssl"],
"pwhash": ["argon2-cffi"],
}, },
"entry_points": {"console_scripts": ["copyparty = copyparty.__main__:main"]}, "entry_points": {"console_scripts": ["copyparty = copyparty.__main__:main"]},
"scripts": ["bin/partyfuse.py", "bin/u2c.py"], "scripts": ["bin/partyfuse.py", "bin/u2c.py"],