add sftp server (powered by 39c3)

README.md

@@ -5,7 +5,7 @@
 turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser
 
 * server only needs Python (2 or 3), all dependencies optional
-* 🔌 protocols: [http](#the-browser) // [webdav](#webdav-server) // [ftp](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
+* 🔌 protocols: [http(s)](#the-browser) // [webdav](#webdav-server) // [sftp](#sftp-server) // [ftp(s)](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
 * 📱 [android app](#android-app) // [iPhone shortcuts](#ios-shortcuts)
 
 👉 **[Get started](#quickstart)!** or visit the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running on a nuc in my basement
@@ -14,7 +14,7 @@ turn almost any device into a file server with resumable uploads/downloads using
 
 🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm) // 👉 **[feature-showcase](https://a.ocv.me/pub/demo/showcase-hq.webm)** ([youtube](https://www.youtube.com/watch?v=15_-hgsX2V0))
 
-made in Norway 🇳🇴
+built in Norway 🇳🇴 with contributions from [not-norway](https://github.com/9001/copyparty/graphs/contributors)
 
 
 ## readme toc
@@ -1398,6 +1398,26 @@ config file example, which restricts FTP to only use ports 3921 and 12000-12099
 ```
 
 
+## sftp server
+
+goes roughly 700 MiB/s (slower than webdav and ftp)
+
+> this is **not** [ftps](#ftp-server) (which copyparty also supports); [ftps](#ftp-server) is ftp-tls (think http/https), while **sftp** is ssh-based and (preferably) uses ssh-keys for authentication
+
+the sftp-server requires the optional dependency [paramiko](https://pypi.org/project/paramiko/);
+* if you are **not** using docker, then install paramiko somehow
+* if you **are** using docker, then use one of the following image variants: `ac` / `iv` / `dj`
+
+enable sftpd with `--sftp 3922` to listen on port 3922;
+* use global-option `sftp-key` to associate an ssh-key with a user;
+  * commandline: `--sftp-key 'david ssh-ed25519 AAAAC3NzaC...'`
+  * config-file: `sftp-key: david ssh-ed25519 AAAAC3NzaC...`
+* `--sftp-pw` enables login with passwords (default is ssh-keys only)
+* `--sftp-anon foo` enables login with username `foo` and no password; gives the same access/permissions as the website does when not logged in
+
+see the [sftp section in --help](https://copyparty.eu/cli/#g-sftp) for the other options
+
+
 ## webdav server
 
 with read-write support,  supports winXP and later, macos, nautilus/gvfs  ... a great way to [access copyparty straight from the file explorer in your OS](#mount-as-drive)
@@ -3030,12 +3050,15 @@ set any of the following environment variables to disable its associated optiona
 | `PRTY_NO_FFPROBE`    | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips, **metadata-scanning** must be handled by mutagen |
 | `PRTY_NO_MAGIC`      | do not use [magic](https://pypi.org/project/python-magic/) for filetype detection |
 | `PRTY_NO_MUTAGEN`    | do not use [mutagen](https://pypi.org/project/mutagen/) for reading metadata from media files; will fallback to ffprobe |
+| `PRTY_NO_PARAMIKO`   | disable sftp server ([paramiko](https://www.paramiko.org/)-based) |
+| `PRTY_NO_PARTFTPY`   | disable tftp server ([partftpy](https://github.com/9001/partftpy)-based) |
 | `PRTY_NO_PIL`        | disable all [Pillow](https://pypi.org/project/pillow/)-based thumbnail support; will fallback to libvips or ffmpeg |
 | `PRTY_NO_PILF`       | disable Pillow `ImageFont` text rendering, used for folder thumbnails |
 | `PRTY_NO_PIL_AVIF`   | disable Pillow avif support (internal and/or [plugin](https://pypi.org/project/pillow-avif-plugin/)) |
 | `PRTY_NO_PIL_HEIF`   | disable 3rd-party Pillow plugin for [HEIF support](https://pypi.org/project/pillow-heif/) |
 | `PRTY_NO_PIL_WEBP`   | disable use of native webp support in Pillow |
 | `PRTY_NO_PSUTIL`     | do not use [psutil](https://pypi.org/project/psutil/) for reaping stuck hooks and plugins on Windows |
+| `PRTY_NO_PYFTPD`     | disable ftp(s) server ([pyftpdlib](https://pypi.org/project/pyftpdlib/)-based) |
 | `PRTY_NO_RAW`        | disable all [rawpy](https://pypi.org/project/rawpy/)-based thumbnail support for RAW images |
 | `PRTY_NO_VIPS`       | disable all [libvips](https://pypi.org/project/pyvips/)-based thumbnail support; will fallback to Pillow or ffmpeg |
 

contrib/package/arch/PKGBUILD

@@ -5,7 +5,7 @@
 pkgname=copyparty
 pkgver="1.19.23"
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -14,6 +14,7 @@ makedepends=("python-wheel" "python-setuptools" "python-build" "python-installer
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
             "cfssl: generate TLS certificates on startup"
             "python-mutagen: music tags (alternative)"
+            "python-paramiko: sftp server",
             "python-pillow: thumbnails for images"
             "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)"
             "libkeyfinder: detection of musical keys"

contrib/package/makedeb-mpr/PKGBUILD

@@ -4,7 +4,7 @@
 pkgname=copyparty
 pkgver=1.19.23
 pkgrel=1
-pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
+pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
 url="https://github.com/9001/${pkgname}"
 license=('MIT')
@@ -13,6 +13,7 @@ makedepends=("python3-wheel" "python3-setuptools" "python3-build" "python3-insta
 optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags"
             "golang-cfssl: generate TLS certificates on startup"
             "python3-mutagen: music tags (alternative)"
+            "python3-paramiko: sftp server"
             "python3-pil: thumbnails for images"
             "python3-openssl: ftps functionality"
             "python3-zmq: send zeromq messages from event-hooks"

contrib/package/nix/copyparty/default.nix

@@ -15,6 +15,7 @@
   pyzmq,
   ffmpeg,
   mutagen,
+  paramiko,
   pyftpdlib,
   magic,
   partftpy,
@@ -44,6 +45,9 @@
   # send ZeroMQ messages from event-hooks
   withZeroMQ ? true,
 
+  # enable SFTP server
+  withSFTP ? false,
+
   # enable FTP server
   withFTP ? true,
 
@@ -131,6 +135,7 @@ buildPythonApplication {
       fusepy
     ]
     ++ lib.optional withSMB impacket
+    ++ lib.optional withSFTP paramiko
     ++ lib.optional withFTP pyftpdlib
     ++ lib.optional withFTPS pyopenssl
     ++ lib.optional withTFTP partftpy
@@ -152,7 +157,7 @@ buildPythonApplication {
   meta = {
     description = "Turn almost any device into a file server";
     longDescription = ''
-      Portable file server with accelerated resumable uploads, dedup, WebDAV,
+      Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP,
       FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
     '';
     homepage = "https://github.com/9001/copyparty";

contrib/package/nix/overlay.nix

@@ -8,6 +8,7 @@ let
     withMediaProcessing = true;
     withBasicAudioMetadata = true;
     withZeroMQ = true;
+    withSFTP = true;
     withFTP = true;
     withFTPS = true;
     withTFTP = true;

contrib/package/rpm/copyparty.spec

@@ -5,7 +5,7 @@ License:        MIT
 Group:          Utilities
 URL:            https://github.com/9001/copyparty
 Source0:        copyparty-$pkgver.tar.gz
-Summary:        File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++
+Summary:        File server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++
 BuildArch:      noarch
 BuildRequires:  python3, python3-devel, pyproject-rpm-macros, python-setuptools, python-wheel, make
 Requires:       python3, (python3-jinja2 or python-jinja2), lsof
@@ -13,7 +13,7 @@ Recommends:     ffmpeg, (golang-github-cloudflare-cfssl or cfssl), python-mutage
 Recommends:     qm-vamp-plugins, python-argon2-cffi, (python-pyopenssl or pyopenssl), python-impacket
 
 %description
-Portable file server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
+Portable file server with accelerated resumable uploads, dedup, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps
 
 See release at https://github.com/9001/copyparty/releases
 

copyparty/__main__.py

@@ -48,6 +48,7 @@ from .util import (
     HAVE_IPV6,
     IMPLICATIONS,
     JINJA_VER,
+    MIKO_VER,
     MIMES,
     PARTFTPY_VER,
     PY_DESC,
@@ -1419,6 +1420,21 @@ def add_zc_ssdp(ap):
     ap2.add_argument("--zsid", metavar="UUID", type=u, default=zsid, help="USN (device identifier) to announce")
 
 
+def add_sftp(ap):
+    ap2 = ap.add_argument_group("SFTP options")
+    ap2.add_argument("--sftp", metavar="PORT", type=int, default=0, help="enable SFTP server on \033[33mPORT\033[0m, for example \033[32m3922")
+    ap2.add_argument("--sftpv", action="store_true", help="verbose")
+    ap2.add_argument("--sftpvv", action="store_true", help="verboser")
+    ap2.add_argument("--sftp4", action="store_true", help="only listen on IPv4")
+    ap2.add_argument("--sftp-key", metavar="U K", type=u, action="append", help="\033[34mREPEATABLE:\033[0m add ssh-key \033[33mK\033[0m for user \033[33mU\033[0m (username, space, key-type, space, base64); if user has multiple keys, then repeat this option for each key\n └─commandline example: --sftp-key 'david ssh-ed25519 AAAAC3NzaC...'\n └─config-file example: sftp-key: david ssh-ed25519 AAAAC3NzaC...")
+    ap2.add_argument("--sftp-key2u", action="append", help=argparse.SUPPRESS)
+    ap2.add_argument("--sftp-pw", action="store_true", help="allow password-authentication with sftp (not just ssh-keys)")
+    ap2.add_argument("--sftp-anon", metavar="TXT", type=u, default="", help="allow anonymous/unauthenticated connections with \033[33mTXT\033[0m as username")
+    ap2.add_argument("--sftp-hostk", metavar="FP", type=u, default=E.cfg, help="path to folder with hostkeys, for example 'ssh_host_rsa_key'; missing keys will be generated")
+    ap2.add_argument("--sftp-banner", metavar="T", type=u, default="", help="bannertext to send when someone connects; can be @filepath")
+    ap2.add_argument("--sftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m (comma-separated); specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m / \033[33m--ipar\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+
+
 def add_ftp(ap):
     ap2 = ap.add_argument_group("FTP options (TCP only)")
     ap2.add_argument("--ftp", metavar="PORT", type=int, default=0, help="enable FTP server on \033[33mPORT\033[0m, for example \033[32m3921")
@@ -1927,6 +1943,7 @@ def run_argparse(
     add_thumbnail(ap)
     add_transcoding(ap)
     add_rss(ap)
+    add_sftp(ap)
     add_ftp(ap)
     add_webdav(ap)
     add_tftp(ap)
@@ -1994,7 +2011,7 @@ def main(argv: Optional[list[str]] = None) -> None:
 
     init_E(E)
 
-    f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n   sqlite {} | jinja {} | pyftpd {} | tftp {}\n\033[0m'
+    f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n   sqlite {} | jinja {} | pyftpd {} | tftp {} | miko {}\n\033[0m'
     f = f.format(
         S_VERSION,
         CODENAME,
@@ -2004,6 +2021,7 @@ def main(argv: Optional[list[str]] = None) -> None:
         JINJA_VER,
         PYFTPD_VER,
         PARTFTPY_VER,
+        MIKO_VER,
     )
     lprint(f)
 

copyparty/ftpd.py

@@ -86,7 +86,7 @@ class FtpAuth(DummyAuthorizer):
             if args.usernames:
                 alts = ["%s:%s" % (username, password)]
             else:
-                alts = password, username
+                alts = [password, username]
 
             for zs in alts:
                 zs = asrv.iacct.get(asrv.ah.hash(zs), "")
@@ -249,7 +249,33 @@ class FtpFs(AbstractedFS):
                 need_unlink = False
                 td = 0
 
-        if w and need_unlink:
+            xbu = vfs.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.log,
+                    None,
+                    self.hub.up2k,
+                    "xbu.ftp",
+                    xbu,
+                    ap,
+                    filename,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "1.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (filename,)
+                    self.log(t, 3)
+                    raise FSE(t)
+
+        if w and need_unlink:  # type: ignore  # !rm
             assert td  # type: ignore  # !rm
             if td >= -1 and td <= self.args.ftp_wt:
                 # within permitted timeframe; allow overwrite or resume

copyparty/sftpd.py

@@ -0,0 +1,788 @@
+# coding: utf-8
+from __future__ import print_function, unicode_literals
+
+import errno
+import hashlib
+import logging
+import os
+import select
+import socket
+import time
+from threading import ExceptHookArgs
+
+import paramiko
+import paramiko.common
+import paramiko.sftp_attr
+from paramiko.common import AUTH_FAILED, AUTH_SUCCESSFUL
+from paramiko.sftp import SFTP_FAILURE, SFTP_OK, SFTP_PERMISSION_DENIED
+
+from .__init__ import ANYWIN, TYPE_CHECKING
+from .authsrv import LEELOO_DALLAS, VFS, AuthSrv
+from .bos import bos
+from .util import (
+    FN_EMB,
+    VF_CAREFUL,
+    Daemon,
+    ODict,
+    Pebkac,
+    ipnorm,
+    min_ex,
+    read_utf8,
+    relchk,
+    runhook,
+    sanitize_fn,
+    ub64enc,
+    undot,
+    vjoin,
+    wunlink,
+)
+
+if TYPE_CHECKING:
+    from .svchub import SvcHub
+
+if True:  # pylint: disable=using-constant-test
+    import typing
+    from typing import Any, Optional, Union
+
+SATTR = paramiko.sftp_attr.SFTPAttributes
+
+
+class SSH_Srv(paramiko.ServerInterface):
+    def __init__(self, hub: "SvcHub", addr: Any):
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.uname = "*"
+
+        self.addr = addr
+        self.ip = addr[0]
+        if self.ip.startswith("::ffff:"):
+            self.ip = self.ip[7:]
+
+        zsl = []
+        if args.sftp_anon:
+            zsl.append("none")
+        if args.sftp_key2u:
+            zsl.append("publickey")
+        if args.sftp_pw or args.sftp_anon:
+            zsl.append("password")
+        self._auths = ",".join(zsl)
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def get_allowed_auths(self, username: str) -> str:
+        return self._auths
+
+    def get_banner(self) -> tuple[Optional[str], Optional[str]]:
+        if self.args.sftpv:
+            self.log("get_banner")
+        t = self.args.sftp_banner
+        if not t:
+            return (None, None)
+        if t.startswith("@"):
+            t = read_utf8(self.log, t[1:], False)
+        if t and not t.endswith("\n"):
+            t += "\n"
+        return (t, "en-US")
+
+    def check_channel_request(self, kind: str, chanid: int) -> int:
+        if self.args.sftpv:
+            self.log("channel-request: %r, %r" % (kind, chanid))
+        if kind == "session":
+            return paramiko.common.OPEN_SUCCEEDED
+        return paramiko.common.OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
+
+    def check_auth_none(self, username: str) -> int:
+        try:
+            return self._check_auth_none(username)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_none(self, uname: str) -> int:
+        args = self.args
+        if uname != args.sftp_anon or not uname:
+            return AUTH_FAILED
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        self.uname = "*"
+        self.log("auth-none OK: *")
+        return AUTH_SUCCESSFUL
+
+    def check_auth_password(self, username: str, password: str) -> int:
+        try:
+            return self._check_auth_password(username, password)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_password(self, uname: str, pw: str) -> int:
+        args = self.args
+        if args.sftpv:
+            logpw = pw
+            if args.log_badpwd == 0:
+                logpw = ""
+            elif args.log_badpwd == 2:
+                zb = hashlib.sha512(pw.encode("utf-8", "replace")).digest()
+                logpw = "%" + ub64enc(zb[:12]).decode("ascii")
+            self.log("auth-pw: %r, %r" % (uname, logpw))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-pw OK: *")
+            return AUTH_SUCCESSFUL
+
+        if not args.sftp_pw:
+            return AUTH_FAILED
+
+        if args.usernames:
+            alts = ["%s:%s" % (uname, pw)]
+        else:
+            alts = [pw, uname]
+
+        attempt = "%s:%s" % (uname, pw)
+        uname = ""
+        asrv = self.hub.asrv
+        for zs in alts:
+            zs = asrv.iacct.get(asrv.ah.hash(zs), "")
+            if zs:
+                uname = zs
+                break
+
+        if args.ipu and uname == "*":
+            uname = args.ipu_iu[args.ipu_nm.map(self.ip)]
+        if args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            g = self.hub.gpwd
+            if g.lim:
+                bonk, ip = g.bonk(self.ip, attempt)
+                if bonk:
+                    logging.warning("client banned: invalid passwords")
+                    bans[self.ip] = bonk
+                    try:
+                        # only possible if multiprocessing disabled
+                        self.hub.broker.httpsrv.bans[ip] = bonk  # type: ignore
+                        self.hub.broker.httpsrv.nban += 1  # type: ignore
+                    except:
+                        pass
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-pw OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+    def check_auth_publickey(self, username: str, key: paramiko.PKey) -> int:
+        try:
+            return self._check_auth_publickey(username, key)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return AUTH_FAILED
+
+    def _check_auth_publickey(self, uname: str, key: paramiko.PKey) -> int:
+        args = self.args
+        if args.sftpv:
+            zs = key.get_name() + "," + key.get_base64()[:32]
+            self.log("auth-key: %r, %r" % (uname, zs))
+
+        ipn = ipnorm(self.ip)
+        bans = self.hub.bans
+        if ipn in bans:
+            rt = bans[ipn] - time.time()
+            if rt < 0:
+                self.log("client unbanned")
+                del bans[ipn]
+            else:
+                self.log("client is banned")
+                return AUTH_FAILED
+
+        anon = args.sftp_anon
+        if anon and uname == anon:
+            self.uname = "*"
+            self.log("auth-key OK: *")
+            return AUTH_SUCCESSFUL
+
+        attempt = "%s %s" % (key.get_name(), key.get_base64())
+        ok = args.sftp_key2u.get(attempt) == uname
+
+        if ok and args.ipr and uname in args.ipr_u:
+            if not args.ipr_u[uname].map(self.ip):
+                logging.warning("username [%s] rejected by --ipr", uname)
+                return AUTH_FAILED
+
+        asrv = self.hub.asrv
+        if not ok or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
+            self.log("auth-key REJECTED: %s" % (uname,))
+            return AUTH_FAILED
+
+        self.uname = uname
+        self.log("auth-key OK: %s" % (uname,))
+        return AUTH_SUCCESSFUL
+
+
+class SFTP_FH(paramiko.SFTPHandle):
+    def stat(self):
+        try:
+            return SATTR.from_stat(os.fstat(self.readfile.fileno()))
+        except OSError as ex:
+            print("a", repr(ex))
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, attr):
+        # python doesn't have equivalents to fchown or fchmod, so we have to
+        # use the stored filename
+        if not self.writefile:
+            return SFTP_PERMISSION_DENIED
+        try:
+            paramiko.SFTPServer.set_file_attr(self.filename, attr)
+            return SFTP_OK
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+
+class SFTP_Srv(paramiko.SFTPServerInterface):
+    def __init__(self, ssh: paramiko.ServerInterface, *a, **ka):
+        super(SFTP_Srv, self).__init__(ssh, *a, **ka)
+        self.ssh = ssh
+        self.ip: str = ssh.ip  # type: ignore
+        self.hub: "SvcHub" = ssh.hub  # type: ignore
+        self.uname: str = ssh.uname  # type: ignore
+        self.args = self.hub.args
+        self.asrv: "AuthSrv" = self.hub.asrv
+
+        if self.uname == LEELOO_DALLAS:
+            raise Exception("send her back")
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp:%s" % (self.ip,), msg, c)
+
+    def v2a(
+        self,
+        vpath: str,
+        r: bool = False,
+        w: bool = False,
+        m: bool = False,
+        d: bool = False,
+    ) -> tuple[str, VFS, str]:
+        vpath = vpath.replace(os.sep, "/").strip("/")
+        rd, fn = os.path.split(vpath)
+        if relchk(rd):
+            self.log("malicious vpath: %s", vpath)
+            raise Exception("Unsupported characters in [%s]" % (vpath,))
+
+        fn = sanitize_fn(fn or "")
+        vpath = vjoin(rd, fn)
+        vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+        if (
+            w
+            and fn.lower() in FN_EMB
+            and self.uname not in vn.axs.uread
+            and "wo_up_readme" not in vn.flags
+        ):
+            fn = "_wo_" + fn
+            vpath = vjoin(rd, fn)
+            vn, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+
+        if not vn.realpath:
+            # return "", vn, rem
+            raise OSError(errno.ENOENT, "no filesystem mounted at [/%s]" % (vpath,))
+
+        if "xdev" in vn.flags or "xvol" in vn.flags:
+            ap = vn.canonical(rem)
+            avn = vn.chk_ap(ap)
+            t = "Permission denied in [{}]"
+            if not avn:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+
+            cr, cw, cm, cd, _, _, _, _, _ = avn.uaxs[self.uname]
+            if r and not cr or w and not cw or m and not cm or d and not cd:
+                raise OSError(errno.EPERM, "permission denied in [/%s]" % (vpath,))
+
+        if "bcasechk" in vn.flags and not vn.casechk(rem, True):
+            raise OSError(errno.ENOENT, "file does not exist case-sensitively")
+
+        return os.path.join(vn.realpath, rem), vn, rem
+
+    def list_folder(self, path: str) -> list[SATTR] | int:
+        try:
+            return self._list_folder(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _list_folder(self, path: str) -> list[SATTR] | int:
+        try:
+            ap, vn, rem = self.v2a(path, True, False, False, False)
+        except Pebkac:
+            try:
+                self.v2a(path, False, True, False, False)
+                return []  # display write-only folders as empty
+            except:
+                pass
+            if self.asrv.vfs.realpath or path.strip("/"):
+                return SFTP_PERMISSION_DENIED
+            # list of accessible volumes
+            ret = []
+            zi = int(time.time())
+            vst = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+            for vn in self.asrv.vfs.all_vols.values():
+                if "/" in vn.vpath or not vn.vpath:
+                    continue  # only include toplevel-mounted vols
+                try:
+                    self.hub.asrv.vfs.get(vn.vpath, self.uname, True, False)
+                    ret.append(SATTR.from_stat(vst, filename=vn.vpath))
+                except:
+                    pass
+            ret.sort(key=lambda x: x.filename)
+            return ret
+
+        _, vfs_ls, vfs_virt = vn.ls(
+            rem,
+            self.uname,
+            not self.args.no_scandir,
+            [[True, False], [False, True]],
+            throw=True,
+        )
+        ret = [SATTR.from_stat(x[1], filename=x[0]) for x in vfs_ls]
+        for zs, vn2 in vfs_virt.items():
+            if not vn2.realpath:
+                continue
+            st = bos.stat(vn2.realpath)
+            ret.append(SATTR.from_stat(st, filename=zs))
+        if self.uname not in vn.axs.udot:
+            ret = [x for x in ret if not x.filename.split("/")[-1].startswith(".")]
+        ret.sort(key=lambda x: x.filename)
+        return ret
+
+    def stat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def lstat(self, path: str) -> SATTR | int:
+        try:
+            return self._stat(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _stat(self, vp: str) -> SATTR | int:
+        try:
+            ap = self.v2a(vp, True, False, False, False)[0]
+            st = bos.stat(ap)
+        except:
+            if vp.strip("/") or self.asrv.vfs.realpath:
+                try:
+                    self.v2a(vp, False, True, False, False)[0]
+                except:
+                    return SFTP_PERMISSION_DENIED
+            zi = int(time.time())
+            st = os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, zi, zi, zi))
+        return SATTR.from_stat(st)
+
+    def open(self, path: str, flags: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        try:
+            return self._open(path, flags, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _open(self, vp: str, iflag: int, attr: SATTR) -> paramiko.SFTPHandle | int:
+        if ANYWIN:
+            iflag |= os.O_BINARY
+        if iflag & os.O_WRONLY:
+            rd = False
+            wr = True
+            if iflag & os.O_APPEND:
+                smode = "ab"
+            else:
+                smode = "wb"
+        elif iflag & os.O_RDWR:
+            rd = wr = True
+            if iflag & os.O_APPEND:
+                smode = "a+b"
+            else:
+                smode = "r+b"
+        else:
+            rd = True
+            wr = False
+            smode = "rb"
+
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, rd, wr)
+            ap = os.path.join(vn.realpath, rem)
+            vf = vn.flags
+        except Pebkac as ex:
+            t = "denied open file [%s], iflag=%s, attr=%s, read=%s, write=%s: %s"
+            self.log(t % (vp, iflag, attr, rd, wr, ex))
+            return SFTP_PERMISSION_DENIED
+
+        if wr:
+            try:
+                st = bos.stat(ap)
+                td = time.time() - st.st_mtime
+                need_unlink = True
+            except:
+                need_unlink = False
+                td = 0
+
+            xbu = vn.flags.get("xbu")
+            if xbu:
+                hr = runhook(
+                    self.log,
+                    None,
+                    self.hub.up2k,
+                    "xbu.sftp",
+                    xbu,
+                    ap,
+                    vp,
+                    "",
+                    "",
+                    "",
+                    0,
+                    0,
+                    "7.3.8.7",
+                    time.time(),
+                    None,
+                )
+                t = hr.get("rejectmsg") or ""
+                if t or hr.get("rc") != 0:
+                    if not t:
+                        t = "upload blocked by xbu server config: %r" % (vp,)
+                    self.log(t, 3)
+                    return SFTP_PERMISSION_DENIED
+
+        if wr and need_unlink:  # type: ignore  # !rm
+            assert td  # type: ignore  # !rm
+            if td >= -1 and td <= self.args.ftp_wt:
+                # within permitted timeframe; allow overwrite or resume
+                do_it = True
+            elif self.args.no_del or self.args.ftp_no_ow:
+                # file too old, or overwrite not allowed; reject
+                do_it = False
+            else:
+                # allow overwrite if user has delete permission
+                do_it = self.uname in vn.axs.udel
+
+            if not do_it:
+                t = "file already exists and no permission to overwrite: %s"
+                self.log(t % (vp,))
+                return SFTP_PERMISSION_DENIED
+
+            # Don't unlink file for append mode
+            elif "a" not in smode:
+                wunlink(self.log, ap, VF_CAREFUL)
+
+        chmod = getattr(attr, "st_mode", None)
+        if chmod is None:
+            chmod = vf.get("chmod_f", 644)
+
+        try:
+            fd = os.open(ap, iflag, chmod)
+        except OSError as ex:
+            t = "failed to os.open [%s] -> [%s] with iflag [%s] and chmod [%s]"
+            self.log(t % (vp, ap, iflag, chmod), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        if iflag & os.O_CREAT:
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+
+        try:
+            f = os.fdopen(fd, smode)
+        except OSError as ex:
+            t = "failed to os.fdpen [%s] -> [%s] with smode [%s]"
+            self.log(t % (vp, ap, smode), 3)
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+        ret = SFTP_FH(iflag)
+        ret.filename = ap
+        ret.readfile = f if rd else None
+        ret.writefile = f if wr else None
+        return ret
+
+    def remove(self, path: str) -> int:
+        try:
+            return self._remove(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _remove(self, vp: str) -> int:
+        if self.args.no_del:
+            self.log("The delete feature is disabled in server config")
+            return SFTP_PERMISSION_DENIED
+        try:
+            self.hub.up2k.handle_rm(self.uname, self.ip, [vp], [], False, False)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied delete [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rename(self, oldpath: str, newpath: str) -> int:
+        try:
+            return self._rename(oldpath, newpath)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rename(self, svp: str, dvp: str) -> int:
+        if self.args.no_mv:
+            self.log("The rename/move feature is disabled in server config")
+        svp = svp.strip("/")
+        dvp = dvp.strip("/")
+        try:
+            self.hub.up2k.handle_mv("", self.uname, self.ip, svp, dvp)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rename [%s] to [%s]: %s"
+            self.log(t % (svp, dvp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def mkdir(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._mkdir(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _mkdir(self, vp: str, attr: SATTR) -> int:
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True)
+            ap = os.path.join(vn.realpath, rem)
+            bos.makedirs(ap, vf=vn.flags)  # filezilla expects this
+            if attr is not None:
+                paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied mkdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def rmdir(self, path: str) -> int:
+        try:
+            return self._rmdir(path)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _rmdir(self, vp: str) -> int:
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, False, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            bos.rmdir(ap)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied rmdir [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def chattr(self, path: str, attr: SATTR) -> int:
+        try:
+            return self._chattr(path, attr)
+        except:
+            self.log("unhandled exception: %s" % (min_ex(),), 1)
+            return SFTP_FAILURE
+
+    def _chattr(self, vp: str, attr: SATTR) -> int:
+        try:
+            vn, rem = self.asrv.vfs.get(vp, self.uname, False, True, will_del=True)
+            ap = os.path.join(vn.realpath, rem)
+            paramiko.SFTPServer.set_file_attr(ap, attr)
+            return SFTP_OK
+        except Pebkac as ex:
+            t = "denied chattr [%s]: %s"
+            self.log(t % (vp, ex))
+            return SFTP_PERMISSION_DENIED
+        except OSError as ex:
+            return paramiko.SFTPServer.convert_errno(ex.errno)
+
+    def symlink(self, target_path: str, path: str) -> int:
+        return paramiko.SFTPServer.SFTP_OP_UNSUPPORTED
+
+    def readlink(self, path: str) -> str | int:
+        return path
+
+    def canonicalize(self, path: str) -> str:
+        return "/%s" % (undot(path),)
+
+
+class Sftpd(object):
+    def __init__(self, hub: "SvcHub") -> None:
+        self.hub = hub
+        self.args = args = hub.args
+        self.log_func = hub.log
+        self.srv: list[socket.socket] = []
+        self.bound: list[str] = []
+        self.sessions = {}
+
+        ips = args.i
+        if "::" in ips:
+            ips.append("0.0.0.0")
+
+        ips = [x for x in ips if not x.startswith(("unix:", "fd:"))]
+
+        if args.sftp4:
+            ips = [x for x in ips if ":" not in x]
+
+        if not ips:
+            self.log("cannot start sftp-server; no compatible IPs in -i", 1)
+            return
+
+        self.hostkeys = []
+        hostkeytypes = (
+            ("ed25519", "Ed25519Key", {}),  # best
+            ("ecdsa", "ECDSAKey", {"bits": 384}),
+            ("rsa", "RSAKey", {"bits": 4096}),
+            ("dsa", "DSSKey", {}),  # worst
+        )
+        for fname, aname, opts in hostkeytypes:
+            fpath = "%s/ssh_host_%s_key" % (args.sftp_hostk, fname.lower())
+            try:
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            except Exception as ex:
+                try:
+                    genfun = getattr(paramiko, aname).generate
+                except Exception as ex2:
+                    if args.sftpv or fname not in ("dsa", "ed25519"):
+                        # dsa dropped in 4.0
+                        # ed25519 not supported yet
+                        self.log("cannot generate %s hostkey: %r" % (aname, ex2), 3)
+                    continue
+                self.log("generating hostkey [%s] due to %r" % (fpath, ex))
+                pkey = genfun(**opts)
+                pkey.write_private_key_file(fpath)
+                pkey = getattr(paramiko, aname).from_private_key_file(fpath)
+            self.hostkeys.append(pkey)
+            if args.sftpv:
+                self.log("loaded hostkey %r" % (pkey,))
+
+        ips = list(ODict.fromkeys(ips))  # dedup
+
+        for ip in ips:
+            self._bind(ip)
+
+        self.log("listening on %s port %s" % (self.srv, args.sftp))
+
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("sftp", msg, c)
+
+    def _bind(self, ip: str) -> None:
+        port = self.args.sftp
+        try:
+            ipv = socket.AF_INET6 if ":" in ip else socket.AF_INET
+            srv = socket.socket(ipv, socket.SOCK_STREAM)
+            if not ANYWIN or self.args.reuseaddr:
+                srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            srv.settimeout(0)  # == srv.setblocking(False)
+            try:
+                srv.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False)
+            except:
+                pass  # will create another ipv4 socket instead
+            if getattr(self.args, "freebind", False):
+                srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
+            srv.bind((ip, port))
+            srv.listen(10)
+            self.srv.append(srv)
+            self.bound.append(ip)
+        except Exception as ex:
+            if ip == "0.0.0.0" and "::" in self.bound:
+                return  # dualstack
+            self.log("could not listen on (%s,%s): %r" % (ip, port, ex), 3)
+
+    def _accept(self, srv: socket.socket) -> None:
+        cli, addr = srv.accept()
+        # cli.settimeout(0)  # == srv.setblocking(False)
+        self.log("%r is connecting" % (addr,))
+        zs = "sftp-%s" % (addr[0],)
+        # Daemon(self._accept2, zs, (cli, addr))
+        self._accept2(cli, addr)
+
+    def _accept2(self, cli, addr) -> None:
+        tra = paramiko.Transport(cli)
+        for hkey in self.hostkeys:
+            tra.add_server_key(hkey)
+        tra.set_subsystem_handler("sftp", paramiko.SFTPServer, SFTP_Srv)
+        psrv = SSH_Srv(self.hub, addr)
+        try:
+            tra.start_server(server=psrv)
+        except Exception as ex:
+            self.log("%r could not establish connection: %r" % (addr, ex), 3)
+            cli.close()
+            return
+
+        chan = tra.accept()
+        if chan is None:
+            self.log("%r did not open an sftp channel" % (addr,), 3)
+            cli.close()
+            return
+
+        self.sessions[addr] = (chan, tra, psrv)
+        # tra.join()
+        # self.log("%r disconnected" % (addr,))
+
+    def run(self):
+        lgr = logging.getLogger("paramiko.transport")
+        lgr.setLevel(logging.DEBUG if self.args.sftpvv else logging.INFO)
+
+        if self.args.no_poll:
+            fun = self._run_select
+        else:
+            fun = self._run_poll
+        Daemon(fun, "sftpd")
+
+    def _run_select(self):
+        while not self.hub.stopping:
+            rx, _, _ = select.select(self.srv, [], [], 180)
+            for sck in rx:
+                self._accept(sck)
+
+    def _run_poll(self):
+        fd2sck = {}
+        poll = select.poll()
+        for sck in self.srv:
+            fd = sck.fileno()
+            fd2sck[fd] = sck
+            poll.register(fd, select.POLLIN)
+        while not self.hub.stopping:
+            pr = poll.poll(180 * 1000)
+            rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
+            for sck in rx:
+                self._accept(sck)

copyparty/svchub.py

@@ -413,6 +413,11 @@ class SvcHub(object):
         if not args.http_only:
             zms += "D"
 
+        if args.sftp:
+            from .sftpd import Sftpd
+
+            self.sftpd: Optional[Sftpd] = None
+
         if args.ftp or args.ftps:
             from .ftpd import Ftpd
 
@@ -424,7 +429,7 @@ class SvcHub(object):
 
             self.tftpd: Optional[Tftpd] = None
 
-        if args.ftp or args.ftps or args.tftp:
+        if args.sftp or args.ftp or args.ftps or args.tftp:
             Daemon(self.start_ftpd, "start_tftpd")
 
         if args.smb:
@@ -751,12 +756,28 @@ class SvcHub(object):
     def start_ftpd(self) -> None:
         time.sleep(30)
 
+        if hasattr(self, "sftpd") and not self.sftpd:
+            self.restart_sftpd()
+
         if hasattr(self, "ftpd") and not self.ftpd:
             self.restart_ftpd()
 
         if hasattr(self, "tftpd") and not self.tftpd:
             self.restart_tftpd()
 
+    def restart_sftpd(self) -> None:
+        if not hasattr(self, "sftpd"):
+            return
+
+        from .sftpd import Sftpd
+
+        if self.sftpd:
+            return  # todo
+
+        self.sftpd = Sftpd(self)
+        self.sftpd.run()
+        self.log("root", "started SFTPd")
+
     def restart_ftpd(self) -> None:
         if not hasattr(self, "ftpd"):
             return
@@ -893,9 +914,9 @@ class SvcHub(object):
             return
 
         ar = self.args
-        for _ in range(10 if ar.ftp or ar.ftps else 0):
+        for _ in range(10 if ar.sftp or ar.ftp or ar.ftps else 0):
             time.sleep(0.03)
-            if self.ftpd:
+            if self.ftpd if ar.ftp or ar.ftps else ar.sftp:
                 break
 
         if self.tcpsrv.qr:
@@ -1147,9 +1168,15 @@ class SvcHub(object):
             zs = zs[3:]
         al.idp_chsub_tr = umktrans(zs1, zs2)
 
+        al.sftp_ipa_nm = build_netmap(al.sftp_ipa or al.ipa or al.ipar, True)
         al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa or al.ipar, True)
         al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa or al.ipar, True)
 
+        al.sftp_key2u = {
+            "%s %s" % (x[1], x[2]): x[0]
+            for x in [x.split(" ") for x in al.sftp_key or []]
+        }
+
         mte = ODict.fromkeys(DEF_MTE.split(","), True)
         al.mte = odfusion(mte, al.mte)
 

copyparty/tcpsrv.py

@@ -422,6 +422,7 @@ class TcpSrv(object):
         self.hub.broker.say("httpsrv.set_netdevs", self.netdevs)
         self.hub.start_zeroconf()
         gencert(self.log, self.args, self.netdevs)
+        self.hub.restart_sftpd()
         self.hub.restart_ftpd()
         self.hub.restart_tftpd()
 

copyparty/util.py

@@ -654,22 +654,41 @@ except:
     JINJA_VER = "(None)"
 
 try:
+    if os.environ.get("PRTY_NO_PYFTPD"):
+        raise Exception()
+
     from pyftpdlib.__init__ import __ver__ as PYFTPD_VER
 except:
     PYFTPD_VER = "(None)"
 
 try:
+    if os.environ.get("PRTY_NO_PARTFTPY"):
+        raise Exception()
+
     from partftpy.__init__ import __version__ as PARTFTPY_VER
 except:
     PARTFTPY_VER = "(None)"
 
+try:
+    if os.environ.get("PRTY_NO_PARAMIKO"):
+        raise Exception()
+
+    from paramiko import __version__ as MIKO_VER
+except:
+    MIKO_VER = "(None)"
+
 
 PY_DESC = py_desc()
 
-VERSIONS = (
+VERSIONS = "copyparty v{} ({})\n{}\n   sqlite {} | jinja {} | pyftpd {} | tftp {} | miko {}".format(
-    "copyparty v{} ({})\n{}\n   sqlite {} | jinja {} | pyftpd {} | tftp {}".format(
+    S_VERSION,
-        S_VERSION, S_BUILD_DT, PY_DESC, SQLITE_VER, JINJA_VER, PYFTPD_VER, PARTFTPY_VER
+    S_BUILD_DT,
-    )
+    PY_DESC,
+    SQLITE_VER,
+    JINJA_VER,
+    PYFTPD_VER,
+    PARTFTPY_VER,
+    MIKO_VER,
 )
 
 

docs/devnotes.md

@@ -367,6 +367,7 @@ pip install jinja2 strip_hints  # MANDATORY
 pip install argon2-cffi  # password hashing
 pip install pyzmq  # send 0mq from hooks
 pip install mutagen  # audio metadata
+pip install paramiko  # sftp server
 pip install pyftpdlib  # ftp server
 pip install partftpy  # tftp server
 pip install impacket  # smb server -- disable Windows Defender if you REALLY need this on windows
@@ -377,7 +378,7 @@ pip install black==21.12b0 click==8.0.2 bandit pylint flake8 isort mypy  # vscod
 ```
 
 * on archlinux you can do this:
-  * `sudo pacman -Sy --needed python-{pip,isort,jinja,argon2-cffi,pyzmq,mutagen,pyftpdlib,pillow}`
+  * `sudo pacman -Sy --needed python-{pip,isort,jinja,argon2-cffi,pyzmq,mutagen,paramiko,pyftpdlib,pillow}`
   * then, as user: `python3 -m pip install --user --break-system-packages -U strip_hints black==21.12b0 click==8.0.2`
   * for building docker images: `sudo pacman -Sy --needed qemu-user-static{,-binfmt} podman{,-docker} jq`
 

docs/versus.md

@@ -242,7 +242,7 @@ symbol legend,
 | serve ftp  (tcp)        | █ |   |   |   |   | █ |   |   |   |   |   | █ | █ |
 | serve ftps (tls)        | █ |   |   |   |   | █ |   |   |   |   |   | █ |   |
 | serve tftp (udp)        | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| serve sftp (ssh)        |   |   |   |   |   | █ |   |   |   |   |   | █ | █ |
+| serve sftp (ssh)        | █ |   |   |   |   | █ |   |   |   |   |   | █ | █ |
 | serve smb/cifs          | ╱ |   |   |   |   | █ |   |   |   |   |   |   |   |
 | serve dlna              |   |   |   |   |   | █ |   |   |   |   |   |   |   |
 | listen on unix-socket   | █ |   |   | █ | █ |   | █ | █ | █ | █ | █ | █ |   |
@@ -640,8 +640,7 @@ symbol legend,
 * ⚠️ impractical directory URLs
 * ⚠️ AGPL licensed
 * 🔵 uploading small files is fast; `340` files per sec (copyparty does `670`/sec)
-* 🔵 ftp, ftps, webdav
+* 🔵 sftp, ftp, ftps, webdav
-* ✅ sftp server
 * ✅ settings gui
 * ✅ acme (automatic tls certs)
   * 💾 relies on caddy/certbot/acme.sh
@@ -667,7 +666,6 @@ symbol legend,
 * ⚠️ not self-contained (pulls from jsdelivr)
 * ⚠️ has an audio player, but supports less filetypes
 * ⚠️ limited support for configuring real-ip detection
-* ✅ sftp server
 * ✅ settings gui
 * ✅ good-looking gui
 * ✅ an IDE, msoffice viewer, rich host integration, much more

pyproject.toml

@@ -2,7 +2,7 @@
 name = "copyparty"
 description = """
   Portable file server with accelerated resumable uploads, \
-  deduplication, WebDAV, FTP, zeroconf, media indexer, \
+  deduplication, WebDAV, SFTP, FTP, zeroconf, media indexer, \
   video thumbnails, audio transcoding, and write-only folders"""
 readme = "README.md"
 authors = [{ name = "ed", email = "copyparty@ocv.me" }]
@@ -47,6 +47,7 @@ classifiers = [
 [project.optional-dependencies]
 all = [
     "argon2-cffi",
+    "paramiko",
     "partftpy>=0.4.0",
     "Pillow",
     "pyftpdlib",
@@ -56,6 +57,7 @@ all = [
 thumbnails = ["Pillow"]
 thumbnails2 = ["pyvips"]
 audiotags = ["mutagen"]
+sftp = ["paramiko"]
 ftpd = ["pyftpdlib"]
 ftps = ["pyftpdlib", "pyopenssl"]
 tftpd = ["partftpy>=0.4.0"]

scripts/docker/Dockerfile.ac

@@ -9,7 +9,8 @@ ENV     XDG_CONFIG_HOME=/cfg
 
 RUN     apk --no-cache add !pyc \
             tzdata wget mimalloc2 mimalloc2-insecure \
-            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-openssl py3-pillow \
+            py3-jinja2 py3-argon2-cffi py3-pyzmq \
+            py3-openssl py3-paramiko py3-pillow \
             ffmpeg
 
 COPY    i/dist/copyparty-sfx.py innvikler.sh ./

scripts/docker/Dockerfile.dj

@@ -12,7 +12,8 @@ COPY    i/bin/mtag/audio-bpm.py /mtag/
 COPY    i/bin/mtag/audio-key.py /mtag/
 RUN     apk add -U !pyc \
             tzdata wget mimalloc2 mimalloc2-insecure \
-            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-openssl py3-pillow \
+            py3-jinja2 py3-argon2-cffi py3-pyzmq \
+            py3-openssl py3-paramiko py3-pillow \
             py3-pip py3-cffi \
             ffmpeg \
             py3-magic \

scripts/docker/Dockerfile.iv

@@ -9,7 +9,8 @@ ENV     XDG_CONFIG_HOME=/cfg
 
 RUN     apk add -U !pyc \
             tzdata wget mimalloc2 mimalloc2-insecure \
-            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-openssl py3-pillow \
+            py3-jinja2 py3-argon2-cffi py3-pyzmq \
+            py3-openssl py3-paramiko py3-pillow \
             py3-pip py3-cffi \
             ffmpeg \
             py3-magic \

scripts/make-sfx.sh

@@ -48,8 +48,12 @@ help() { exec cat <<'EOF'
 #
 # `no-tfp` saves ~10k by removing the tftp server, disabling --tftp
 #
+# `no-sfp` saves ~?k by removing the sftp server, disabling --sftp
+#
 # `no-zm` saves ~7k by removing the zeroconf mDNS server
 #
+# `no-z` saves ~7k by removing all zeroconf (mDNS, SSDP)
+#
 # `no-smb` saves ~3.5k by removing the smb / cifs server
 #
 # _____________________________________________________________________
@@ -133,10 +137,12 @@ while [ ! -z "$1" ]; do
 		xz)     use_xz=1 ; ;;
 		gz)     use_gz=1 ; ;;
 		gzz)    shift;use_gzz=$1;use_gz=1; ;;
+		no-sfp) no_sfp=1 ; ;;
 		no-ftp) no_ftp=1 ; ;;
 		no-tfp) no_tfp=1 ; ;;
 		no-smb) no_smb=1 ; ;;
 		no-zm)  no_zm=1  ; ;;
+		no-z)   no_zm=1;no_z=1; ;;
 		no-pf)  no_pf=1  ; ;;
 		no-fnt) no_fnt=1 ; ;;
 		no-hl)  no_hl=1  ; ;;
@@ -451,6 +457,14 @@ unhelp() {
 		{sub(/help=.*/,"help=argparse.SUPPRESS)")}1' copyparty/__main__.py
 }
 
+unhelpg() {
+    iawk '/^def/{m=0}
+        /^def add_'$1'/{m=1}
+        m>1{sub(/, help=".*"\)$/, ", help=argparse.SUPPRESS)")}
+        m==1&&/, help="/{m++;sub(/, help=".*"\)$/, ", help=\"not available in this build\")")}
+        1' copyparty/__main__.py
+}
+
 [ $no_ftp ] && {
 	unhelp ftp
 	rm -rf copyparty/ftpd.py ftp
@@ -461,6 +475,11 @@ unhelp() {
 	rm -rf copyparty/tftpd.py partftpy
 }
 
+[ $no_sfp ] && {
+	unhelp sftp
+	rm -rf copyparty/sftpd.py
+}
+
 [ $no_smb ] && {
 	unhelp smb
 	rm -f copyparty/smbd.py
@@ -468,8 +487,14 @@ unhelp() {
 }
 
 [ $no_zm ] &&
+    iawk '$1=="],"{s=0}/"mDNS debugging"/{s=1;sub(/".*/,"\"not available in this build\",\"\"");print};!s' copyparty/__main__.py &&
+    unhelpg zc_mdns &&
 	rm -rf copyparty/mdns.py copyparty/stolen/dnslib
 
+[ $no_z ] &&
+    unhelpg '(zeroconf|zc_ssdp)' &&
+	rm -rf copyparty/ssdp.py copyparty/multicast.py
+
 [ $no_pf ] &&
 	rm -rf copyparty/web/a/partyfuse.py copyparty/web/deps/fuse.py
 

scripts/sfx.ls

@@ -30,6 +30,7 @@ copyparty/res,
 copyparty/res/__init__.py,
 copyparty/res/COPYING.txt,
 copyparty/res/insecure.pem,
+copyparty/sftpd.py,
 copyparty/smbd.py,
 copyparty/ssdp.py,
 copyparty/star.py,

setup.py

@@ -84,7 +84,7 @@ args = {
     "version": about["__version__"],
     "description": (
         "Portable file server with accelerated resumable uploads, "
-        + "deduplication, WebDAV, FTP, TFTP, zeroconf, media indexer, "
+        + "deduplication, WebDAV, SFTP, FTP, TFTP, zeroconf, media indexer, "
         + "video thumbnails, audio transcoding, and write-only folders"
     ),
     "long_description": long_description,
@@ -137,10 +137,11 @@ args = {
     ],
     "install_requires": ["jinja2"],
     "extras_require": {
-        "all": ["argon2-cffi", "partftpy>=0.4.0", "Pillow", "pyftpdlib", "pyopenssl", "pyzmq"],
+        "all": ["argon2-cffi", "paramiko", "partftpy>=0.4.0", "Pillow", "pyftpdlib", "pyopenssl", "pyzmq"],
         "thumbnails": ["Pillow"],
         "thumbnails2": ["pyvips"],
         "audiotags": ["mutagen"],
+        "sftp": ["paramiko"],
         "ftpd": ["pyftpdlib"],
         "ftps": ["pyftpdlib", "pyopenssl"],
         "tftpd": ["partftpy>=0.4.0"],