generic header auth (closes #504);

extends idp-auth to also accept a collection of headers (and
expected values of those headers) and map those to certain users

useful for Tailscale-User-Login and similar

README.md

@@ -91,6 +91,7 @@ made in Norway 🇳🇴
     * [handlers](#handlers) - redefine behavior with plugins ([examples](./bin/handlers/))
     * [ip auth](#ip-auth) - autologin based on IP range (CIDR)
     * [identity providers](#identity-providers) - replace copyparty passwords with oauth and such
+        * [generic header auth](#generic-header-auth) - other ways to auth by header
     * [user-changeable passwords](#user-changeable-passwords) - if permitted, users can change their own passwords
     * [using the cloud as storage](#using-the-cloud-as-storage) - connecting to an aws s3 bucket and similar
     * [hiding from google](#hiding-from-google) - tell search engines you don't wanna be indexed
@@ -1915,6 +1916,20 @@ a more complete example of the copyparty configuration options [look like this](
 but if you just want to let users change their own passwords, then you probably want [user-changeable passwords](#user-changeable-passwords) instead
 
 
+### generic header auth
+
+other ways to auth by header
+
+if you have a middleware which adds a header with a user identifier, for example tailscale's `Tailscale-User-Login: alice.m@forest.net` then you can automatically auth as `alice` by defining that mapping with `--idp-hm-usr '^Tailscale-User-Login^alice.m@forest.net^alice'` or the following config file:
+
+```yaml
+[global]
+  idp-hm-usr: ^Tailscale-User-Login^alice.m@forest.net^alice
+```
+
+repeat the whole `idp-hm-usr` option to add more mappings
+
+
 ## user-changeable passwords
 
 if permitted, users can change their own passwords  in the control-panel

copyparty/__main__.py

@@ -614,6 +614,36 @@ def get_sects():
 
             consider the config file for more flexible account/volume management,
             including dynamic reload at runtime (and being more readable w)
+
+            see \033[32m--help-auth\033[0m for ways to provide the password in requests;
+            see \033[32m--help-idp\033[0m for replacing it with SSO and auth-middlewares
+            """
+            ),
+        ],
+        [
+            "auth",
+            "how to login from a client",
+            dedent(
+                """
+            different ways to provide the password so you become authenticated:
+
+            login with the ui:
+              go to \033[36mhttp://127.0.0.1:3923/?h\033[0m and login there
+
+            send the password in the '\033[36mPW\033[0m' http-header:
+              \033[36mPW: \033[35mhunter2\033[0m
+            or if you have \033[33m--accounts\033[0m enabled,
+              \033[36mPW: \033[35med:hunter2\033[0m
+
+            send the password in the URL itself:
+              \033[36mhttp://127.0.0.1:3923/\033[35m?pw=hunter2\033[0m
+            or if you have \033[33m--accounts\033[0m enabled,
+              \033[36mhttp://127.0.0.1:3923/\033[35m?pw=ed:hunter2\033[0m
+
+            use basic-authentication:
+              \033[36mhttp://\033[35med:hunter2\033[36m@127.0.0.1:3923/\033[0m
+            which should be the same as this header:
+              \033[36mAuthorization: Basic \033[35mZWQ6aHVudGVyMg==\033[0m
             """
             ),
         ],
@@ -765,6 +795,36 @@ def get_sects():
             the upload speed can easily drop to 10% for small files)"""
             ),
         ],
+        [
+            "idp",
+            "replacing the login system with fancy middleware",
+            dedent(
+                """
+            if you already have a centralized service which handles
+            user-authentication for other services already, you can
+            integrate copyparty with that for automatic login
+
+            if the middleware is providing the username in an http-header
+            named '\033[35mtheUsername\033[0m' then do this: \033[36m--idp-h-usr theUsername\033[0m
+
+            if the middleware is providing a list of groups in the header
+            named '\033[35mtheGroups\033[0m' then do this: \033[36m--idp-h-grp theGroup\033[0m
+
+            if the list of groups is separated by '\033[35m%\033[0m' then \033[36m--idp-gsep %\033[0m
+
+            if the middleware is providing a header named '\033[35mAccount\033[0m'
+            and the value is '\033[35malice@forest.net\033[0m' but the username is
+            actually '\033[35mmarisa\033[0m' then do this for each user:
+            \033[36m--idp-hm-usr ^Account^alice@forest.net^marisa\033[0m
+            (the separator '\033[35m^\033[0m' can be any character)
+
+            make ABSOLUTELY SURE that the header can only be set by your
+            middleware and not by clients! and, as an extra precaution,
+            send a header named '\033[36mfinalmasterspark\033[0m' (a secret keyword)
+            and then \033[36m--idp-h-key finalmasterspark\033[0m to require that
+            """
+            ),
+        ],
         [
             "urlform",
             "how to handle url-form POSTs",
@@ -1153,7 +1213,8 @@ def add_auth(ap):
     idp_db = os.path.join(E.cfg, "idp.db")
     ses_db = os.path.join(E.cfg, "sessions.db")
     ap2 = ap.add_argument_group("IdP / identity provider / user authentication options")
-    ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks if the request-header \033[33mHN\033[0m contains a username to associate the request with (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
+    ap2.add_argument("--idp-h-usr", metavar="HN", type=u, action="append", help="\033[34mREPEATABLE:\033[0m bypass the copyparty authentication checks if the request-header \033[33mHN\033[0m contains a username to associate the request with (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
+    ap2.add_argument("--idp-hm-usr", metavar="TXT", type=u, action="append", help="\033[34mREPEATABLE:\033[0m bypass the copyparty authentication checks if the request-header \033[33mHN\033[0m is provided, and its value exists in a mapping defined by this option; see --help-idp")
     ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
     ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
     ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
@@ -1168,6 +1229,7 @@ def add_auth(ap):
     ap2.add_argument("--no-ses", action="store_true", help="disable sessions; use plaintext passwords in cookies")
     ap2.add_argument("--grp-all", metavar="NAME", type=u, default="acct", help="the name of the auto-generated group which contains every username which is known")
     ap2.add_argument("--ipu", metavar="CIDR=USR", type=u, action="append", help="\033[34mREPEATABLE:\033[0m users with IP matching \033[33mCIDR\033[0m are auto-authenticated as username \033[33mUSR\033[0m; example: [\033[32m172.16.24.0/24=dave]")
+    ap2.add_argument("--have-idp-hdrs", type=u, default="", help=argparse.SUPPRESS)
 
 
 def add_chpw(ap):

copyparty/authsrv.py

@@ -1689,6 +1689,8 @@ class AuthSrv(object):
                     self.log("\n{0}\n{1}{0}".format(t, "\n".join(slns)))
                     raise
 
+        self.args.have_idp_hdrs = bool(self.args.idp_h_usr or self.args.idp_hm_usr)
+
         self.setup_pwhash(acct)
         defpw = acct.copy()
         self.setup_chpw(acct)
@@ -1701,7 +1703,7 @@ class AuthSrv(object):
 
             mount = cased
 
-        if not mount and not self.args.idp_h_usr:
+        if not mount and not self.args.have_idp_hdrs:
             # -h says our defaults are CWD at root and read/write for everyone
             axs = AXS(["*"], ["*"], None, None)
             ehint = ""
@@ -1874,7 +1876,7 @@ class AuthSrv(object):
 
         if missing_users:
             zs = ", ".join(k for k in sorted(missing_users))
-            if self.args.idp_h_usr:
+            if self.args.have_idp_hdrs:
                 t = "the following users are unknown, and assumed to come from IdP: "
                 self.log(t + zs, c=6)
             else:
@@ -2551,7 +2553,7 @@ class AuthSrv(object):
             if not self.args.no_voldump:
                 self.log(t)
 
-            if have_e2d or self.args.idp_h_usr:
+            if have_e2d or self.args.have_idp_hdrs:
                 t = self.chk_sqlite_threadsafe()
                 if t:
                     self.log("\n\033[{}\033[0m\n".format(t))
@@ -2841,7 +2843,7 @@ class AuthSrv(object):
     def load_idp_db(self, quiet=False) -> None:
         # mutex me
         level = self.args.idp_store
-        if level < 2 or not self.args.idp_h_usr:
+        if level < 2 or not self.args.have_idp_hdrs:
             return
 
         assert sqlite3  # type: ignore  # !rm
@@ -2898,7 +2900,7 @@ class AuthSrv(object):
         n = []
         q = "insert into us values (?,?,?)"
         accs = list(self.acct)
-        if self.args.idp_h_usr and self.args.idp_cookie:
+        if self.args.have_idp_hdrs and self.args.idp_cookie:
             accs.extend(self.idp_accs.keys())
         for uname in accs:
             if uname not in ases:

copyparty/httpcli.py

@@ -624,8 +624,22 @@ class HttpCli(object):
             or "*"
         )
 
-        if self.args.idp_h_usr:
+        if self.args.have_idp_hdrs:
-            idp_usr = self.headers.get(self.args.idp_h_usr) or ""
+            idp_usr = ""
+            if self.args.idp_hm_usr:
+                for hn, hmv in self.args.idp_hm_usr_p.items():
+                    zs = self.headers.get(hn)
+                    if zs:
+                        for zs1, zs2 in hmv.items():
+                            if zs == zs1:
+                                idp_usr = zs2
+                                break
+                    if idp_usr:
+                        break
+            for hn in self.args.idp_h_usr:
+                if idp_usr:
+                    break
+                idp_usr = self.headers.get(hn)
             if idp_usr:
                 idp_grp = (
                     self.headers.get(self.args.idp_h_grp) or ""

copyparty/svchub.py

@@ -243,7 +243,7 @@ class SvcHub(object):
             t = "WARNING: --th-ram-max is very small (%.2f GiB); will not be able to %s"
             self.log("root", t % (args.th_ram_max, zs), 3)
 
-        if args.chpw and args.idp_h_usr:
+        if args.chpw and args.have_idp_hdrs:
             t = "ERROR: user-changeable passwords is incompatible with IdP/identity-providers; you must disable either --chpw or --idp-h-usr"
             self.log("root", t, 1)
             raise Exception(t)
@@ -268,7 +268,7 @@ class SvcHub(object):
             args.no_ses = True
             args.shr = ""
 
-        if args.idp_store and args.idp_h_usr:
+        if args.idp_store and args.have_idp_hdrs:
             self.setup_db("idp")
 
         if not self.args.no_ses:
@@ -1011,10 +1011,23 @@ class SvcHub(object):
             al.sus_urls = None
 
         al.xff_hdr = al.xff_hdr.lower()
-        al.idp_h_usr = al.idp_h_usr.lower()
+        al.idp_h_usr = [x.lower() for x in al.idp_h_usr or []]
         al.idp_h_grp = al.idp_h_grp.lower()
         al.idp_h_key = al.idp_h_key.lower()
 
+        al.idp_hm_usr_p = {}
+        for zs0 in al.idp_hm_usr or []:
+            try:
+                sep = zs0[:1]
+                hn, zs1, zs2 = zs0[1:].split(sep)
+                hn = hn.lower()
+                if hn in al.idp_hm_usr_p:
+                    al.idp_hm_usr_p[hn][zs1] = zs2
+                else:
+                    al.idp_hm_usr_p[hn] = {zs1: zs2}
+            except:
+                raise Exception("invalid --idp-hm-usr [%s]" % (zs0,))
+
         al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa, True)
         al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa, True)
 

copyparty/up2k.py

@@ -903,7 +903,7 @@ class Up2k(object):
         self.iacct = self.asrv.iacct
         self.grps = self.asrv.grps
 
-        have_e2d = self.args.idp_h_usr or self.args.chpw or self.args.shr
+        have_e2d = self.args.have_idp_hdrs or self.args.chpw or self.args.shr
         vols = list(all_vols.values())
         t0 = time.time()
 

copyparty/web/svcs.html

@@ -42,7 +42,7 @@
 
 
 
-        {% if args.idp_h_usr %}
+        {% if args.have_idp_hdrs %}
         <p style="line-height:2em"><b>WARNING:</b> this server is using IdP-based authentication, so this stuff may not work as advertised. Depending on server config, these commands can probably only be used to access areas which don't require authentication, unless you auth using any non-IdP accounts defined in the copyparty config. Please see <a href="https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients">the IdP docs</a></p>
         {% endif %}
 

tests/test_idp.py

@@ -63,7 +63,7 @@ class TestVFS(unittest.TestCase):
         cfgdir = os.path.join(here, "res", "idp")
 
         # globals are applied by main so need to cheat a little
-        xcfg = {"idp_h_usr": "x-idp-user", "idp_h_grp": "x-idp-group"}
+        xcfg = {"idp_h_usr": ["x-idp-user"], "idp_h_grp": "x-idp-group"}
 
         return here, cfgdir, xcfg
 

tests/util.py

@@ -164,13 +164,13 @@ class Cfg(Namespace):
         ex = "ctl_re db_act forget_ip idp_cookie idp_store k304 loris no304 nosubtle qr_pin re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo u2ow zipmaxn zipmaxs"
         ka.update(**{k: 0 for k in ex.split()})
 
-        ex = "ah_alg bname chmod_f chpw_db doctitle df exit favico idp_h_usr ipa html_head lg_sba lg_sbf log_fk md_sba md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i shr tcolor textfiles txt_eol unlist vname xff_src zipmaxt R RS SR"
+        ex = "ah_alg bname chmod_f chpw_db doctitle df exit favico ipa html_head lg_sba lg_sbf log_fk md_sba md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i shr tcolor textfiles txt_eol unlist vname xff_src zipmaxt R RS SR"
         ka.update(**{k: "" for k in ex.split()})
 
         ex = "ban_403 ban_404 ban_422 ban_pw ban_pwc ban_url spinner"
         ka.update(**{k: "no" for k in ex.split()})
 
-        ex = "ext_th grp on403 on404 xac xad xar xau xban xbc xbd xbr xbu xiu xm"
+        ex = "ext_th grp idp_h_usr idp_hm_usr on403 on404 xac xad xar xau xban xbc xbd xbr xbu xiu xm"
         ka.update(**{k: [] for k in ex.split()})
 
         ex = "exp_lg exp_md"