Commit graph

220 commits

Author SHA1 Message Date
ed b377791be7 support cidr notation for --xff-src, --ipa, --*-ipa
the old `10.88.` syntax is still supported,
translating to `10.88.0.0/16`

also fix `--tftp-ipa` when optimizations are enabled
2024-03-14 19:07:35 +01:00
ed d71f844b43 IdP: add safeguard --idp-h-key and also require --xff-src 2024-03-12 22:57:47 +00:00
ed e8b7f65f82 IdP: parallel user init + rename idp-h-sep to idp-gsep
`--idp-h-sep` is still supported and will map to its new name
2024-03-12 21:21:53 +00:00
ed f193f398c1 Merge branch 'hovudstraum' into idp 2024-03-12 17:31:27 +00:00
ed d744f3ff8f improve smoketests, warnings and error-messages:
* docker: warn if there are config-files in ~/.config/copyparty
   because somebody copied their config into
   /cfg/copyparty instead of /cfg as intended

* docker: warn if there are no config-files in an included directory

* make misconfigured reverse-proxies more obvious
  * explain cors rejections in server log
  * indicate cors rejection in error toast
2024-03-07 19:47:38 +00:00
ed 096de50889 fix race in config reloader
nothing dangerous, just confusing log messages if an
admin hammers the reload button 100+ times per second,
or another linux process rapidly sends SIGUSR1
2024-02-28 20:08:20 +00:00
ed 1b52ef1f8a Merge branch 'hovudstraum' into idp 2024-02-23 22:25:48 +00:00
ed acbb8267e1 tftp: add directory listing 2024-02-10 23:50:17 +00:00
ed d636316a19 add tftp server 2024-02-10 18:37:21 +00:00
ed a96d9ac6cb idp: users can be in multiple groups 2024-02-08 20:25:32 +00:00
ed caf7e93f5e IdP (#62): add groups + dynamic vols (non-persistent)
features which should be good to go:
* user groups
* assigning permissions by group
* dynamically created volumes based on username/groupname
* rebuild vfs when new users/groups appear

but several important features still pending;
* detect dangerous configurations
   * dynamic vol below readable path
* remember volumes created during previous runs
   * helps prevent unintended access
   * correct filesystem-scan on startup
2024-01-30 19:13:42 +01:00
ed 3313503ea5 retry deleting busy files on windows:
some clients (clonezilla-webdav) rapidly create and delete files;
this fails if copyparty is still hashing the file (usually the case)

and the same thing can probably happen due to antivirus etc

add global-option --rm-retry (volflag rm_retry) specifying
for how long (and how quickly) to keep retrying the deletion

default: retry for 5sec on windows, 0sec (disabled) on everything else
because this is only a problem on windows
2024-01-17 20:27:53 +00:00
ed 86419b8f47 suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
ed 987caec15d v1.9.28 2023-12-31 18:49:42 +00:00
ed bed133d3dd pad log source when logging to file too 2023-12-31 17:21:02 +00:00
ed 59688bc8d7 * rename hdr-au-usr to idp-h-usr
* ensure lowercase idp-h-*, xff-hdr
* more macos support in tooling
2023-12-24 13:46:12 +01:00
ed c057c5e8e8 extend --th-covers with dotfiles; closes #67 2023-12-14 10:53:15 +00:00
ed cba2e10d29 cleanup 2023-12-14 10:47:52 +00:00
ed 842817d9e3 improve handling of malicious clients;
* start banning malicious clients according to --ban-422
* reply with a blank 500 to stop firefox from retrying like 20 times
* allow Cc's in a few specific URL params (filenames, dirnames)
2023-12-01 23:08:16 +00:00
ed 626b5770a5 add --ftp-ipa 2023-11-30 23:36:46 +00:00
ed 00812cb1da new option --ipa; client IP allowlist:
connections from outside the specified list of IP prefixes are rejected
(docker-friendly alternative to -i 127.0.0.1)

also mkdir any missing folders when logging to file
2023-11-30 20:45:43 +00:00
ed c9fd26255b support environment variables mostly everywhere,
useful for docker/systemd stuff

also makes logfiles flush to disk per line by default;
can be disabled for a small performance gain with --no-logflush
2023-11-30 10:22:52 +00:00
ed ee3333362f v1.9.17 2023-11-11 17:38:43 +00:00
ed 1f75314463 placeholder expansion in readme and logues; closes #56
also fixes the "scan" volflag which broke in v1.9.14
2023-10-24 16:37:32 +00:00
ed 2529aa151d tersen volume listing on startup 2023-10-21 12:11:49 +00:00
ed fc658e5b9e utcfromtimestamp was deprecated and nobody told me,
not even the deprecationwarning that got silently generated burning
20~30% of all CPU-time without actually displaying it anywhere, nice

python 3.12.0 is now only 5% slower than 3.11.6

also fixes some other, less-performance-fatal deprecations
2023-10-20 23:41:58 +00:00
ed 801da8079b only 404-ban accounts with permission [gGh]:
never bonk anyone with read-access (able to see directory-listing)
or write-only (not able to retrieve any files at all) due to
either --ban-404 or --ban-url

fixes accidental ban when webdav-uploading files which
match any of the --ban-url patterns (#55)

also default-enables --ban-404 since it is now generally safe
(even when up2k is in turbo mode), plus make turbo smart enough to
disengage when necessary
2023-10-18 22:14:09 +00:00
ed 4b5a0787ab option to show upload timestamps in directory listing;
enable with -mte +.ip_at
or volflag mte=+.ip_at

worst-case performance impact: 18%
2023-10-17 17:51:27 +00:00
ed 0815dce4c1 ensure indexing runs with --ign-ebind-all 2023-09-22 23:20:57 +00:00
ed 163e3fce46 improve reverse-proxy support when containerized:
the x-forwarded-for header would get rejected since the reverse-proxy
is not asking from 127.0.0.1 or ::1, so make this allowlist configurable
2023-09-22 22:39:20 +00:00
ed aa96a1acdc misc optimizations / cleanup:
* slightly faster startup / shutdown
* forgot a jinja2 golf
* waste 4KiB changing prismjs back to gz since brotli is https-gated ;_;
* broke support for firefox<52 (non-var functions must be toplevel
   or immediately within another function), now even firefox 10 /
   centos 6 is somewhat supported again
2023-09-17 13:02:18 +00:00
ed 50e01d6904 add more autoban triggers:
* --ban-url: URLs which 404 and also match --sus-urls (bot-scan)
* --ban-403: trying to access volumes that dont exist or require auth
* --ban-422: invalid POST messages, fuzzing and such
* --nonsus-urls: regex of 404s which  shouldn't trigger --ban-404

in may situations it makes sense to handle this logic inside copyparty,
since stuff like cloudflare and running copyparty on another physical
box than the nginx frontend is on becomes fairly clunky
2023-08-26 13:52:24 +00:00
ed 9b46291a20 add option to force-disable turbo,
making it safer to enable --ban-404
(u2c can still get banned inadvertently)
2023-08-26 13:19:38 +00:00
ed 4f80e44ff7 option to exactly specify browser title prefix 2023-08-15 03:17:01 +02:00
ed 406e413594 hint at additional context in exceptions 2023-08-15 01:42:13 +02:00
ed bee26e853b show server hostname in html titles:
* --doctitle defines most titles, prefixed with "--name: " by default
* the file browser is only prefixed with the --name itself
* --nth ("no-title-hostname") removes it
* also removed by --nih ("no-info-hostname")
2023-08-14 23:50:13 +02:00
ed 04a1f7040e adjustable timestamp resolution in log messages 2023-08-14 17:22:22 +02:00
ed 10caafa34c v1.7.6 2023-06-11 08:14:45 +00:00
ed 705b58c741 support the NO_COLOR environment variable
https://no-color.org/ and more importantly
https://youtu.be/biW5UVGkPMA?t=150
2023-06-02 20:22:57 +00:00
ed cb6de0387d a bit faster 2023-04-26 19:56:27 +00:00
ed 55c74ad164 30% faster folder listings (wtf...) 2023-04-26 18:55:53 +00:00
ed 8f5f8a3cda expand userhomes everywhere:
* -c
* -lo
* --hist
* hist volflag
* --ssl-log
2023-04-14 18:55:19 +02:00
ed 2be2e9a0d8 index folder thumbs in db 2023-03-11 11:43:29 +00:00
ed c7c6e48b1a didn't compress numbered logfiles 2023-03-09 21:59:59 +00:00
ed c98fff1647 fix chunkpost-handshake race (affects --no-dedup only);
a handshake arriving in the middle of the final chunk could cause
dupes to become empty -- worst case leading to loss of data
2023-03-05 19:45:50 +00:00
ed 6deaf5c268 add jitter simlation 2023-02-20 21:34:30 +00:00
ed 8adfcf5950 win10-based copyparty64.exe 2023-02-14 21:50:14 +00:00
ed 44909cc7b8 print ffmpeg download url on windows 2023-02-11 17:22:24 +00:00
ed 8f61e1568c transcode chiptunes to opus;
* new audio/MPT formats: apac bonk dfpwm ilbc it itgz itr itz mo3 mod mptm mt2 okt s3gz s3m s3r s3z xm xmgz xmr xmz xpk
* new image/PIL formats: blp dcx emf eps fits flc fli fpx im j2k j2p psd spi wmf
2023-02-11 11:17:37 +00:00
ed a4b56c74c7 support long filepaths on win7 + misc windows fixes 2023-02-10 18:37:37 +00:00
ed 99cc434779 add config explainer + generator (#20) 2023-02-05 22:09:17 +00:00
ed d4c5fca15b sandbox readme.md / prologue / epilogue 2023-01-28 21:24:40 +00:00
ed 741d781c18 add cors controls + improve preflight + pw header 2023-01-28 00:59:04 +00:00
ed 9b7b9262aa promote dedup control to volflags 2023-01-25 21:46:15 +00:00
ed 577d23f460 zeroconf: detect network change and reannounce 2023-01-18 21:27:27 +00:00
ed 18942ed066 location-based rproxy fixes 2023-01-16 20:09:45 +00:00
ed b4c503657b ignore loss of stdout 2023-01-14 07:35:44 +00:00
ed 7fd1d6a4e8 rename --webroot to --rp-loc and fix related bugs 2022-12-11 21:09:50 +00:00
ed 19cd96e392 cleanup + optimizations 2022-12-11 14:16:51 +00:00
ed db194ab519 support location-based rproxy 2022-12-10 23:43:31 +00:00
ed 416ebfdd68 right, windows nic names have whitespace 2022-12-05 17:35:12 +00:00
ed 2fbdc0a85e misc fixes / cleanup 2022-12-02 23:42:46 +00:00
ed 7c76d08958 drop one of the slowloris detectors 2022-12-02 17:53:23 +00:00
ed cfd41fcf41 zeroconf: add network filtering options 2022-11-26 22:37:12 +00:00
ed 5cd9d11329 add ssdp responder 2022-11-22 21:40:12 +00:00
ed d326ba9723 ftp: ban password-bruteforcing IPs 2022-11-20 11:06:07 +00:00
ed 8709d4dba0 macos smb: avoid hang on shutdown 2022-11-17 21:17:54 +00:00
ed 0cedaf4fa9 isort 2022-11-15 22:41:35 +00:00
ed a234aa1f7e cleaner shutdown of smbd, mdns 2022-11-15 21:55:02 +00:00
ed cd2513ec16 logging fixes 2022-11-15 21:28:27 +00:00
ed 97ff0ebd06 xz-compress logs only if -lo ends with .xz 2022-11-15 20:16:41 +00:00
ed b3eb117e87 add mdns zeroconf announcer 2022-11-13 20:05:16 +00:00
ed e442cb677a improve ftp/smb logging 2022-11-06 13:30:16 +00:00
ed 357e7333cc cleanup 2022-11-04 20:27:16 +00:00
ed 89d1f52235 cursory slowloris / buggy-webdav-client detector 2022-11-01 22:18:20 +00:00
ed d4ba644d07 autodefault -nc based on OS limits 2022-10-31 19:37:37 +00:00
ed 79303dac6d webdav: default-disable recursive listing 2022-10-30 16:47:20 +00:00
ed edad3246e0 make pylance happier 2022-10-29 20:40:25 +00:00
ed 835f8a20e6 default-enable webdav 2022-10-23 23:37:32 +02:00
ed f3a501db30 add SMB/CIFS server 2022-10-23 23:08:00 +02:00
ed 4bcd30da6b cleaner daemon instancing 2022-10-23 12:05:44 +02:00
ed 138b932c6a add webdav move/delete 2022-10-22 00:04:51 +02:00
ed d8bddede6a new permission G returns filekey on write-only uploads 2022-10-08 01:17:41 +02:00
ed dbb3edec77 print qr-code on startup 2022-10-07 00:47:26 +02:00
ed abb3224cc5 option to save a copy of corrupted uploads 2022-09-26 22:01:49 +02:00
ed 1c66d06702 cleanup versionchecks 2022-09-25 21:31:47 +02:00
ed 32e71a43b8 reinvent fail2ban 2022-09-21 22:27:20 +02:00
ed 0006f933a2 hmac uploader-ip when avoiding filename collisions 2022-09-11 08:27:45 +02:00
ed e430b2567a add pyoxidizer (windows-only) 2022-09-10 17:33:04 +02:00
ed 5d19f23372 accurate num.cores detection 2022-08-29 19:24:48 +02:00
ed 11df36f3cf add option to exit after scanning volumes 2022-08-12 21:20:13 +02:00
ed c00b80ca29 v1.3.11 2022-08-10 23:35:21 +02:00
ed 51c152ff4a indicate sqlite thread-safety + some cleanup 2022-08-05 01:20:16 +02:00
ed 4aaa111925 v1.3.9 2022-08-04 00:39:37 +02:00
ed e31248f018 include version info on startup and in crash dumps 2022-08-04 00:11:52 +02:00
ed 56a62db591 force-exit by hammering ctrl-c 2022-08-03 20:58:23 +02:00
ed 1c3894743a fix filekeys inside symlinked volumes 2022-08-02 20:26:51 +02:00
ed 3fa377a580 sqlite diag 2022-07-16 20:43:26 +02:00
ed c2a0b1b4c6 autopotato 2022-07-15 02:39:32 +02:00
ed 9ad3af1ef6 misc tweaks 2022-06-19 16:05:48 +02:00