mirrors/copyparty
1
0
Fork 0
mirror of https://github.com/9001/copyparty.git synced 2025-08-17 09:02:15 -06:00
Code Issues Actions Packages Projects Releases Wiki Activity
3854 commits 4 branches 342 tags 55 MiB
Commit graph

220 commits

Author SHA1 Message Date
ed 1b52ef1f8a Merge branch 'hovudstraum' into idp 2024-02-23 22:25:48 +00:00
ed eeecc50757 v1.10.1 2024-02-18 15:54:38 +00:00
ed 33f41f3e61 add hi-res thumbs (togglebtn/servercfg) 2024-02-18 13:04:22 +00:00
ed 6f8a588c4d up2k: fix a mostly-harmless race 
as each chunk is written to the file, httpcli calls
up2k.confirm_chunk to register the chunk as completed, and the reply
indicates whether that was the final outstanding chunk, in which case
httpcli closes the file descriptors since there's nothing more to write

the issue is that the final chunk is registered as completed before the
file descriptors are closed, meaning there could be writes that haven't
finished flushing to disk yet

if the client decides to issue another handshake during this window,
up2k sees that all chunks are complete and calls up2k.finish_upload
even as some threads might still be flushing the final writes to disk

so the conditions to hit this bug were as follows (all must be true):
* multiprocessing is disabled
* there is a reverse-proxy
* a client has several idle connections and reuses one of those
* the server's filesystem is EXTREMELY slow, to the point where
   closing a file takes over 30 seconds

the fix is to stop handshakes from being processed while a file is
being closed, which is unfortunately a small bottleneck in that it
prohibits initiating another upload while one is being finalized, but
the required complexity to handle this better is probably not worth it
(a separate mutex for each upload session or something like that)

this issue is mostly harmless, partially because it is super tricky to
hit (only aware of it happening synthetically), and because there is
usually no harmful consequences; the worst-case is if this were to
happen exactly as the server OS decides to crash, which would make the
file appear to be fully uploaded even though it's missing some data
(all extremely unlikely, but not impossible)

there is no performance impact; if anything it should now accept
new tcp connections slightly faster thanks to more granular locking
 2024-02-13 19:24:06 +00:00
ed acbb8267e1 tftp: add directory listing 2024-02-10 23:50:17 +00:00
ed a96d9ac6cb idp: users can be in multiple groups 2024-02-08 20:25:32 +00:00
ed caf7e93f5e IdP (#62): add groups + dynamic vols (non-persistent) 
features which should be good to go:
* user groups
* assigning permissions by group
* dynamically created volumes based on username/groupname
* rebuild vfs when new users/groups appear

but several important features still pending;
* detect dangerous configurations
   * dynamic vol below readable path
* remember volumes created during previous runs
   * helps prevent unintended access
   * correct filesystem-scan on startup
 2024-01-30 19:13:42 +01:00
ed 14bccbe45f backports from IdP branch: 
* allow mounting `/` (the entire filesystem) as a volume
  * not that you should (really, you shouldn't)
* improve `-v` helptext
* change IdP group symbol to @ because % is used for file inclusion
  * not technically necessary but is less confusing in docs
 2024-01-25 21:39:30 +00:00
ed 987caec15d v1.9.28 2023-12-31 18:49:42 +00:00
ed 59688bc8d7 * rename hdr-au-usr to idp-h-usr 
* ensure lowercase idp-h-*, xff-hdr
* more macos support in tooling
 2023-12-24 13:46:12 +01:00
ed 10bc2d9205 unsuccessful attempt at dirkeys (#64) 2023-12-17 22:30:22 +00:00
ed 0c50ea1757 list dotfiles only for specific volumes or users (#66): 
* permission `.` grants dotfile visibility if user has `r` too
* `-ed` will grant dotfiles to all `r` accounts (same as before)
* volflag `dots` likewise

also drops compatibility for pre-0.12.0 `-v` syntax
(`-v .::red` will no longer translate to `-v .::r,ed`)
 2023-12-16 15:38:48 +00:00
ed 842817d9e3 improve handling of malicious clients; 
* start banning malicious clients according to --ban-422
* reply with a blank 500 to stop firefox from retrying like 20 times
* allow Cc's in a few specific URL params (filenames, dirnames)
 2023-12-01 23:08:16 +00:00
ed 8c52b88767 make linters happier 2023-11-30 17:33:07 +00:00
ed ee3333362f v1.9.17 2023-11-11 17:38:43 +00:00
ed 1f75314463 placeholder expansion in readme and logues; closes #56 
also fixes the "scan" volflag which broke in v1.9.14
 2023-10-24 16:37:32 +00:00
ed fc658e5b9e utcfromtimestamp was deprecated and nobody told me, 
not even the deprecationwarning that got silently generated burning
20~30% of all CPU-time without actually displaying it anywhere, nice

python 3.12.0 is now only 5% slower than 3.11.6

also fixes some other, less-performance-fatal deprecations
 2023-10-20 23:41:58 +00:00
ed 0dc3c23b42 add alternative filekey generator; closes #52 2023-10-06 13:41:22 +00:00
ed 71c3ad63b3 fix tests 2023-09-11 01:46:25 +00:00
ed 50e01d6904 add more autoban triggers: 
* --ban-url: URLs which 404 and also match --sus-urls (bot-scan)
* --ban-403: trying to access volumes that dont exist or require auth
* --ban-422: invalid POST messages, fuzzing and such
* --nonsus-urls: regex of 404s which  shouldn't trigger --ban-404

in may situations it makes sense to handle this logic inside copyparty,
since stuff like cloudflare and running copyparty on another physical
box than the nginx frontend is on becomes fairly clunky
 2023-08-26 13:52:24 +00:00
ed fc0405c8f3 add prometheus metrics; closes #49 2023-08-20 17:58:06 +00:00
ed bee26e853b show server hostname in html titles: 
* --doctitle defines most titles, prefixed with "--name: " by default
* the file browser is only prefixed with the --name itself
* --nth ("no-title-hostname") removes it
* also removed by --nih ("no-info-hostname")
 2023-08-14 23:50:13 +02:00
ed d0aa20e17c v1.8.7 2023-07-23 15:43:38 +00:00
ed fcc3aa98fd add path-traversal scanners 2023-07-16 13:09:31 +00:00
ed 22fc4bb938 add event-hook for banning users 2023-07-13 22:29:32 +00:00
ed 551d99b71b add permission "a" to show uploader IPs (#45) 2023-07-12 21:36:55 +00:00
ed b54b7213a7 more thumbnailer configs available as volflags: 
--th-convt = convt
--th-no-crop = nocrop
--th-size = thsize
 2023-07-11 22:15:37 +00:00
ed 5d8cb34885 404/403 can be handled with plugins 2023-07-07 21:33:40 +00:00
ed a0c1239246 v1.8.0 2023-06-26 00:05:12 +00:00
ed 9c28ba417e option to regex-exclude files in browser listings 2023-06-02 21:54:25 +00:00
ed 705b58c741 support the NO_COLOR environment variable 
https://no-color.org/ and more importantly
https://youtu.be/biW5UVGkPMA?t=150
 2023-06-02 20:22:57 +00:00
ed deca082623 v1.7.1 2023-05-07 18:34:39 +00:00
ed d3ccd3f174 v1.6.15 2023-04-26 23:00:55 +00:00
ed 03193de6d0 socket read/write timeout 2023-04-24 20:04:22 +00:00
ed fdd6f3b4a6 tar/zip: use volume name as toplevel fallback 2023-04-23 20:55:34 +00:00
ed ca04a00662 v1.6.9 2023-03-16 21:06:18 +00:00
ed 05e0c2ec9e add xiu (batching hook; runs on idle after uploads) + 
bunch of tweaks/fixes for hooks
 2023-02-26 18:23:32 +00:00
ed 6deaf5c268 add jitter simlation 2023-02-20 21:34:30 +00:00
ed a4b56c74c7 support long filepaths on win7 + misc windows fixes 2023-02-10 18:37:37 +00:00
ed e413007eb0 hide dotfiles from search results by default 2023-01-31 18:13:33 +00:00
ed c2ace91e52 v1.6.0 2023-01-29 02:55:44 +00:00
ed 75cea4f684 misc 2023-01-28 13:35:49 +00:00
ed fbc2424e8f v1.5.2 2022-12-12 22:59:31 +00:00
ed 56b73dcc8a up2k: add option to replace existing file 2022-12-10 19:22:16 +00:00
ed 195eb53995 merge wal on shutdown 2022-12-07 23:09:40 +00:00
ed 8ef4a0aa71 fix testrunner + packaging 2022-12-03 15:07:47 +00:00
ed 7c76d08958 drop one of the slowloris detectors 2022-12-02 17:53:23 +00:00
ed 89d1f52235 cursory slowloris / buggy-webdav-client detector 2022-11-01 22:18:20 +00:00
ed 3312c6f5bd autoclose connection-flooding clients 2022-10-31 22:42:47 +00:00
ed 79303dac6d webdav: default-disable recursive listing 2022-10-30 16:47:20 +00:00
ed 138b932c6a add webdav move/delete 2022-10-22 00:04:51 +02:00
ed 20eeacaac3 add webdav write support + fix http 200/201 2022-10-21 18:47:48 +02:00
ed b213de7e62 update readme + tests 2022-10-08 14:18:52 +02:00
ed 24de360325 v1.4.0 2022-09-23 22:53:51 +02:00
ed ab36c8c9de fix tests 2022-09-18 00:16:40 +02:00
ed d2ae822e15 more socket cleanup fiddling 2022-09-07 23:06:12 +02:00
ed 4aaa111925 v1.3.9 2022-08-04 00:39:37 +02:00
ed 1c3894743a fix filekeys inside symlinked volumes 2022-08-02 20:26:51 +02:00
ed 74a3f97671 cleanup + bump deps 2022-07-27 00:15:49 +02:00
ed 062730c70c cleanup 2022-07-06 11:12:36 +02:00
ed 438384425a add types, isort, errorhandling 2022-06-16 01:07:15 +02:00
ed 4c4b3790c7 fix read-spin on d/c during json post + errorhandling 2022-06-07 19:02:52 +02:00
ed 214a367f48 be loud about segfaults and such 2022-05-12 20:26:48 +02:00
ed 125d0efbd8 good stuff 2022-04-29 02:06:56 +02:00
ed ec4daacf9e v1.2.2 2022-03-20 06:15:57 +01:00
ed ded0567cbf v1.1.12 2022-01-18 22:28:33 +01:00
ed abc404a5b7 v1.1.6 2021-12-07 01:17:56 +01:00
ed b8945ae233 fix tests and readme 2021-12-04 18:52:14 +01:00
ed 3375377371 update tests 2021-11-06 23:27:21 +01:00
ed f7a4ea5793 add --js-browser 2021-10-24 00:26:47 +02:00
ed 2f021a0c2b skip indexing files by regex 2021-10-12 01:40:19 +02:00
ed 0249fa6e75 fix tests 2021-10-03 19:59:47 +02:00
ed 9f52c169d0 more python3 shebangs 2021-09-16 00:28:38 +02:00
ed 5849c446ed new access level g 2021-09-15 01:01:20 +02:00
ed f273253a2b ( ´ w `) 2021-09-08 00:16:08 +02:00
ed 8d755d41e0 per-volume rescan interval 2021-08-09 01:31:20 +02:00
ed 3993605324 add -mth (deafult-hidden columns) 2021-08-02 00:47:07 +02:00
ed c164fc58a2 add unpost 2021-07-29 23:53:08 +02:00
ed e65f127571 list server ips on windows 2021-07-28 01:18:38 +02:00
ed 72574da834 hide fileman buttons when argv-disabled 2021-07-26 23:35:55 +02:00
ed d5a79455d1 cleanup 2021-07-26 23:31:45 +02:00
ed a4e1a3738a more deletion progress 2021-07-23 23:42:07 +02:00
ed 4339dbeb8d mv/rm handlers 2021-07-23 01:14:49 +02:00
ed 5b0605774c add move/delete permission flags 2021-07-22 23:48:29 +02:00
ed 99820d854c oh that wasnt enough ok then 2021-07-17 16:45:25 +02:00
ed 62df0a0eb2 thx osx 2021-07-17 16:43:22 +02:00
ed 5c7debd900 improve signal handling + emit sd-notify on start 2021-07-17 04:15:07 +02:00
ed bac301ed66 get rid of iffy default-args 2021-07-12 00:15:13 +02:00
ed 397396ea4a apply -nw to PUT uploads too 2021-07-06 00:49:39 +02:00
ed d6b5351207 add cachebuster because chrome ignores no-cache 2021-07-01 20:10:02 +02:00
ed 05345ddf8b add per-connection request counting 2021-06-30 01:00:00 +02:00
ed 241a143366 add --rproxy for explicit proxy level 2021-06-13 22:22:31 +02:00
ed fa0a7f50bb add image gallery 2021-06-12 20:25:08 +02:00
ed 60ac68d000 single authsrv instance per process 2021-06-11 23:01:13 +02:00
ed 5e03b3ca38 use parent db/thumbs in jump-volumes 2021-06-10 20:43:19 +02:00
ed 1078d933b4 adding --no-hash 2021-06-10 18:08:30 +02:00
ed d6bf300d80 option to store state out-of-volume (mostly untested) 2021-06-10 01:27:04 +02:00
ed 4dd5d4e1b7 when rootless, blank instead of block rootdir 2021-06-08 18:35:55 +02:00
ed acd8149479 dont track workloads unless multiprocessing 2021-06-08 18:01:59 +02:00
ed 89e48cff24 detect recursive symlinks 2021-06-07 20:09:18 +02:00
ed 7d20eb202a optimize 2021-06-04 19:35:08 +02:00
ed 273ca0c8da run tests on commit 2021-06-01 05:49:41 +02:00
ed d89329757e fix permission check in tar/zip generator (gdi) 2021-06-01 03:55:31 +02:00
ed 27a03510c5 quick upload test too 2021-04-24 03:35:58 +02:00
ed ed7727f7cb fix write-only volumes + add regression test 2021-04-24 02:48:41 +02:00
ed fd490af434 explain the jank 2021-03-29 06:11:33 +02:00
ed 0850b8ae2b v0.9.5 2021-03-07 19:25:24 +01:00
ed 482dd7a938 v0.9.3 2021-03-05 00:00:22 +01:00
ed 35324ceb7c tests: support windows 2020-12-04 23:26:46 +01:00
ed db65d05cb5 fix unittest for recent macos versions 2020-11-27 03:24:55 +01:00
ed eec3efd683 show vfs nodes in browser 2020-04-19 00:53:45 +00:00
ed 63e089a5f0 support mojibake and py3.2 2019-06-12 16:39:43 +00:00
ed 36fc87aa6f file access 2019-06-10 01:23:13 +00:00
ed 2414766678 browser getting close 2019-06-07 18:42:15 +00:00
ed fe0330f6f7 delinting 2019-06-07 08:54:41 +00:00
ed a0ccd2b68f safer vfs defaults 2019-06-06 08:47:01 +02:00
ed 7cb3887996 support osx in tests 2019-06-06 08:44:52 +02:00
ed b3de0712d3 config file parser 2019-05-31 18:03:44 +00:00
ed df276b6a84 vfs ls 2019-05-30 13:17:45 +00:00
ed 250d0bdf57 vfs construction ok 2019-05-29 23:46:17 +00:00