Commit graph

77 commits

Author SHA1 Message Date
ed 1b2d39857b reset x-forwarded-for before next req;
assume the following stack: cpp <- rproxyA <- rproxyB <- WAN

if A also accepts WAN requests, and A muxes both B and WAN
onto a single connection to cpp, then WAN requests may get
tagged with the IP-address of the most recent B request

aside from the confusing logs, this could break
unpost on servers with shared accounts
2025-07-07 08:47:24 +00:00
ed b7f9bf5a28 cidr-based autologin 2024-10-13 21:56:26 +00:00
ed 58cf01c2ad fix linter warnings 2024-09-20 22:24:39 +00:00
Shiz a462a644fb
Python 3.7 package resources support (#98)
add support for reading webdeps and jinja-templates using either
importlib_resources or pkg_resources, which removes the need for
extracting these to a temporary folder on the filesystem

* util: add helper functions to abstract embedded resource access
* http*: serve embedded resources through resource abstraction
* main: check webdeps through resource abstraction
* httpconn: remove unused method `respath(name)`
* use __package__ to find package resources
* util: use importlib_resources backport if available
* pass E.pkg as module object for importlib_resources compatibility
* util: add pkg_resources compatibility to resource abstraction
2024-09-19 09:00:34 +00:00
ed 2927bbb2d6 strip dev-only asserts at build stage 2024-09-14 22:17:35 +00:00
ed 72361c99e1 add import chickenbits 2024-08-01 18:29:25 +00:00
ed 2f7f9de3f5 pipe: optimize (1 GiB/s @ ryzen5-4500U) 2024-04-20 20:13:31 +00:00
ed 2def4cd248 fix linter warnings + a test 2024-03-18 15:25:10 +00:00
ed b377791be7 support cidr notation for --xff-src, --ipa, --*-ipa
the old `10.88.` syntax is still supported,
translating to `10.88.0.0/16`

also fix `--tftp-ipa` when optimizations are enabled
2024-03-14 19:07:35 +01:00
ed 6f8a588c4d up2k: fix a mostly-harmless race
as each chunk is written to the file, httpcli calls
up2k.confirm_chunk to register the chunk as completed, and the reply
indicates whether that was the final outstanding chunk, in which case
httpcli closes the file descriptors since there's nothing more to write

the issue is that the final chunk is registered as completed before the
file descriptors are closed, meaning there could be writes that haven't
finished flushing to disk yet

if the client decides to issue another handshake during this window,
up2k sees that all chunks are complete and calls up2k.finish_upload
even as some threads might still be flushing the final writes to disk

so the conditions to hit this bug were as follows (all must be true):
* multiprocessing is disabled
* there is a reverse-proxy
* a client has several idle connections and reuses one of those
* the server's filesystem is EXTREMELY slow, to the point where
   closing a file takes over 30 seconds

the fix is to stop handshakes from being processed while a file is
being closed, which is unfortunately a small bottleneck in that it
prohibits initiating another upload while one is being finalized, but
the required complexity to handle this better is probably not worth it
(a separate mutex for each upload session or something like that)

this issue is mostly harmless, partially because it is super tricky to
hit (only aware of it happening synthetically), and because there is
usually no harmful consequences; the worst-case is if this were to
happen exactly as the server OS decides to crash, which would make the
file appear to be fully uploaded even though it's missing some data
(all extremely unlikely, but not impossible)

there is no performance impact; if anything it should now accept
new tcp connections slightly faster thanks to more granular locking
2024-02-13 19:24:06 +00:00
ed 86419b8f47 suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
ed cba2e10d29 cleanup 2023-12-14 10:47:52 +00:00
ed a00ff2b086 v1.7.4 2023-06-11 00:07:38 +00:00
ed 2b2d8e4e02 tls / gencert fixes 2023-06-10 23:34:34 +00:00
ed 6acf436573 u2idx pool instead of per-socket;
prevents running out of FDs thanks to thousands of sqlite3 sessions
and neatly sidesteps what could possibly be a race in python's
sqlite3 bindings where it sometimes forgets to close the fd
2023-04-20 20:36:13 +00:00
ed 741d781c18 add cors controls + improve preflight + pw header 2023-01-28 00:59:04 +00:00
ed 999b7ae919 safer to merge wal on startup instead 2022-12-09 19:58:13 +00:00
ed 195eb53995 merge wal on shutdown 2022-12-07 23:09:40 +00:00
ed 3312c6f5bd autoclose connection-flooding clients 2022-10-31 22:42:47 +00:00
ed edad3246e0 make pylance happier 2022-10-29 20:40:25 +00:00
ed 8510804e57 initial webdav support 2022-10-18 19:36:52 +02:00
ed add04478e5 multiprocessing: fix listening-socket config 2022-09-15 22:25:11 +02:00
ed 0006f933a2 hmac uploader-ip when avoiding filename collisions 2022-09-11 08:27:45 +02:00
ed e430b2567a add pyoxidizer (windows-only) 2022-09-10 17:33:04 +02:00
ed 558bfa4e1e siocoutq-based shutdown 2022-08-31 01:16:09 +02:00
ed 4a76663fb2 ensure free disk space 2022-07-17 22:33:08 +02:00
ed 903b9e627a ux snappiness + keepalive on http-1.0 2022-06-16 20:33:09 +02:00
ed 438384425a add types, isort, errorhandling 2022-06-16 01:07:15 +02:00
ed 4c4b3790c7 fix read-spin on d/c during json post + errorhandling 2022-06-07 19:02:52 +02:00
ed 214a367f48 be loud about segfaults and such 2022-05-12 20:26:48 +02:00
ed 0a1d9b4dfd nevermind, not reliable when rproxied 2022-05-01 22:35:34 +02:00
ed b50d090946 add logout on inactivity + related errorhandling 2022-05-01 22:12:25 +02:00
ed b64cabc3c9 thumbnails: add pyvips as alt/supp. to pillow 2022-04-10 14:16:09 +02:00
ed f050647b43 rescan volumes on sigusr1 2021-11-06 18:20:31 +01:00
ed 0d5ca67f32 up2k-srv: add option to reuse file-handles 2021-09-26 03:44:22 +02:00
ed 578a915884 stack/thread monitors in mpw + better thread names 2021-07-12 23:03:52 +02:00
ed 8fcde2a579 move tcp accept into mp-worker 2021-07-09 15:49:36 +02:00
ed 05345ddf8b add per-connection request counting 2021-06-30 01:00:00 +02:00
ed 6249bd4163 add pebkac hints 2021-06-22 17:18:34 +02:00
ed c50f1b64e5 dodge android-chrome bug: canvas aspect ratio 2021-06-18 21:46:15 +02:00
ed 10362aa02e v0.11.18 2021-06-18 00:30:37 +02:00
ed 60ac68d000 single authsrv instance per process 2021-06-11 23:01:13 +02:00
ed d6bf300d80 option to store state out-of-volume (mostly untested) 2021-06-10 01:27:04 +02:00
ed acd8149479 dont track workloads unless multiprocessing 2021-06-08 18:01:59 +02:00
ed 6b065d507d crop thumbs for AESTHETICS 2021-05-28 01:46:27 +02:00
ed 4dff726310 initial thumbnail and icon stuff 2021-05-25 03:37:01 +02:00
ed 78ba16f722 log filtering by url regex 2021-05-16 07:29:34 +02:00
ed 9e850fc3ab zip selection 2021-03-27 15:48:52 +01:00
ed 4ed9528d36 5x faster reply on 1st req on new conns 2021-03-25 19:29:16 +01:00
ed 57d994422d logging cleanup 2021-03-06 17:38:56 +01:00