Commit graph

412 commits

Author SHA1 Message Date
ed 87c60a1ec9 ensure OS signals hit main-thread as intended;
use sigmasks to block SIGINT, SIGTERM, SIGUSR1 from all other threads

also initiate shutdown by calling sighandler directly,
in case this misses anything and that is still unreliable
(discovered by `--exit=idx` being noop once in a blue moon)
2024-05-09 22:28:16 +00:00
ed feed08deb2 doc: export --help to html and link it 2024-05-08 22:01:58 +00:00
ed 560d7b6672 option to add or change mimetype mappings 2024-05-08 21:12:14 +00:00
ed da091aec85 "volume" is too overloaded, make it --au-vol instead 2024-05-05 21:27:07 +00:00
ed e9eb5affcd and option to set default audio/video volume 2024-05-05 19:10:29 +00:00
ed 36f2c446af opengraph stuff:
* template-based title formatting
* picture embeds are no longer ant-sized
* `--og-color` sets accent color; default #333
* `--og-s-title` forces default title, ignoring e2t
* add a music indicator to song titles because discord doesn't
2024-05-03 00:11:40 +00:00
ed 69517e4624 add general-purpose query-string parcelling;
currently only being used to workaround discord discarding
query strings in opengraph tags, but i'm sure there will be
plenty more wonderful usecases for this atrocity
2024-05-02 22:49:27 +00:00
ed ea270ab9f2 add og / opengraph / discord embeds 2024-05-01 23:40:56 +00:00
ed b6cf2d3089 --html-head can take a filepath and/or jinja2 2024-05-01 20:24:18 +00:00
ed 27485a4cb1 add pyz builder 2024-04-24 23:45:01 +00:00
ed f6e693f0f5 reevaluate support for sparse files periodically
if a given filesystem were to disappear (e.g. removable storage)
followed by another filesystem appearing at the same location,
this would not get noticed by up2k in a timely manner

fix this by discarding the mtab cache after `--mtab-age` seconds and
rebuild it from scratch, unless the previous values are definitely
correct (as indicated by identical output from `/bin/mount`)

probably reduces windows performance by an acceptable amount
2024-04-24 21:18:26 +00:00
ed 4bb0e6e75a pipe: windows: make it safe with aggressive flushing 2024-04-20 22:15:08 +00:00
ed f31ac90932 less confusing help-text for --re-dhash 2024-04-20 16:42:56 +00:00
ed c06126cc9d pipe: add volflag to disable 2024-04-19 23:54:23 +00:00
ed bf5850785f add opt-out from storing uploader IPs 2024-04-18 17:16:00 +00:00
ed c8e3ed3aae retry failed renames on windows
theoretical issue which nobody has ran into yet,
probably because nobody uses this on windows
2024-04-12 20:38:30 +00:00
ed 7b94e4edf3 configurable basic-auth preference;
adds options `--bauth-last` to lower the preference for
taking the basic-auth password in case of conflict,
and `--no-bauth` to entirely disable basic-authentication

if a client is providing multiple passwords, for example when
"logged in" with one password (the `cppwd` cookie) and switching
to another account by also sending a PW header/url-param, then
the default evaluation order to determine which password to use is:

url-param `pw`, header `pw`, basic-auth header, cookie (cppwd/cppws)

so if a client supplies a basic-auth header, it will ignore the cookie
and use the basic-auth password instead, which usually makes sense

but this can become a problem if you have other webservers running
on the same domain which also support basic-authentication

--bauth-last is a good choice for cooperating with such services, as
--no-bauth currently breaks support for the android app...
2024-04-11 20:15:49 +00:00
ed 5ab54763c6 remove pyoxidizer (unmaintained)
partially reverts e430b2567a

the remaining stuff might be useful for other cpython alternatives
2024-04-05 17:51:26 +00:00
ed a080759a03 add transcoding to mp3
because CU's car stereo can't do opus...

incidentally adds support for playing any audio format in ie11
2024-03-29 16:36:56 +00:00
ed 20870fda79 Merge branch 'dirkeys' into hovudstraum 2024-03-25 10:34:08 +00:00
ed c6acd3a904 add option --s-rd-sz (socket read size):
counterpart of `--s-wr-sz` which existed already

the default (256 KiB) appears optimal in the most popular scenario
(linux host with storage on local physical disk, usually NVMe)

was previously 32 KiB, so large uploads should now use 17% less CPU

also adds sanchecks for values of `--iobuf`, `--s-rd-sz`, `--s-wr-sz`

also adds file-overwrite feature for multipart posts
2024-03-23 16:35:14 +00:00
ed 2b24c50eb7 add option --iobuf (file r/w buffersize):
the default (256 KiB) appears optimal in the most popular scenario
(linux host with storage on local physical disk, usually NVMe)

was previously a mix of 64 and 512 KiB;
now the same value is enforced everywhere

download-as-tar is now 20% faster with the default value
2024-03-23 16:17:40 +00:00
ed 51d31588e6 parse xff before deciding to reject a connection
this commit partially fixes the following issue:
if a client manages to escape real-ip detection, copyparty will
try to ban the reverse-proxy instead, effectively banning all clients

this can happen if the configuration says to obtain client real-ip
from a cloudflare header, but the server is not configured to reject
connections from non-cloudflare IPs, so a scanner will eventually
hit the server IP with malicious-looking requests and trigger a ban

copyparty will now continue to process requests from banned IPs until
the header has been parsed and the real-ip has been obtained (or not),
causing an increased server load from malicious clients

assuming the `--xff-src` and `--xff-hdr` config is correct,
this issue should no longer be hitting innocent clients

the old behavior of immediately rejecting a banned IP address
can be re-enabled with the new option `--early-ban`
2024-03-17 02:36:03 +00:00
ed 66abf17bae black 2024-03-14 18:37:05 +00:00
ed b377791be7 support cidr notation for --xff-src, --ipa, --*-ipa
the old `10.88.` syntax is still supported,
translating to `10.88.0.0/16`

also fix `--tftp-ipa` when optimizations are enabled
2024-03-14 19:07:35 +01:00
ed d71f844b43 IdP: add safeguard --idp-h-key and also require --xff-src 2024-03-12 22:57:47 +00:00
ed e8b7f65f82 IdP: parallel user init + rename idp-h-sep to idp-gsep
`--idp-h-sep` is still supported and will map to its new name
2024-03-12 21:21:53 +00:00
ed f193f398c1 Merge branch 'hovudstraum' into idp 2024-03-12 17:31:27 +00:00
ed 3f05b6655c add UI to abort an unfinished upload; suggested in #77
to abort an upload, refresh the page and access the unpost tab,
which now includes unfinished uploads (sorted before completed ones)

can be configured through u2abort (global or volflag);
by default it requires both the IP and account to match

https://a.ocv.me/pub/g/nerd-stuff/2024-0310-stoltzekleiven.jpg
2024-03-11 01:32:02 +01:00
ed 1c011ff0bb hide k304 config from controlpanel by default;
as this option is very rarely useful, add global-option `--k304` to
unhide the button and/or set it default-enabled

the toggle will still appear when the feature was previously enabled by
a client, and the feature is still default-enabled for all IE clients
2024-03-09 17:50:24 +00:00
ed d744f3ff8f improve smoketests, warnings and error-messages:
* docker: warn if there are config-files in ~/.config/copyparty
   because somebody copied their config into
   /cfg/copyparty instead of /cfg as intended

* docker: warn if there are no config-files in an included directory

* make misconfigured reverse-proxies more obvious
  * explain cors rejections in server log
  * indicate cors rejection in error toast
2024-03-07 19:47:38 +00:00
ed 1b52ef1f8a Merge branch 'hovudstraum' into idp 2024-02-23 22:25:48 +00:00
ed 33f41f3e61 add hi-res thumbs (togglebtn/servercfg) 2024-02-18 13:04:22 +00:00
ed 0504b010a1 tftp: support ipv6 and utf-8 filenames + ...
* fix winexe
* missing newline after dirlist
* optimizations
2024-02-17 21:31:58 +00:00
ed acbb8267e1 tftp: add directory listing 2024-02-10 23:50:17 +00:00
ed 8796c09f56 add --tftp-pr to specify portrange instead of ephemerals 2024-02-10 21:45:57 +00:00
ed d636316a19 add tftp server 2024-02-10 18:37:21 +00:00
ed a96d9ac6cb idp: users can be in multiple groups 2024-02-08 20:25:32 +00:00
ed 35165f8472 Merge branch 'hovudstraum' into idp 2024-02-03 19:14:49 +00:00
ed c354a38b4c up2k: warn about browser cap on num connections 2024-02-02 23:46:00 +00:00
ed caf7e93f5e IdP (#62): add groups + dynamic vols (non-persistent)
features which should be good to go:
* user groups
* assigning permissions by group
* dynamically created volumes based on username/groupname
* rebuild vfs when new users/groups appear

but several important features still pending;
* detect dangerous configurations
   * dynamic vol below readable path
* remember volumes created during previous runs
   * helps prevent unintended access
   * correct filesystem-scan on startup
2024-01-30 19:13:42 +01:00
ed 14bccbe45f backports from IdP branch:
* allow mounting `/` (the entire filesystem) as a volume
  * not that you should (really, you shouldn't)
* improve `-v` helptext
* change IdP group symbol to @ because % is used for file inclusion
  * not technically necessary but is less confusing in docs
2024-01-25 21:39:30 +00:00
ed b9d0c8536b avoid sendfile bugs on 32bit machines:
https://github.com/python/cpython/issues/114077
2024-01-17 20:56:44 +00:00
ed 3313503ea5 retry deleting busy files on windows:
some clients (clonezilla-webdav) rapidly create and delete files;
this fails if copyparty is still hashing the file (usually the case)

and the same thing can probably happen due to antivirus etc

add global-option --rm-retry (volflag rm_retry) specifying
for how long (and how quickly) to keep retrying the deletion

default: retry for 5sec on windows, 0sec (disabled) on everything else
because this is only a problem on windows
2024-01-17 20:27:53 +00:00
ed 95a599961e add RAM usage tracking to thumbnailer;
prevents server OOM from high RAM usage by FFmpeg when generating
spectrograms and waveforms: https://trac.ffmpeg.org/ticket/10797
2024-01-14 04:15:09 +00:00
ed d0eb014c38 improve applefilters + add missing newline in curl 404
* webdav: extend applesan regex with more stuff to exclude
* on macos, set applesan as default `--no-idx` to avoid indexing them
   (they didn't show up in search since they're dotfiles, but still)
2024-01-12 00:13:35 +01:00
ed 86419b8f47 suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
ed 987caec15d v1.9.28 2023-12-31 18:49:42 +00:00
ed ab40ff5051 add permission "A" (alias of "rwmda."); closes #70 2023-12-31 18:20:24 +00:00
ed 5b26ab0096 add option to specify default num parallel uploads 2023-12-28 01:41:17 +01:00