Commit graph

486 commits

Author SHA1 Message Date
ed b8b5214f44 option to restrict recent-uploads visibility 2025-01-25 11:13:12 +00:00
ed cdd3b67a5c msg-to-log includes xm / ZeroMQ response 2025-01-25 10:59:15 +00:00
ed 28c9de3f6a add opus-weba transcoding (for iOS 18 and newer)
support for "owa", audio-only webm, was introduced in iOS 17.5

owa is a more compliant alternative to opus-caf from iOS 11,
which was technically limited to CBR opus, a limitation which
we ignored since it worked mostly fine for regular opus too

being the new officially-recommended way to do things,
we'll default to owa for iOS 18 and later, even though
iOS still has some bugs affecting our use specifically:

if a weba file is preloaded into a 2nd audio object,
safari will throw a spurious exception as playback is
initiated, even as the file is playing just fine

the `.ld` stuff is an attempt at catching and ignoring this
spurious error without eating any actual network exceptions
2025-01-25 10:15:44 +00:00
ed f3b9bfc114 option to disable caf transcoding + misc cleanup 2025-01-24 22:40:52 +00:00
ed c4cbc32cc5 v1.16.9 2025-01-22 23:26:17 +00:00
ed 6a0aaaf069 md/logue sandbox: custom allow prop
add global-option and volflag to specify the
value of the iframe's allow-property
2025-01-21 22:51:00 +00:00
ed 170cbe98c5 refactor github urls 2025-01-17 05:48:49 +00:00
ed da5ad2ab9f warn on ambiguous comments in config files 2024-12-19 00:25:10 +00:00
ed eaa4b04a22 list recent uploads
also makes the unpost lister 5x faster
2024-12-18 22:17:30 +01:00
ed 64501fd7f1 hybrid IdP (check regular users too); closes #122
previously, when IdP was enabled, the password-based login would be
entirely disabled. This was a semi-conscious decision, based on the
assumption that you would always want to use IdP after enabling it.

it makes more sense to keep password-based login working as usual,
conditionally disengaging it for requests which contains a valid
IdP username header. This makes it possible to define fallback
users, or API-only users, and all similar escape hatches.
2024-12-08 17:18:20 +00:00
ed db3c0b0907 nice 2024-12-07 22:24:13 +00:00
ed 9082c4702f accesslog: exclude thumbnails by default 2024-12-03 22:17:49 +00:00
ed 016708276c add sorting granularity options for media URLs 2024-12-03 20:01:19 +00:00
ed 6c1cf68bca mdns: add workaround for https://github.com/avahi/avahi/issues/379
Avahi's mDNS-reflection feature does not understand NSEC, so
it corrupts mDNS packets by not rewriting compressed labels
2024-12-01 19:01:40 +00:00
ed 395af051bd mdns: option to ignore invalid packets on the lan 2024-12-01 15:38:24 +00:00
ed 697a4fa8a4 exclude search results by regex (#120)
a better alternative to using `--no-idx` for this purpose since
this also excludes recent uploads, not just during fs-indexing,
and it doesn't prevent deduplication

also speeds up searches by a tiny amount due to building the
sanchecks into the exclude-filter while parsing the config,
instead of during each search query
2024-11-26 23:57:01 +00:00
ed 8f7ffcf350 add nsort option/volflag 2024-11-19 18:39:40 +00:00
ed 4d15dd6e17 cbz thumbnails 2024-11-15 21:33:37 +00:00
ed 1f177528c1 fix advanced options for password-hashing
and allow raising scrypt ram usage past OpenSSL's default 32 MiB
2024-11-15 00:42:08 +00:00
ed 2bf9055cae detect free RAM on startup for sane defaults
* if free ram on startup is less than 2 GiB,
   use smaller chunks for parallel file hashing

* if --th-max-ram is lower than 0.25 (256 MiB),
   print a warning that thumbnails will not work

* make thumbnail cleaner immediately do a sweep on startup,
   forgetting any failed conversions so they can be retried
   in case the memory limit was increased since last run
2024-11-10 15:43:19 +00:00
ed 8aba5aed4f list active downloads in controlpanel 2024-11-10 02:12:18 +00:00
ed cacec9c1f3 support copying files/folders; closes #115
behaves according to the target volume's deduplication config;
will create symlinks / hardlinks instead if dedup is enabled
2024-11-07 21:41:53 +00:00
ed dd6dbdd90a http 304: client-option to force-disable cache
an extremely brutish workaround for issues such as #110 where
browsers receive an HTTP 304 and misinterpret as HTTP 200

option `--no304=1` adds the button `no304` to the controlpanel
which can be enabled to force-disable caching in that browser

the button is default-disabled; by specifying `--no304=2`
instead of `--no304=1` the button becomes default-enabled

can also always be enabled by accessing `/?setck=no304=y`
2024-10-26 17:56:54 +00:00
ed 7678a91b0e debug: --ohead (log response headers) 2024-10-25 20:00:19 +00:00
ed 7ffd805a03 add RSS feed output; closes #109 2024-10-18 23:24:12 +00:00
ed a9b4436cdc up2k: improve upload retry/timeout
* `js:` make handshake retries more aggressive
* `u2c:` reduce chunks timeout + ^
* `main:` reduce tcp timeout to 128sec (js is 42s)
* `httpcli:` less confusing log messages
2024-10-18 16:24:31 +00:00
ed fc8298c468 up2k: avoid cloudflare upload size-limit
previously, the biggest file that could be uploaded through
cloudflare was 383 GiB, due to max num chunks being 4096

`--u2sz`, which takes three ints (min-size, target, max-size)
can now be used to enforce a max chunksize; chunks larger
than max-size gets split into smaller subchunks / chunklets

subchunks cannot be stitched/joined, and subchunks of the
same chunk must be uploaded sequentially, one at a time

if a subchunk fails due to bitflips or connection-loss,
then the entire chunk must (and will) be reuploaded
2024-10-16 19:29:08 +00:00
ed ee62836383 bitflip logging 2024-10-13 22:37:35 +00:00
ed b7f9bf5a28 cidr-based autologin 2024-10-13 21:56:26 +00:00
ed 1d68acf8f0 add preadme.md; closes #105 2024-10-11 21:52:44 +00:00
ed 1e7697b551 misc cleanup;
* more typos
* python 3.13 deprecations
2024-10-11 20:46:40 +00:00
ed 3d7facd774 add option to entirely disable dedup
global-option `--no-clone` / volflag `noclone` entirely disables
serverside deduplication; clients will then fully upload dupe files

can be useful when `--safe-dedup=1` is not an option due to other
software tampering with the on-disk files, and your filesystem has
prohibitively slow or expensive reads
2024-10-08 21:27:19 +00:00
ed c398553748 pkgres: fix multiprocessing 2024-10-05 17:32:08 +00:00
ed dc4d0d8e71 smb: upto 2x faster; but still very buggy:
* do not absreal paths unless necessary
* do not determine username if no users configured
* impacket 0.12 fixed the foldersize limit, but now
   you get extremely poor performance in large folders
   so the previous workaround is still default-enabled
2024-09-27 17:09:48 +00:00
ed d866841c19 pkgres:
* pyz: yeet the resource tar which is now pointless thanks to pkgres
* cache impresource stuff because pyz lookups are Extremely slow
* prefer tx_file when possible for slightly better performance
* use hardcoded list of expected resources instead of dynamic
   discovery at runtime; much simpler and probably safer
* fix some forgotten resources (copying.txt, insecure.pem)
* fix loading jinja templates on windows
2024-09-19 22:04:49 +00:00
Shiz a462a644fb
Python 3.7 package resources support (#98)
add support for reading webdeps and jinja-templates using either
importlib_resources or pkg_resources, which removes the need for
extracting these to a temporary folder on the filesystem

* util: add helper functions to abstract embedded resource access
* http*: serve embedded resources through resource abstraction
* main: check webdeps through resource abstraction
* httpconn: remove unused method `respath(name)`
* use __package__ to find package resources
* util: use importlib_resources backport if available
* pass E.pkg as module object for importlib_resources compatibility
* util: add pkg_resources compatibility to resource abstraction
2024-09-19 09:00:34 +00:00
ed 427597b603 show total directory size in listings
sizes are computed during `-e2ds` indexing, and new uploads
are counted, but a rescan is necessary after a move or delete
2024-09-15 23:01:18 +00:00
ed 7d64879ba8 more optimizations,
* 5% less cpu load from clients fetching thumbnails
* and slight improvement to up2k stuff
2024-09-15 17:46:43 +00:00
ed 2927bbb2d6 strip dev-only asserts at build stage 2024-09-14 22:17:35 +00:00
ed 0527b59180 cosmetic: only print hostname warning once 2024-09-14 20:37:56 +00:00
ed 609c5921d4 list incoming files + ETA in controlpanel 2024-09-10 21:24:05 +00:00
ed e9d962f273 v1.15.1 2024-09-09 23:43:43 +00:00
ed b5405174ec add login sessions 2024-09-09 23:39:20 +00:00
ed a2e0f98693 disable upload deduplication by default;
dedup is still encouraged and fully supported, but
being default-enabled has caused too many surprises

enabling `--dedup` restores the previous default behavior

also renames `--never-symlink` to `--hardlink-only`
2024-09-08 17:09:14 +00:00
ed 6e671c5245 verify on-disk contents before dedup;
previously, the assumption was made that the database and filesystem
would not desync, and that an upload could safely be substituted with
a symlink to an existing copy on-disk, assuming said copy still
existed on-disk at all

this is fine if copyparty is the only software that makes changes to
the filesystem, but that is a shitty assumption to make in hindsight

add `--safe-dedup` which takes a "safety level", and by default (50)
it will no longer blindly expect that the filesystem has not been
altered through other means; the file contents will now be hashed
and compared to the database

deduplication can be much slower as a result, but definitely worth it
as this avoids some potentially very unpleasant surprises

the previous behavior can be restored with `--safe-dedup 1`
2024-09-06 19:08:14 +00:00
ed b599fbae97 use local timezone in log messages; closes #96
timezone can be changed with `export TZ=Europe/Oslo` before launch

using naive timestamps like this appears to be safe as of 3.13-rc1,
no deprecation warnings, just a tiny bit slower than assuming UTC
2024-09-05 19:31:33 +00:00
ed ad2371f810 shares: add revival and expiration extension 2024-08-30 22:25:50 +00:00
ed ba52590ae4 translation tweaks 2024-08-22 19:52:20 +00:00
ed 7ff46966da fix some issues with shares mentioned in #84;
* crash when root volume is unmapped
* rephrase login-page for shares
* add chrome support (lol)
* fix confusing helptext
* improve ux
  * placeholders in share creator
  * button to disable expiration in share creator
  * human-readable timestamps in share listing
2024-08-19 21:38:47 +00:00
ed 7c2beba555 add file/folder sharing; closes #84 2024-08-18 22:49:13 +00:00