Commit graph

3286 commits

Author SHA1 Message Date
ed 250c8c56f0 fix deadlock on IBM mainframes (s390x) 2024-08-02 23:05:44 +00:00
ed e136231c8e docker: add portainer howto 2024-08-02 23:01:32 +00:00
ed 98ffaadf52 docker: use less RAM at runtime
compile to bytecode so cpython doesn't have to keep it in memory

ram usage reduced by:
* min: 5.4 MiB (32.6 to 27.2)
* ac/im: 5.2 MiB (39.0 to 33.8)
* dj/iv: 10.6 MiB (67.3 to 56.7)

startup time reduced from:
* min: 1.3s to 0.6s
* ac/im: 1.6s to 0.9s
* dj/iv: 2.0s to 1.1s

image size increased by 4 MiB (min), 6 MiB (ac/im/iv), 9 MiB (dj)

ram usage measured on idle with:
while true; do ps aux | grep -E 'R[S]S|no[-]crt'; read -n1; echo; done

startup time measured with:
time podman run --rm -it localhost/copyparty-min-amd64 --exit=idx
2024-08-02 22:11:23 +00:00
ed ebb1981803 py2: reduce ram usage 2024-08-01 20:01:42 +00:00
ed 72361c99e1 add import chickenbits 2024-08-01 18:29:25 +00:00
ed d5c9c8ebbd make it 5% faster 2024-07-31 17:51:53 +00:00
ed 746229846d add test for zip-download 2024-07-30 22:44:29 +00:00
ed ffd7cd3ca8 update pkgs to 1.13.6 2024-07-29 20:56:00 +00:00
ed b3cecabca3 v1.13.6 2024-07-29 20:28:51 +00:00
ed 662541c64c audio-player: show status while loading 2024-07-29 20:14:39 +00:00
ed 225bd80ea8 up2k.js: fix overshoot in chunk stitcher 2024-07-29 19:19:22 +00:00
ed 85e54980cc up2k.js: set timeouts for uploads
in the event that an upload chunk gets stuck, the js would
never stop waiting for a response, requiring a page reload

improves reliability when running behind a reverse-proxy
which is configured to never timeout requests (can make
sense when combined with other services on the same box)
2024-07-29 19:17:03 +00:00
ed a19a0fa9f3 fix modal wordwrap in firefox;
with overflow:auto, firefox picks the div-width before estimating
the height, causing it to undershoot by the scrollbar width
and then messing up the text alignment

fix: conditionally set overflow-y:scroll using js
2024-07-29 18:04:35 +00:00
ed 9bb6e0dc62 misc ux:
* wait until page (au) has loaded to register hotkeys
* hotkey `m` would grow sidebar if tree was minimized
* more exact warning about num.parallel uploads
* keep more console logs in memory
* message phrasing
2024-07-29 17:59:34 +00:00
ed 15ddcf53e7 add bsod theme 2024-07-26 22:09:59 +00:00
ed 6b54972ec0 update comparison vs similar software:
* general changes:
  * upload speed comparisons considering v1.13.5

* hfs2:
  * dead project with unfixed vulnerabilities

* hfs3:
  * has replaced hfs2
  * uploads are now resumable
  * add new functionality:
    * write-only folders
    * unmap subfolders
    * move and delete files
    * folder-rproxy
    * themes
    * basic audio player, image viewer

* filebrowser:
  * uploads are now parallelized, resumable, segmented
    * but single large files are not accelerated
  * can listen on unix sockets
  * folder-rproxy is supported
  * more cpu efficient than copyparty
2024-07-26 19:46:03 +00:00
ed 0219eada23 cleanup: strip trailing whitespace 2024-07-26 19:33:56 +00:00
ed 8916bce306 u2c fixes:
* `--sz` was num.chunks, not the intended MiB
* crash on exit with `-z` and no modified files
* summary upload elapsed-time could exceed wallclock
2024-07-26 19:28:47 +00:00
ed 99edba4fd9 change xm examples to reject users without write-access; #68 2024-07-25 19:23:08 +00:00
ed 64de3e01e8 update pkgs to 1.13.5 2024-07-22 23:48:24 +00:00
ed 8222ccc40b v1.13.5 2024-07-22 23:23:53 +00:00
ed dc449bf8b0 fix grid toolbar undocking after viewing a pic/vid 2024-07-22 23:09:25 +00:00
ed ef0ecf878b recommend rclone over davfs2; closes #90 2024-07-22 22:46:24 +00:00
ed 53f1e3c91d ui option to play video as audio
audio extraction happens serverside to opus or mp3
depending on browser support

remuxing (extracting audio without transcoding)
is currently not supported, and is not planned
2024-07-22 22:30:21 +00:00
ed eeef80919f css-fix for firefox52 (centos6) 2024-07-22 20:59:05 +00:00
ed 987bce2182 u2c fixes:
* don't stitch across deduplicated blocks
* print speed/time for hash/upload
* more compact json in handshakes
2024-07-22 20:55:32 +00:00
ed b511d686f0 up2k fixes:
* progress donuts should include inflight bytes
* changes to stitch-size in settings didn't apply until next refresh
* serverlog was too verbose; truncate chunk hashes
* mention absolute cloudflare limit in readme
2024-07-22 19:06:01 +00:00
ed 132a83501e add chunk stitching; twice as fast long-distance uploads:
rather than sending each file chunk as a separate HTTP request,
sibling chunks will now be fused together into larger HTTP POSTs
which results in unreasonably huge speed boosts on some routes
( `2.6x` from Norway to US-East,  `1.6x` from US-West to Finland )

the `x-up2k-hash` request header now takes a comma-separated list
of chunk hashes, which must all be sibling chunks, resulting in
one large consecutive range of file data as the post body

a new global-option `--u2sz`, default `1,64,96`, sets the target
request size as 64 MiB, allowing the settings ui to specify any
value between 1 and 96 MiB, which is cloudflare's max value

this does not cause any issues for resumable uploads; thanks to the
streaming HTTP POST parser, each chunk will be verified and written
to disk as they arrive, meaning only the untransmitted chunks will
have to be resent in the event of a connection drop -- of course
assuming there are no misconfigured WAFs or caching-proxies

the previous up2k approach of uploading each chunk in a separate HTTP
POST was inefficient in many real-world scenarios, mainly due to TCP
window-scaling behaving erratically in some IXPs / along some routes

a particular link from Norway to Virginia,US is unusably slow for
the first 4 MiB, only reaching optimal speeds after 100 MiB, and
then immediately resets the scale when the request has been sent;
connection reuse does not help in this case

on this route, the basic-uploader was somehow faster than up2k
with 6 parallel uploads; only time i've seen this
2024-07-21 23:35:37 +00:00
ed e565ad5f55 better errors through broker 2024-07-21 20:36:50 +00:00
ed f955d2bd58 dangit 2024-07-20 22:28:40 +00:00
ed 5953399090 add helptext exporters (html, txt) 2024-07-17 23:06:01 +00:00
ed d26a944d95 hooks: add cache-warmer 2024-07-17 21:00:59 +00:00
ed 50dac15568 update pkgs to 1.13.4 2024-07-16 05:48:45 +00:00
ed ac1e11e4ce v1.13.4 2024-07-16 04:57:26 +00:00
ed d749683d48 hooks: add permission filtering, argv-prepend;
hooks can be restricted to users with certain permissions, for example
`--xm aw,notify-send` will only `notify-send` if user has write-access

the user's list of permissions are now also included in the json
that is passed to the hook if enabled; `--xm aw,j,notify-send`

will now also stop parsing flags when encountering a blank value,
allowing to specify any initial arguments to the command:
`--xm aw,j,,notify-send,hey` would run `notify-send` with `hey`
as its first argument, and the json would be the 2nd argument,
similarly `--xm ,notify-send,hey` when no flags specified

this is somewhat explained in `--help-hooks`, but
additional related features are planned in the near future
and will all be better documented when the dust settles
2024-07-16 04:45:02 +00:00
ed 84e8e1ddfb ftpd: only mention vols that user can access
if an ftp client tried to list the toplevel folder on a server
where nothing is mounted toplevel, it would syntheisze a
directory listing which included all volumes, even those
which the user would not be able to access

so basically not a problem, just very confusing
2024-07-15 21:24:26 +00:00
ed 6e58514b84 update deps:
* win10:
  * python 3.12
  * pillow 10.4
  * pyinstaller 6.9
* win: upx 4.2.4
* web: dompurify 3.1.6
2024-07-15 21:16:19 +00:00
ed 803e156509 hooks: improve torrent downloader 2024-07-14 17:57:36 +00:00
ed c06aa683eb allow audio-DL regardless of current folder 2024-07-13 17:10:24 +02:00
ed 6644ceef49 mention davfs2 workaround, closes #91 2024-07-13 16:55:27 +02:00
ed bd3b3863ae hooks: add bittorrent downloader 2024-07-13 01:37:17 +02:00
ed ffd4f9c8b9 hooks: describe examples better 2024-07-13 01:32:26 +02:00
ed 760ff2db72 other linter nitpicks (not actually bugs) 2024-07-13 01:18:14 +02:00
ed f37187a041 fix bugs detected by pyright but not pylance:
* race-the-beam broke in v1.13.3 (i'm good at this)

* wrong logger type in certgen
2024-07-13 01:09:19 +02:00
ed 1cdb170290 order-significant --th-covers;
the first matching filename as listed in the
`--th-covers` global-option will always be selected
2024-07-13 00:54:38 +02:00
ed d5de3f2fe0 improve --cgen (configfile generator) 2024-07-12 22:57:57 +02:00
ed d76673e62d use correct mtime for folder thumbs;
mtime the file that was used to produce the folder thumbnail
(rather than the folder itself) since the folder-thumb is
always resolved to the file's thumb in the on-disk cache
2024-07-11 23:12:51 +02:00
ed c549f367c1 reduce timeout of unbounded socket reads;
if a request body is expected, but request has no content-length,
set the timeout to 1/20 of `--s-tbody`, so 9 seconds by default,
or 3 seconds if it's 60 as recommended in helptext

this gives less confusing behavior if a client accidentally does
something invalid, replying with an error response before the
previous timeout of 186 seconds

also raise the slowloris flag, in case a client bugs out and
keeps making such requests
2024-07-10 11:14:42 +02:00
ed 927c3bce96 support descript.ion; makes listings 2% slower 2024-07-06 17:02:33 +02:00
ed d75a2c77da og: fix viewing readmes 2024-07-06 16:55:15 +02:00