copyparty/tests
ed 132a83501e add chunk stitching; twice as fast long-distance uploads:
rather than sending each file chunk as a separate HTTP request,
sibling chunks will now be fused together into larger HTTP POSTs
which results in unreasonably huge speed boosts on some routes
( `2.6x` from Norway to US-East,  `1.6x` from US-West to Finland )

the `x-up2k-hash` request header now takes a comma-separated list
of chunk hashes, which must all be sibling chunks, resulting in
one large consecutive range of file data as the post body

a new global-option `--u2sz`, default `1,64,96`, sets the target
request size as 64 MiB, allowing the settings ui to specify any
value between 1 and 96 MiB, which is cloudflare's max value

this does not cause any issues for resumable uploads; thanks to the
streaming HTTP POST parser, each chunk will be verified and written
to disk as they arrive, meaning only the untransmitted chunks will
have to be resent in the event of a connection drop -- of course
assuming there are no misconfigured WAFs or caching-proxies

the previous up2k approach of uploading each chunk in a separate HTTP
POST was inefficient in many real-world scenarios, mainly due to TCP
window-scaling behaving erratically in some IXPs / along some routes

a particular link from Norway to Virginia,US is unusably slow for
the first 4 MiB, only reaching optimal speeds after 100 MiB, and
then immediately resets the scale when the request has been sent;
connection reuse does not help in this case

on this route, the basic-uploader was somehow faster than up2k
with 6 parallel uploads; only time i've seen this
2024-07-21 23:35:37 +00:00
..
res/idp idp: precise expansion of ${u} (fixes #79); 2024-03-21 20:10:27 +00:00
__init__.py fix write-only volumes + add regression test 2021-04-24 02:48:41 +02:00
ptrav.py make linters happier 2023-11-30 17:33:07 +00:00
run.py make linters happier 2023-11-30 17:33:07 +00:00
test_dots.py make thumbnails compatible with dirkeys/filekeys 2024-06-16 21:35:43 +02:00
test_dxml.py make linters happier 2023-11-30 17:33:07 +00:00
test_httpcli.py make linters happier 2023-11-30 17:33:07 +00:00
test_idp.py fix tests on windows 2024-04-25 22:25:38 +00:00
test_vfs.py list dotfiles only for specific volumes or users (#66): 2023-12-16 15:38:48 +00:00
util.py add chunk stitching; twice as fast long-distance uploads: 2024-07-21 23:35:37 +00:00