Commit graph

322 commits

Author SHA1 Message Date
ed 6cb3b35a54 fix #65 (symlinks die when moved) 2023-12-08 21:28:20 +00:00
ed 9672b8c9b3 ensure nested symlinks are not broken during deletes;
when moving/deleting a file, all symlinked dupes are verified to ensure
this action does not break any symlinks, however it did this by checking
the realpath of each link. This was not good enough, since the deleted
file may be a part of a series of nested symlinks

this situation occurs because the deduper tries to keep relative
symlinks as close as possible, only traversing into parent/sibling
folders as required, which can lead to several levels of nested links
2023-12-08 01:11:03 +00:00
ed e70ecd98ef don't freak out when deleting a broken symlink,
also invoke the hooks with the corret lastmod time
2023-12-08 01:01:10 +00:00
ed 5f7ce78d7f avoid duplicate database entries when replacing files,
either from --daw, or by using u2c with --dr
2023-12-08 01:00:01 +00:00
ed 8c52b88767 make linters happier 2023-11-30 17:33:07 +00:00
ed c9fd26255b support environment variables mostly everywhere,
useful for docker/systemd stuff

also makes logfiles flush to disk per line by default;
can be disabled for a small performance gain with --no-logflush
2023-11-30 10:22:52 +00:00
ed 3c36727d07 fix filekeys not appearing in up2k in world-writable vols 2023-11-19 11:19:08 +00:00
ed fc4b51ad00 make dhash more volatile; probably fixes #61:
if any volumes were added or removed since last use,
drop dhash to verify that there are no files to shadow
2023-11-18 20:48:56 +00:00
ed bac733113c up2k-hasher robustness:
webdav clients tend to upload and then immediately delete
files to test for write-access and available disk space,
so don't crash and burn when that happens
2023-11-11 16:21:54 +00:00
ed 4b720f4150 add more prometheus metrics; breaking changes:
* cpp_uptime is now a gauge
* cpp_bans is now cpp_active_bans (and also a gauge)

and other related fixes:
* stop emitting invalid cpp_disk_size/free for offline volumes
* support overriding the spec-mandatory mimetype with ?mime=foo
2023-11-04 20:32:34 +00:00
ed 2523d76756 windows: fix symlinks 2023-11-03 17:16:12 +00:00
ed 1f75314463 placeholder expansion in readme and logues; closes #56
also fixes the "scan" volflag which broke in v1.9.14
2023-10-24 16:37:32 +00:00
ed 2529aa151d tersen volume listing on startup 2023-10-21 12:11:49 +00:00
ed e1d78d8b23 increase timeout of unfinished uploads from 6 to 24 hours
plus make it configurable
2023-10-20 18:31:28 +00:00
ed 4b5a0787ab option to show upload timestamps in directory listing;
enable with -mte +.ip_at
or volflag mte=+.ip_at

worst-case performance impact: 18%
2023-10-17 17:51:27 +00:00
ed dc2b67f155 ui-button to use upload-time instead of local last-modified 2023-10-15 08:46:23 +00:00
ed 753e3cfbaf revert 68c6794d (v1.6.2) and fix it better:
moving deduplicated files between volumes could drop some links
2023-10-07 22:25:44 +00:00
ed 0dc3c23b42 add alternative filekey generator; closes #52 2023-10-06 13:41:22 +00:00
ed a62f744a18 prevent losing an out-of-volume index
if the server is started while an external drive is not mounted,
it would drop the database because all the files are missing
2023-09-22 23:05:07 +00:00
ed c1efd227b7 fix inconsistent use of symlink mtimes in database;
on upload, dupes are by default handled by symlinking to the existing
copy on disk, writing the uploader's local mtime into the symlink mtime,
which is also what gets indexed in the db

this worked as intended, however during an -e2dsa rescan on startup the
symlink destination timestamps would be used instead, causing a reindex
and the resulting loss of uploader metadata (ip, timestamp)

will now always use the symlink's mtime;
worst-case 1% slower startup (no dhash)

this change will cause a reindex of incorrectly indexed files, however
as this has already happened at least once due to the bug being fixed,
there will be no additional loss of metadata
2023-09-01 20:29:55 +00:00
ed 5a38311481 mark offline volumes in directory tree sidebar 2023-08-26 19:00:46 +00:00
ed cafe53c055 v1.9.0 2023-08-20 22:02:40 +00:00
ed 135ece3fbd immediately allow uploading an interrupted and
deleted incomplete upload to another location
2023-08-20 19:16:35 +00:00
ed bd3640d256 change to openmetrics 2023-08-20 18:50:14 +00:00
ed fc0405c8f3 add prometheus metrics; closes #49 2023-08-20 17:58:06 +00:00
ed 015b87ee99 performance / cosmetic:
* js: use .call instead of .bind when possible
* when running without e2d, the message on startup regarding
  unfinished uploads didn't show the correct filesystem path
2023-08-16 19:32:43 +00:00
ed 406e413594 hint at additional context in exceptions 2023-08-15 01:42:13 +02:00
ed 1441ccee4f v1.8.4 2023-07-18 07:46:22 +00:00
ed 8f59afb159 fix another race (unpost):
unposting could collide with most other database-related activities,
causing one or the other to fail.
luckily the unprotected query performed by the unpost API happens to be
very cheap, so also the most likely to fail, and would succeed upon a
manual reattempt from the UI.
even in the worst case scenario, there would be no unrecoverable damage
as the next rescan would auto-repair any resulting inconsistencies.
2023-07-14 15:21:14 +00:00
ed 77f1e51444 fix unlikely race (e2tsr):
if someone with admin rights refreshes the homepage exactly as the
directory indexer decides to `_drop_caches`, the indexer thread would
die and the up2k instance would become inoperable...
luckily the probability of hitting this by chance is absolutely minimal,
and the worst case scenario is having to restart copyparty if this
happens immediately after startup; there is no risk of database damage
2023-07-14 15:20:25 +00:00
ed b54b7213a7 more thumbnailer configs available as volflags:
--th-convt = convt
--th-no-crop = nocrop
--th-size = thsize
2023-07-11 22:15:37 +00:00
ed e197895c10 support hashed passwords; closes #39 2023-06-25 21:50:33 +00:00
ed 8b0cf2c982 volflags to limit volume size / num files; closes #40 2023-06-19 00:42:45 +00:00
ed e64b87b99b dont hardlink symlinks (they could be relative) 2023-05-12 20:41:09 +00:00
ed b1359f039f linter cleanup 2023-05-07 14:38:30 +00:00
ed 83178d0836 preserve empty folders (closes #23):
* when deleting files, do not cascade upwards through empty folders
* when moving folders, also move any empty folders inside

the only remaining action which autoremoves empty folders is
files getting deleted as they expire volume lifetimes

also prevents accidentally moving parent folders into subfolders
(even though that actually worked surprisingly well)
2023-04-29 11:30:43 +00:00
ed cb6de0387d a bit faster 2023-04-26 19:56:27 +00:00
ed 209b49d771 remind sqlite we have indexes 2023-03-30 21:45:58 +00:00
ed e0c4829ec6 verify covers against db instead of fs 2023-03-15 19:48:43 +00:00
ed b0cc396bca v1.6.8 2023-03-12 16:10:07 +00:00
ed 2be2e9a0d8 index folder thumbs in db 2023-03-11 11:43:29 +00:00
ed 974ca773be just to be extra sure 2023-03-09 21:49:29 +00:00
ed c98fff1647 fix chunkpost-handshake race (affects --no-dedup only);
a handshake arriving in the middle of the final chunk could cause
dupes to become empty -- worst case leading to loss of data
2023-03-05 19:45:50 +00:00
ed db95e880b2 thats not how it works 2023-02-28 22:19:06 +00:00
ed 1c2c9db8f0 retain upload time (but not ip) on file reindex 2023-02-26 19:09:24 +00:00
ed 7ea183baef let http thread handle upload verification plugins 2023-02-26 19:07:49 +00:00
ed ab87fac6d8 db got the wrong lastmod when linking dupes 2023-02-26 18:52:04 +00:00
ed 1e3b7eee3b dont rmdir volume top on cleanup 2023-02-26 18:28:37 +00:00
ed 4de028fc3b let controlpanel rescan button override lack of e2dsa 2023-02-26 18:27:10 +00:00
ed 604e5dfaaf improve error handling / messages 2023-02-26 18:26:13 +00:00