Commit Graph

6526 Commits

Author SHA1 Message Date
Nick Craig-Wood ab8be2663f Version v1.60.1 2022-11-17 13:56:19 +00:00
asdffdsazqqq 9e2aaa0f67 docs: faq: how to use a proxy server that requires a username and password - fixes #6565 2022-11-17 13:44:24 +00:00
Nick Craig-Wood f8997b8336 webdav: fix Move/Copy/DirMove when using -server-side-across-configs
Before this change, when using -server-side-across-configs rclone
would direct Move/Copy/DirMove to the destination server.

However this should be directed to the source server. This is a little
unclear in the RFC, but the name of the parameter "Destination:" seems
clear and this is how dCache and Rucio have implemented it.

See: https://forum.rclone.org/t/webdav-copy-request-implemented-incorrectly/34072/
2022-11-17 13:44:02 +00:00
Nick Craig-Wood 567d8fc8d1 smb: fix Failed to sync: context canceled at the end of syncs
Before this change we were putting connections into the connection
pool which had a local context in.

This meant that when the operation had finished the context was
cancelled and the connection became unusable.

See: https://forum.rclone.org/t/failed-to-sync-context-canceled/34017/
2022-11-16 10:56:09 +00:00
Nick Craig-Wood 4ffe9dcfef vfs: fix deadlock caused by cache cleaner and upload finishing
Before this patch a deadlock could occur if the cache cleaner was
running when an object upload finished.

This fixes the problem by delaying marking the object as clean until
we have notified the VFS layer. This means that the cache cleaner
won't consider the object until **after** the VFS layer has been
notified, thus avoiding the deadlock.

See: https://forum.rclone.org/t/rclone-mount-deadlock-when-dir-cache-time-strikes/33486/
2022-11-15 18:01:46 +00:00
techknowlogick 6d0824954b s3: Add additional Wasabi locations 2022-11-15 09:02:48 +00:00
Aaron Gokaslan 1a5160c7aa s3: fix for unchecked err value in s3 listv2 2022-11-15 09:02:25 +00:00
Nick Craig-Wood 5012f78c45 local: fix -L/--copy-links with filters missing directories
In this commit

8d1fff9a82 local: obey file filters in listing to fix errors on excluded files

We introduced the concept of local backend filters.

Unfortunately the filters were being applied before we had resolved
the symlink to point to a directory. This meant that symlinks pointing
to directories were filtered out when they shouldn't have been.

This was fixed by moving the filter check until after the symlink had
been resolved.

See: https://forum.rclone.org/t/copy-links-not-following-symlinks-on-1-60-0/34073/7
2022-11-15 09:00:08 +00:00
Nick Craig-Wood 8f3741419f mailru: allow timestamps to be before the epoch 1970-01-01
Fixes #6547
2022-11-10 11:27:50 +00:00
Nick Craig-Wood 91afa14d28 lib/cache: fix alias backend shutting down too soon
Before this patch, when an alias backend was created it would be
renamed to be canonical and in the process Shutdown would be called on
it. This was particularly noticeable with the dropbox backend which
gave this error when uploading files after the backend was Shutdown.

    Failed to copy: upload failed: batcher is shutting down

This patch fixes the cache Rename code not to finalize objects if the
object that is being overwritten is the same as the existing object.

See: https://forum.rclone.org/t/upload-failed-batcher-is-shutting-down/33900
2022-11-09 16:30:33 +00:00
Nick Craig-Wood 4038c2c316 mailru: note that an app password is now needed - fixes #6398 2022-11-08 20:33:33 +00:00
Nick Craig-Wood ed7c286ef6 docs: remove link to rclone slack as it is no longer supported 2022-11-08 16:12:02 +00:00
Nick Craig-Wood 0866a7b6e2 s3: allow Storj to server side copy since it seems to work now - fixes #6550 2022-11-08 16:08:29 +00:00
João Henrique Franco ac38744e69 wasm: fix walltime link error by adding up-to-date wasm_exec.js
Solves link error while running rclone's wasm version. Go's `walltime1` function was renamed to `walltime`. This commit updates wasm_exec.js with the new name.
2022-11-08 16:08:29 +00:00
Clément Notin dfc21a5b9e docs: fix character that was incorrectly interpreted as markdown 2022-11-08 16:08:29 +00:00
Joda Stößer ff408f5309 docs: fix typo `remove` in rclone_serve_restic command 2022-11-08 16:08:29 +00:00
Nick Craig-Wood e2d2a05361 s3: add provider quirk --s3-might-gzip to fix corrupted on transfer: sizes differ
Before this change, some files were giving this error when downloaded
from Cloudflare and other providers.

    ERROR corrupted on transfer: sizes differ NNN vs MMM

This is because these providers auto gzips the object when rclone
wasn't expecting it to. (AWS does not gzip objects without their being
uploaded gzipped).

This patch adds a quirk to for fix the problem and a flag to control
it. The quirk `might_gzip` is set to `true` for all providers except
AWS.

See: https://forum.rclone.org/t/s3-error-corrupted-on-transfer-sizes-differ-nnn-vs-mmm/33694/
Fixes: #6533
2022-11-04 17:01:03 +00:00
albertony e8002c66a9 docs: add direct download link for windows arm64 2022-11-04 17:01:03 +00:00
coultonluke 6f207668c2 docs: corrected download links in windows install docs 2022-11-04 17:01:03 +00:00
Manoj Ghosh a2491719a1 oracle-object-storage: doc fix
See #6521
2022-11-04 17:01:03 +00:00
albertony 29e1c68ee0 local: clean absolute paths - fixes #6493 2022-11-04 17:01:03 +00:00
Samuel Johnson 05040cc504 docs: Update faq.md with bisync
Updated FAQ to clarify that experimental bi-sync is now available.
2022-11-04 17:01:03 +00:00
Nick Craig-Wood 6944f28dc2 Start v1.60.1-DEV development 2022-11-04 16:57:08 +00:00
Nick Craig-Wood 01dbbff62e Version v1.60.0 2022-10-21 15:06:08 +01:00
Nick Craig-Wood afa61e702c docs: remove hosted by tag as server has moved 2022-10-21 12:49:10 +01:00
Nick Craig-Wood 546dc82793 Add Robert Newson to contributors 2022-10-21 12:49:10 +01:00
Nick Craig-Wood d9c4d95ab3 Add Tom Mombourquette to contributors 2022-10-21 12:49:10 +01:00
Nick Craig-Wood 0fb1b75a02 Add Manoj Ghosh to contributors 2022-10-21 12:49:10 +01:00
Tom Mombourquette 38f1f5b177 rc: Fix mount/listmounts not returning the full Fs entered in mount/mount 2022-10-21 12:48:27 +01:00
Ole Frost 0d2a62a927 docs: Describe connection strings in alias backend 2022-10-21 12:47:51 +01:00
Manoj Ghosh b75c207208 oracle-object-storage: overview, docs update 2022-10-21 12:47:03 +01:00
Ole Frost dff223f195 install.sh: fix arm-v7 download 2022-10-21 12:35:58 +01:00
Robert Newson d2fef05fe4
httplib: Add --xxx-min-tls-version option to select minimum tls values for HTTP servers
This allows administrators to disable TLS 1.0 and 1.1, for example.

Example:

rclone rcd --rc-min-tls-version=tls1.2 --rc-cert <cert> --rc-key <key>
2022-10-19 17:13:12 +01:00
Tom Mombourquette 188b9f8cf1 rc: corrected mount/unmountall help msg and title 2022-10-17 17:34:48 +01:00
Manoj Ghosh daf3162bcf oracle-object-storage: minor docs update 2022-10-17 17:08:44 +01:00
Nick Craig-Wood 5e59e7f442 ftp: Fix hang when using ExplicitTLS to certain servers.
It was discovered that doing the tls Handshake immediately on
connection causes some FTP servers (proftpd and pureftpd) to hang.

This imports a fix for it by temporarily hard forking jlaffaye/ftp to
include the fix submitted as a pull request.

See: https://forum.rclone.org/t/rclone-ftps-explicit-rclone-touch-empty-files-proftpd-unable-to-build-data-connection-operation-not-permitted/22522
See: https://github.com/rclone/rclone/issues/6426#issuecomment-1243993039
See: https://github.com/jlaffaye/ftp/pull/283
See: https://github.com/jlaffaye/ftp/issues/282
2022-10-14 12:10:03 +01:00
Nick Craig-Wood fce22c0065 s3: add --s3-no-system-metadata to suppress read and write of system metadata
See: https://forum.rclone.org/t/problems-with-content-disposition-and-backblaze-b2-using-s3/33292/
2022-10-14 11:12:04 +01:00
Nick Craig-Wood bb3272e837 Add Bachue Zhou to contributors 2022-10-14 11:11:56 +01:00
Nick Craig-Wood cb5b5635c7 Add Manoj Ghosh to contributors 2022-10-14 11:11:56 +01:00
Bachue Zhou 66ed0ca726
s3: add Qiniu KODO to s3 provider list - fixes #6195 2022-10-13 15:49:22 +01:00
Manoj Ghosh b16e50851a
Add a native backend for oracle object storage - fixes #6299 2022-10-13 13:04:56 +01:00
Nick Craig-Wood 90d23139f6 s3: drop binary metadata with an ERROR message
Before this change, rclone would attempt to upload metadata with
binary contents which fail to be uploaded by net/http.

This checks the keys and values for validity as http header values
before uploading.

See: https://forum.rclone.org/t/invalid-metadata-key-names-result-in-a-failure-to-transfer-xattr-results-in-failure-to-upload-net-http-invalid-header-field-value-for-x-amz-meta-samba-pai/33406/
2022-10-13 12:00:45 +01:00
Nick Craig-Wood 5ea9398b63 swift: add --swift-no-large-objects to reduce HEAD requests
Supplying the flag --swift-no-large-objects is a promise to the swift
backend that there are no dynamic or static large objects stored.

Using that knowledge rclone can speed its operations up reducing the
number of HEAD requests.

See: https://forum.rclone.org/t/handling-or-not-of-large-objects-in-swift/33389/
See: https://forum.rclone.org/t/swift-sync-checksum-calls-head-on-every-object-so-is-very-slow/30322
2022-10-13 11:58:19 +01:00
Isaac Aymerich 3f804224f4
rc: validate Daemon option is not set when mounting a volume via RC - fixes #6469 2022-10-12 12:07:48 +01:00
Nick Craig-Wood cf0bf159ab s3: try to keep the maximum precision in ModTime with --user-server-modtime
Before this change if --user-server-modtime was in use the ModTime
could change for an object as we receive it accurate to the nearest ms
in listings, but only accurate to the nearest second in HEAD and GET
requests.

Normally AWS returns the milliseconds as .000 in listings, but if
versions are in use it may not. Storj S3 also seems to return
milliseconds.

This patch tries to keep the maximum precision in the last modified
time, so it doesn't update a last modified time with a truncated
version if the times were the same to the nearest second.

See: https://forum.rclone.org/t/cache-fingerprint-miss-behavior-leading-to-false-positive-stalen-cache/33404/
2022-10-12 09:18:10 +01:00
Lesmiscore 6654b66114 union: propagate SlowHash feature 2022-10-10 07:58:01 +01:00
Nick Craig-Wood 9bf78d0373 local: fix "Failed to read metadata: function not implemented" on old Linux kernels
Before this change rclone used statx() to read the metadata for files
from the local filesystem when `-M` was in use.

Unfortunately statx() was only introduced in kernel 4.11 which was
released in April 2017 so there are current systems (eg Centos 7)
still on kernel versions which don't support statx().

This patch checks to see if statx() is available and if it isn't, it
falls back to using fstatat() which was introduced in Linux 2.6.16
which is guaranteed for all Go versions.

See: https://forum.rclone.org/t/metadata-from-linux-local-s3-failed-to-copy-failed-to-read-metadata-from-source-object-function-not-implemented/33233/
2022-10-07 14:14:16 +01:00
Nick Craig-Wood 0c1fb8b2b7 Add YanceyChiew to contributors 2022-10-07 14:14:08 +01:00
YanceyChiew 966654e23a dlna: run assets_generate to make new icons 2022-10-06 16:59:51 +01:00
YanceyChiew 13b65104eb dlna: add SSDP AnnounceInterval flag option
The current default AnnounceInterval is too short, causing the
multicast domain to be flooded with NOTIFY announcements,
which may prevent other dlna devices from sleeping.

This change allows users to set the announcement interval,
and it's default value also increased to 12 minutes.

Even within the interval, rclone can still passively respond to
M-SEARCH requests from other devices.
2022-10-06 16:59:51 +01:00