Commit Graph

5377 Commits

Author SHA1 Message Date
Nick Craig-Wood 825dd65e1d Version v1.55.1 2021-04-26 08:32:21 +01:00
OleFrost 96cdbc5eaa onedrive: Work around for random "Unable to initialize RPS" errors
OneDrive randomly returns the error message: "InvalidAuthenticationToken: Unable to initialize RPS". These unexpected errors typically caused the entire rclone command to fail.

This work around recognizes these errors and marks them for a low level retry, that mostly succeeds. This will make rclone commands complete without being noticeable affected.

Fixes: #5270
2021-04-25 17:07:07 +01:00
buengese baf626c1d0 compress: fix compressed name regexp 2021-04-25 17:06:49 +01:00
Nick Craig-Wood e8be37bebc build: fix version numbers in android branch builds 2021-04-25 17:06:36 +01:00
Nick Craig-Wood e67ac64f51 dropbox: fix Unable to decrypt returned paths from changeNotify - fixes #5165
This was caused by incorrect use of strings.TrimLeft where
strings.TrimPrefix was required.
2021-04-21 10:53:59 +01:00
Nick Craig-Wood c86fc6e05d vfs: fix a code path which allows dirty data to be removed causing data loss
Before this change the VFS layer could remove a locally cached file
even if it had data which needed to be written back, thus causing data loss.

See: https://forum.rclone.org/t/rclone-1-55-doesnt-save-file-changes-if-the-file-has-been-reopened-during-upload-google-drive-mount/23646
2021-04-20 16:43:07 +01:00
Ansh Mittal a6cab5d0e9 drive: don't open browser when service account...
credentials specified 

Fixes #5104
2021-04-20 16:40:02 +01:00
Nick Craig-Wood 0d8c69b4ea dropbox: add missing team_data.member scope for use with --impersonate
See: https://forum.rclone.org/t/dropbox-business-not-accepting-oauth2/23390/32
2021-04-17 17:40:39 +01:00
Nick Craig-Wood 07f4b9832d dropbox: fix About after scopes changes - rclone config reconnect needed
This adds the missing scope for the About call. To use it it will be
necessary to refresh the token with `rclone config reconnect`.

See: https://forum.rclone.org/t/dropbox-too-many-requests-or-write-operations-trying-again-in-15-seconds/23316/33
2021-04-16 15:07:43 +01:00
Ivan Andreev c764ab2751 backend/ftp: fix implicit TLS after PR #4266 (#5219)
PR #4266 modified ftpConnection to make ftp library into using
a custom dial function which is QoS aware and takes care of TLS.
However the ServerConn.Login function from the ftp library also needs
TLS config passed explicitly as a trigger for sending PSBZ and PROT
options to FTP server. This was not taken care of resulting in
failure to connect via FTP with implicit TLS.
This PR fixes that.

Fixes #5210
2021-04-10 11:49:44 +01:00
Nick Craig-Wood 563c4f76c6 sync: fix incorrect error reported by graceful cutoff - fixes #5203
Before this change, a sync which was finished with a graceful transfer
cutoff could return "context canceled" instead of the correct error.

This fixes the problem by ignoring "context canceled" errors if we
have done a graceful stop.
2021-04-10 11:49:44 +01:00
albertony 2f6afe6005 docs: fixed some typos 2021-04-10 11:45:59 +01:00
buengese b30d7a026f zoho: do not ask for mountpoint twice when using headless setup 2021-04-10 11:45:59 +01:00
albertony 1a5e67fe98 docs: minor cleanup of space around code section 2021-04-10 11:45:59 +01:00
albertony 2d210c2729 docs: WinFsp 2021 is out of beta 2021-04-10 11:45:59 +01:00
Nick Craig-Wood 5603983a5d drive: fix backend copyid of google doc to directory - fixes #5196
Before this change the google doc was being copied to the directory
without an extension.
2021-04-06 11:48:17 +01:00
Nick Craig-Wood 501475a3a3 sftp: fix Update ReadFrom failed: failed to send packet: EOF errors
In

a3fcadddc8 sftp: close idle connections after --sftp-idle-timeout (1m by default)

Idle SFTP connections were closed after 1 minute. However due to the
way SSH multiplexes connections over a single SSH connection this
meant that if uploads or downloads went on for more than one minute
they failed with "EOF errors" as their underlying connection was
closed.

This fixes the problem by not clearing idle connections if there are
any transfers in progress.

Fixes #5197
2021-04-06 10:05:54 +01:00
Nick Craig-Wood 958f5297fe sftp: revert sftp library to v1.12.0 from v1.13.0 to fix performance regression #5197
This reverts the library update done in this commit.

713f8f357d sftp: fix "file not found" errors for read once servers

Reverting this commit triples the performance to a far away sftp server.

See: https://github.com/pkg/sftp/issues/426
2021-04-06 10:05:54 +01:00
buengese 572c6ad601 zoho: fix error when region isn't set 2021-04-06 10:05:54 +01:00
Ivan Andreev 9e2e2bb5fc build: add build tag noselfupdate
Allow downstream packaging to build rclone without selfupdate command:
$ go build -tags noselfupdate

Fixes #5187
2021-04-06 10:05:54 +01:00
Nick Gaya 12d039261f contributing.md: update setup instructions for go1.16 2021-04-06 10:05:54 +01:00
Ivan Andreev 74be50ae92 selfupdate: dont detect FUSE if build is static
Before this patch selfupdate detected ANY build with cmount tag as a build
having libFUSE capabilities. However, only dynamic builds really have it.
The official linux builds are static and have the cmount tag as of the time
of this writing. This results in inability to update official linux binaries.
This patch fixes that. The build can be fixed independently.
2021-04-06 10:05:54 +01:00
Nick Craig-Wood b0608d6f25 install.sh: fix macOS arm64 download - fixes #5183 2021-03-31 21:49:12 +01:00
Nick Craig-Wood b60838221b Start v1.55.1-DEV development 2021-03-31 21:49:02 +01:00
Nick Craig-Wood 6e2e2d9eb2 Version v1.55.0 2021-03-31 19:12:08 +01:00
Nick Craig-Wood 20e15e52a9 vfs: fix Create causing windows explorer to truncate files on CTRL-C CTRL-V
Before this fix, doing CTRL-C and CTRL-V on a file in Windows explorer
caused the **source** and the the destination to be truncated to 0.

This is because Windows opens the source file with Create with flags
`O_RDWR|O_CREATE|O_EXCL` but doesn't write to it - it only reads from
it. Rclone was taking the call to Create as a signal to always make a
new file, but this is incorrect.

This fix reads an existing file from the directory if it exists when
Create is called rather than always creating a new one. This fixes the
problem.

Fixes #5181
2021-03-31 14:48:02 +01:00
Nick Craig-Wood d0f8b4f479 fs/cache: fix recreation of backends after they have expired
Before this change, on the first attempt to create a backend we used a
non-canonicalized string. When the backend expired the second attempt
to create it would use the canonicalized string (because it was in the
remap cache) which would fail because it was now `name{XXXX}:`

This change makes sure that whenever we create a backend we always use
the non-canonicalized string.

See: https://forum.rclone.org/t/connection-string-inconsistencies-on-beta/23171
2021-03-30 18:46:30 +01:00
Nick Craig-Wood 58d82a5c73 rc: allow fs= params to be a JSON blob 2021-03-30 17:07:27 +01:00
Nick Craig-Wood c0c74003f2 fs/cache: add --fs-cache-expire-duration to control the fs cache
This commit makes the previously statically configured fs cache configurable.

It introduces two parameters `--fs-cache-expire-duration` and
`--fs-cache-expire-interval` to control the caching of the items.

It also adds new interfaces to lib/cache to set these.
2021-03-30 12:46:47 +01:00
Nick Craig-Wood 60bc7a079a rc: factor rc.Error out of rcserver for re-use in librclone #4891 2021-03-30 12:46:05 +01:00
Nick Craig-Wood 20c5ca08fb test_all: fix crash when using -clean 2021-03-29 23:12:53 +01:00
Nick Craig-Wood fc57648b75 lib/rest: fix multipart uploads stopping on context cancel
Before this change when the context was cancelled (due to
--max-duration for example) this could deadlock when uploading
multipart uploads.

This change fixes the problem by introducing another go routine to
monitor the context and close the pipe with an error when the context
errors.
2021-03-29 19:09:47 +01:00
Nick Craig-Wood 8c5c91e68f lib/readers: add NewContextReader to error on context errors 2021-03-29 19:09:47 +01:00
Nick Craig-Wood 9dd39e8524 Add x0b to contributors 2021-03-29 19:09:47 +01:00
albertony 9c9186183d
docs: add short description of configuration file format (#5142)
Fixes #572
2021-03-27 17:26:01 +01:00
Nick Craig-Wood 2ccf416e83 build: add version to android builds and fix upload 2021-03-26 09:18:54 +00:00
x0b 5577c7b760 build: replace xgo with NDK for Android builds 2021-03-26 09:18:54 +00:00
Nick Craig-Wood f6dbb98a1d cmount: update cgofuse to the latest version to bring in macfuse 4 fix 2021-03-25 09:02:17 +00:00
Nick Craig-Wood d042f3194f b2: fix html files downloaded via cloudflare
When reading files from B2 via cloudflare using --b2-download-url
cloudflare strips the Content-Length headers (presumably so it can
inject stuff into the body).

This caused rclone to think the file was corrupted as the length
didn't match.

The patch uses the old length read from the listing if there is no
Content-Length.

See: https://forum.rclone.org/t/b2-cloudflare-error-directory-not-found/23026
2021-03-24 17:06:59 +00:00
Nick Craig-Wood 524cd327e6 build: update notes on how to build the release manually with docker 2021-03-24 14:22:27 +00:00
Nick Craig-Wood b8c1cf7451 union: fix initialisation broken in refactor - fixes #5139
This commit broke the initialisation of the union backend

f17d7c0012 union: refactor to use fspath.SplitFs instead of fs.ParseRemote #4996

This patch fixes it.
2021-03-24 09:47:38 +00:00
Nick Craig-Wood 0fa68bda02 fspath: fix path parsing on Windows - fixes #5143
In this commit

8a46dd1b57 fspath: Implement a connection string parser #4996

The parsing code was re-written. This didn't quite work as before,
failing to adjust local paths on Windows when it should.

This patch fixes the problem and implements tests for it.
2021-03-24 09:47:03 +00:00
Nick Craig-Wood 1378bfee63 box: fix transfers getting stuck on token expiry after API change
Box recently changed their API, changing the case of returned API items

> On May 10th, 2021, as part of our continued infrastructure upgrade,
> Box's API response headers will standardize to return in a case
> insensitive manner, in line with industry best practices and our API
> documentation. Applications that are using these headers, such as
> "location" and "retry-after", will need to verify that their
> applications are checking for these headers in a case-insensitive
> fashion.

Rclone was reading the raw headers from the `http.Header` and not
using the `Get` accessor method which meant that it was sensitive to
case changes.

This fixes the problem by using the `Get` accessor method.

See: https://forum.rclone.org/t/box-backend-incompatible-with-box-api-changes-being-deployed/22972
2021-03-24 09:45:17 +00:00
nguyenhuuluan434 d6870473a1 swift: implement copying large objects 2021-03-24 08:56:39 +00:00
albertony 12cd322643
crypt: log hash ok on upload 2021-03-23 18:36:51 +01:00
Ivan Andreev 1406b6c3c9 install.sh: fail on download errors
This patch makes install.sh always run curl with flag "-f"
so it fails on download errors.
2021-03-23 11:29:00 +03:00
Ivan Andreev 088a83872d install.sh: fix some shellcheck warnings 2021-03-23 11:29:00 +03:00
Nick Craig-Wood cb46092883 lib/atexit: unregister interrupt handler once it has fired so users can interrupt again 2021-03-23 08:03:00 +00:00
Nick Craig-Wood a2cd5d8fa3 lib/atexit: fix deadlock calling Finalise while Run is running 2021-03-23 08:03:00 +00:00
Ivan Andreev 1fe2460e38 selfupdate: abort if updating would discard fuse semantics 2021-03-22 22:55:24 +03:00