Commit Graph

4512 Commits

Author SHA1 Message Date
Nick Craig-Wood 25662b9e05 fstest: add ability for mock objects and filesystems to have hashes 2020-06-30 12:01:36 +01:00
Nick Craig-Wood c820576329 fs: define SlowModTime and SlowHash features in the relevant backends 2020-06-30 12:01:36 +01:00
Nick Craig-Wood af601575cb serve ftp: Use new facilities in goftp to fix and simplify auth proxy #4394
- Use Driver.CheckPasswd instead of server.CheckPasswd
- Make server.CheckPasswd return an error
- Remove awful findID to find parent function hack
- Remove Driver.Init as it is no longer called
- Fix backwards incompatible PublicIp -> PublicIP change

See: https://gitea.com/goftp/server/issues/117
2020-06-30 09:34:13 +01:00
Nick Craig-Wood c7eae60944 vendor: update goftp.io/server to v0.3.5-master to fix auth proxy #4394
See upstream issue: https://gitea.com/goftp/server/issues/117
2020-06-30 09:16:43 +01:00
Nick Craig-Wood 0afd5a2204 build: drop xgo builds for macOS, Linux and Windows
The xgo builds for macOS, Linux and Windows are used for testing - the
actual builds are built on the correct platform.

Since the darwin build has stopped working, this can be an excuse for
removing these builds as they really are only for testing.

The Android and IOS builds will continue to be built by xgo

See: https://github.com/billziss-gh/cgofuse/issues/47
2020-06-29 14:51:16 +01:00
Nick Craig-Wood 92cb21f0f2 serve ftp: Add error message if auth proxy fails #4394 2020-06-29 14:45:39 +01:00
Nick Craig-Wood 0031130111 serve ftp: don't compile on < go1.13 after dependency update 2020-06-29 14:45:39 +01:00
Nick Craig-Wood 2a3b377d34 azureblob: don't compile on < go1.13 after dependency update 2020-06-29 14:45:39 +01:00
Nick Craig-Wood 2aed3bf9ab vendor: roll back bazil.org/fuse to the last version which supports macOS #4393
Roll back the bazil.org/fuse update to give us some time to explore
alternatives for macOS.

See upstream issue: https://github.com/bazil/fuse/issues/224
2020-06-29 14:45:02 +01:00
Nick Craig-Wood ec4e0e4d58 vendor: revert goftp.io/server from v0.3.4 to v0.3.3 to fix auth proxy #4394
See upstream issue: https://gitea.com/goftp/server/issues/117
2020-06-29 14:45:02 +01:00
Nick Craig-Wood 696d012c05 vendor: update all dependencies 2020-06-29 14:44:57 +01:00
Nick Craig-Wood 61ff7306ae crypt: add --crypt-server-side-across-configs flag
This can be used for changing filename encryption mode without
re-uploading data.

See: https://forum.rclone.org/t/revert-filename-encryption-method/17454/
2020-06-27 11:40:15 +01:00
Nick Craig-Wood 0bcf4769fe local: make --local-no-updated provide a consistent view of the objects
Before this change the --local-no-updated flag would not error if the
files changed in size during the transfer. The file could still be
read beyond the size advertised though which caused problems with
certain backends.

After this change we attempt to provide a consistent view of the file
once it has been opened.

Once the file has had stat() called on it for the first time we

- Only transfer the size that stat gave
- Only checksum the size that stat gave
- Don't update the stat info for the file

This means that files that are extending can be transferred - rclone
will transfer the length it saw the first time it listed the file.

See: https://forum.rclone.org/t/transport-connection-broken/16494/21
2020-06-27 10:00:43 +01:00
Nick Craig-Wood 0bfbecf9cb build: add btest Makefile target for pasteable download message 2020-06-26 16:26:29 +01:00
David 9058ec32e1 s3: Use regional s3 us-east-1 endpoint 2020-06-26 16:25:52 +01:00
Nick Craig-Wood 61e4b4db42 drive: Allow the use of --drive-impersonate with the root_folder_id "appDataFolder"
In this commit

5c5ad6220 drive: fix --drive-impersonate with cached root_folder_id

We disabled the use of root_folder_id with --drive-impersonate to fix
a problem with a cached root_folder_id giving the wrong results.

This, alas, broke one users setup with a root_folder_id of
appDataFolder. Since this is identifiable and definitely couldn't have
been cached, we can safely skip this check in this case.

See: https://forum.rclone.org/t/rclone-gdrive-no-longer-returning-anything/17215/10
2020-06-25 21:43:11 +01:00
Nick Craig-Wood fd7c63bc78 s3: add backend restore command to restore objects from GLACIER
See: https://forum.rclone.org/t/rclone-settier-fails-with-scaleway-entitytoolarge/17384
2020-06-25 21:33:23 +01:00
Nick Craig-Wood 49a7d08a40 qingstor: cancel in progress multipart uploads on rclone exit #4300 2020-06-25 15:22:53 +01:00
Nick Craig-Wood 2c10ce64aa onedrive: rework cancel of multipart uploads on rclone exit #4300
This now uses the atexit.OnError framework rather than a home grown one.
2020-06-25 15:22:53 +01:00
Nick Craig-Wood a41a294e1d box: cancel in progress multipart uploads and copies on rclone exit #4300 2020-06-25 15:22:53 +01:00
Nick Craig-Wood 47b17dc1bb b2: cancel in progress multipart uploads and copies on rclone exit #4300 2020-06-25 15:22:53 +01:00
Nick Craig-Wood 5f75444ef6 s3: cancel in progress multipart uploads and copies on rclone exit #4300 2020-06-25 12:55:56 +01:00
Nick Craig-Wood 54fda3422e atexit: implement OnError for cancelling multpart uploads 2020-06-25 12:55:56 +01:00
Nick Craig-Wood fcc2db8093 doc: disable smart typography (eg en-dash) in MANUAL.* and man page
Before this change MANUAL.html and rclone.1 would show flags like –addr

Now it shows --addr which is easy to copy and paste.

This was pointed out in #4362
2020-06-25 12:20:01 +01:00
Nick Craig-Wood 89b7ffbd5c Add Tim Burke to contributors 2020-06-25 12:20:01 +01:00
Nick Craig-Wood ada43b0e58 Add Petri Salminen to contributors 2020-06-25 12:20:01 +01:00
Tim Burke 5050c33162 dlna: Mark flags in docs as code
Otherwise, we get en dashes in the man page, making args more difficult
to copy/paste to a command line.

Before:

    Use –addr to specify ...

After:

    Use --addr to specify ...
2020-06-25 12:18:54 +01:00
Harry 4e8fda228d
docs: Updated OneDrive max file size 2020-06-25 12:04:52 +01:00
Petri Salminen cdfb3f7194 docs: make the website navbar stick to top
Navbar will remain visible on the top of the screen, even when
the user has scrolled down.

This makes it easier to navigate the site quickly.
2020-06-25 11:47:56 +01:00
Chaitanya Bankanhal a2dd23efd3 rc: Add tests for operations/uploadfile
rc: Go import file rc_test.go
2020-06-25 11:38:24 +01:00
Chaitanya Bankanhal fa43d02874 rc: Add operations/uploadfile to upload a file through rc using encoding multipart/form-data 2020-06-25 11:38:24 +01:00
Chaitanya d0de39ebcd rc: add NeedsRequest to call. 2020-06-25 11:38:24 +01:00
Nick Craig-Wood 2121c0fa23 dircache: factor DirMove code out of backends into dircache
Before this change there was lots of duplicated code in all the
dircache using backends to support DirMove.

This change factors this code into the dircache library.
2020-06-25 09:41:36 +01:00
Nick Craig-Wood a8652e2252 dircache: simplify interface, fix corner cases and apply to backends
Dircache was changed to:

- Remove special cases for the root directory
- Remove Fatal errors
- Call FindRoot on behalf of the user wherever possible
- Bring up to modern Go standards

Backends were changed to:

- Remove calls to FindRoot
- Change calls to FindRootAndPath to FindPath
- Don't make special cases for the root

This fixes several corner cases, for example removing a non existent
directory if FindRoot hasn't been called.
2020-06-25 09:41:36 +01:00
Nick Craig-Wood 81151523af drive: fix shortcut tests 2020-06-24 15:52:02 +01:00
Nick Craig-Wood 3e82771413 Start v1.52.2-DEV development 2020-06-24 14:35:12 +01:00
Nick Craig-Wood 9445b12328 check: make it show stats by default 2020-06-24 11:23:34 +01:00
Nick Craig-Wood 4bb103ef43 build: speed up tidy-beta script by doing fewer directory traversals 2020-06-24 10:01:24 +01:00
Nick Craig-Wood 0dba7b8a46 swift: speed up deletes by not retrying segment container deletes
Before this fix rclone would continually try to delete non empty
segment containers which made deleting lots of files very slow.

This fix makes rclone just try the delete once and then carry on which
was the original intent of the code before the retry logic got put in.
2020-06-24 10:01:24 +01:00
buengese e247811db5 jottacloud: remove debug Printf accidentally left in 2020-06-23 13:16:23 +02:00
buengese 6768f999ed docs/overview: pcloud now supports link sharing 2020-06-21 17:22:56 +02:00
buengese ce767bc3cf pcloud: implement PublicLink 2020-06-21 17:22:56 +02:00
Caleb Case e780cda1d4 backend/tardigrade: Upgrade to uplink v1.1.1
This fixes issue #4370 by restoring the correct error response.
2020-06-20 16:44:06 +01:00
Nick Craig-Wood a55d882b7b webdav: Fix free/used display for rclone about/df for certain backends - fixes #4348
Before this change if the server sent us xml like this

```
<D:propstat>
<D:prop>
<g0:quota-available-bytes/>
<g0:quota-used-bytes/>
</D:prop>
<D:status>HTTP/1.1 404 Not Found</D:status>
</D:propstat>
```

Rclone would read the empty XML items as containing 0

After this fix we make sure that we have a value before using it.
2020-06-20 15:15:15 +01:00
Nick Craig-Wood 5c5ad62208 drive: fix --drive-impersonate with cached root_folder_id
Before this fix rclone v1.51 and 1.52 would incorrectly use the cached
root_folder_id when the --drive-impersonate flag was in use. This
meant that rclone could be looking up the wrong directory ID with
unpredictable results - usually all files apparently being missing.

This fix makes rclone look up the root_folder_id always when using
--drive-impersonate. It does this by clearing the root_folder_id and
making a NOTICE message that it is ignoring the cached value.

It also stops rclone caching the root_folder_id when using
--drive-impersonate.

See: https://forum.rclone.org/t/rclone-gdrive-no-longer-returning-anything/17215
2020-06-20 15:01:37 +01:00
Nick Craig-Wood 62a1a561cf build: test-repeat.sh add -tag to buildflags 2020-06-20 14:52:04 +01:00
Nick Craig-Wood ce394426b0 check: fix successful retries with --download counting errors
See: https://forum.rclone.org/t/tons-of-data-corruption-after-rclone-copy-to-mega-can-rclone-correct-it/17017
2020-06-20 14:52:04 +01:00
buengese 6606602f1e docs/box: add some info regarding the CleanUp implementation 2020-06-18 23:39:59 +02:00
buengese b6b8958fb4 box: implement CleanUp - fixes #4326 2020-06-18 23:39:59 +02:00
Nick Craig-Wood d8eea0e397 build: run gofmt -s to simplify the code: suggested by Go Report Card 2020-06-18 18:45:39 +01:00