community
/
rclone
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,353 Commits 2 Branches 109 Tags 198 MiB
Commit Graph

5353 Commits

Author SHA1 Message Date
Nick Craig-Wood fea4b753b2 Add Alex JOST to contributors 2021-02-07 12:30:21 +00:00
Nick Craig-Wood b2b5b7598c build: ensure go1.16 go gets the correct versions of tools 
go1.16 uses GO111MODULE=on by default which meant we were picking up
v1 of nfpm instead of the intended v2.
 2021-02-04 17:34:30 +00:00
Nick Craig-Wood 5f943aabc8 build: Use GO386=softfloat instead of deprecated GO386=387 for 386 builds 2021-02-04 16:37:23 +00:00
Nick Craig-Wood 84c785bc36 build: use go1.16-rc1 as default compiler 2021-02-04 16:08:51 +00:00
Nick Craig-Wood 993146375e build: update all dependencies 2021-02-03 21:34:38 +00:00
Nick Craig-Wood bbe791a886 swift: update github.com/ncw/swift to v2.0.0 
The update to v2 of the swift library introduces a context parameter
to each function. This required a lot of mostly mechanical changes
adding context parameters.

See: https://github.com/ncw/swift/issues/159
See: https://github.com/ncw/swift/issues/161
 2021-02-03 20:23:37 +00:00
Nick Craig-Wood 1545ace8f2 build: remove go1.13 build constraints 2021-02-03 17:46:08 +00:00
Nick Craig-Wood bcac8fdc83 Use http.NewRequestWithContext where possible after go1.13 minimum version 2021-02-03 17:41:27 +00:00
Nick Craig-Wood 15e1a6bee7 build: raise minimum go version to go1.13 2021-02-03 17:41:06 +00:00
Nick Craig-Wood 9710ded60f b2: automatically raise upload cutoff to avoid spurious error 
Before this change, if --b2-chunk-size was raised above 200M then this
error would be produced:

    b2: upload cutoff: 200M is less than chunk size 1G

This change automatically reaises --b2-upload-cutoff to be the value
of --b2-chunk-size if it is below it, which stops this error being
generated.

Fixes #4475
 2021-02-03 16:29:32 +00:00
Nick Craig-Wood 5f3672102c s3: add --s3-no-head to reducing costs docs - Fixes #2163 2021-02-03 16:18:29 +00:00
Nick Craig-Wood 644cc69108 build: update GitHub release tool to use gh and put a link to changelog 
Fixes #4994
 2021-02-03 14:44:40 +00:00
Nick Craig-Wood 1415666074 lsjson: fix unterminated JSON in the presence of errors 
See: https://forum.rclone.org/t/rclone-lsjson-invalid-json-produced-no-at-the-end/22046
 2021-02-02 17:46:56 +00:00
Alex JOST bae550c71e
docs: Changelog: Correct link to digitalis.io 2021-02-02 17:26:41 +00:00
Nick Craig-Wood beff081abb Start v1.55.0-DEV development 2021-02-02 16:30:58 +00:00
Nick Craig-Wood 7f5ee5d81f Version v1.54.0 2021-02-02 14:17:09 +00:00
Nick Craig-Wood 8b41dfa50a s3: add --s3-no-head parameter to minimise transactions on upload 
See: https://forum.rclone.org/t/prevent-head-on-amazon-s3-family/21935
 2021-02-02 10:07:48 +00:00
Nick Craig-Wood 0d8bcc08da Revert "fs/accounting: make edge bandwidth limiters have smaller bursts to make smoother" #4987 
This reverts commit 463a18aa07.

In practice this caused Windows to have a max bandwidth limit of 8
MiB/s. This is because of the limited resolution (1ms) of the Windows
timer.

This needs fixing in a different way.

See: https://forum.rclone.org/t/bug-report-for-v1-54-0-beta-5133/22015
 2021-02-01 22:23:50 +00:00
buengese d3b7f14b66 bump sgzip to v1.1.0 - fixes #4970 2021-02-01 18:34:42 +00:00
Nick Craig-Wood f66928a846 drive: fix copyid command with a bare filename: can't use empty string as a path 
Before this change, running

    rclone backend copyid drive: ID file.txt

Failed with the error

    command "copyid" failed: failed copying "ID" "file.txt": can't use empty string as a path

This fixes the problem.
 2021-02-01 15:17:25 +00:00
Nick Craig-Wood 3b1122c888 azureblob: require go1.14+ to compile due to SDK changes 2021-01-30 18:01:12 +00:00
Nick Craig-Wood 463a18aa07 fs/accounting: make edge bandwidth limiters have smaller bursts to make smoother 
This change decreases the edge limiter burst size which dramatically
increases the smoothness of the bandwidth limiting.

The core bandwidth limiter remains with a large burst so it isn't
affected by double rate limiting on the edge limiters.

See: #4395
See: https://forum.rclone.org/t/bwlimit-is-not-really-smooth/20947
 2021-01-30 17:39:30 +00:00
Nick Craig-Wood 0a932dc1f2 Add --bwlimit for upload and download #1873 2021-01-30 17:39:30 +00:00
Nick Craig-Wood 8856e0e559 fshttp: Obey bwlimit in http Transport - fixes #4395 
This change uses the bwlimit code to apply limits to the receive and
transmit data functions in the HTTP Transport.

This means that all HTTP transactions will have limiting applied -
this includes listings for example.

For HTTP based transorts this makes the limiting in Accounting
redundant and possibly counter productive
 2021-01-30 17:39:30 +00:00
Nick Craig-Wood 3b6df71838 accounting: refactor bwlimit code to allow for multiple slots 2021-01-30 17:39:30 +00:00
Nick Craig-Wood 31de631b22 vendor: run go mod tidy 2021-01-30 17:28:27 +00:00
Nick Craig-Wood 189ef5f257 azureblob: fix memory usage by upgrading the SDK and implementing a TransferManager 
In the Azure SDK there was a bug which caused excessive memory use
when doing repeated transfers:

https://github.com/Azure/azure-storage-blob-go/issues/233

This patch updates the SDK to v0.13.0 which allowed us to implement a
custom TransferManager which integrates with rclone's memory pool.

This fixes the excessive memory consumption.

See: https://forum.rclone.org/t/ask-for-settings-recommendation-for-azureblob/21505/
 2021-01-30 17:26:59 +00:00
Nick Craig-Wood 2f67681e3b cmount: don't attempt to unmount if fs has been destroyed already #4957 2021-01-30 17:19:28 +00:00
Nick Craig-Wood 41127965b0 fstest: add Onedrive Business and Onedrive China to the integration tests 2021-01-30 16:32:32 +00:00
Nick Craig-Wood 8171671d82 Add Bob Pusateri to contributors 2021-01-30 16:24:54 +00:00
Nick Craig-Wood 75617c0c3b Add Pau Rodriguez-Estivill to contributors 2021-01-30 16:24:54 +00:00
Nick Craig-Wood 8b9d23916b Add Nicolas Rueff to contributors 2021-01-30 16:24:54 +00:00
Bob Pusateri e43b79e33d
azureblob: add examples for access tier 
Azure Blob access tier values are case-sensitive, though this is
not indicated anywhere in the documentation. Adding examples
with proper casing.
 2021-01-30 16:21:34 +00:00
albertony 459cc70a50 vfs: fix invalid cache path on windows when using :backend: as remote 
The initial ':' is included in the ad-hoc remote name, but is illegal character
in Windows path. Replacing it with '^', which is legal in filesystems but illegal
in regular remote names, so name conflict is avoided.

Fixes #4544
 2021-01-30 16:18:15 +00:00
Ivan Andreev 20578f3f89 Add Sakuragawa Misty to contributors 2021-01-29 23:05:49 +03:00
NyaMisty 15da53696e
onedrive: add support for china region operated by 21vianet #4963 (#4963) 
fixes #3804
obsoletes #3973
obsoletes #4072
 2021-01-29 23:04:21 +03:00
Ivan Andreev 2bddba118e
fstest: apply shellcheck on rclone-serve.bash (#4975) 2021-01-29 19:07:17 +03:00
Ivan Andreev c7e5976e11
build: replace go 1.16-beta1 by 1.16-rc1 (#4974) 2021-01-29 19:05:46 +03:00
Pau Rodriguez-Estivill f0bf9cfda1 drive: add xdg office icons to xdg desktop files 2021-01-28 17:12:48 +00:00
Nguyễn Hữu Luân 671dd047f7
swift: ensure partially uploaded large files are uploaded unless --swift-leave-parts-on-error 
This makes sure that partially uploaded large files are removed
unless the `--swift-leave-parts-on-error` flag is supplied.

- refactor swift.go
- add unit test for swift with chunk
- add unit test for large object with fail case
- add "-" to white list char during encode.
 2021-01-28 17:09:41 +00:00
negative0 6272ca74bc plugins: Move plugins cache path initialization to initPluginsOrError. 
Fixes #4951.
 2021-01-28 16:58:23 +00:00
Nicolas Rueff f5af761466
gphotos: new flag --gphotos-include-archived - fixes #4728 
New flag --gphotos-include-archived to download and view archived media when needed.
 2021-01-28 16:51:31 +00:00
Nick Craig-Wood 06f1c0c61c move: fix data loss when moving the same object 
This checks to see if IDs are the same of the source and destination
object before deleting one of them and potentially causing data loss.

See: https://forum.rclone.org/t/files-deleted-and-not-moved/21830
 2021-01-28 16:14:16 +00:00
Nick Craig-Wood e6a9f005d6 sftp: implement --sftp-use-fstat 
See: https://forum.rclone.org/t/sftp-fails-to-sync-to-local-failed-to-copy-file-does-not-exist/21759
 2021-01-28 16:07:26 +00:00
Nick Craig-Wood 8f6f4b053c obscure: make `rclone osbcure -` ignore newline at end of line 
See: https://forum.rclone.org/t/authentification-issues-with-webdav-server/21891
 2021-01-28 15:54:41 +00:00
Nick Craig-Wood fe15a2eeeb Add Riccardo Iaconelli to contributors 2021-01-28 15:54:41 +00:00
Nick Craig-Wood 019667170f Add Zach Kipp to contributors 2021-01-28 15:54:41 +00:00
albertony 7a496752f3 fs: add support for flag --no-console on windows to hide the console window 2021-01-27 18:44:35 +00:00
Yury Stankevich b569dc11a0 hdfs: support kerberos authentication #42 2021-01-27 18:16:58 +00:00
Riccardo Iaconelli df4e6079f1 local: new flag --local-zero-size-links to fix sync on some virtual filesystems 
Assume the Stat size of links is zero (and read them instead)

On some virtual filesystems (such ash LucidLink), reading a link size via a
Stat call always returns 0.
However, on unix it reads as the length of the text in the link. This may
cause errors like this when syncing:

    Failed to copy: corrupted on transfer: sizes differ 0 vs 13

Setting this flag causes rclone to read the link and use that as the size of
the link instead of 0 which in most cases fixes the problem.

Fixes #4950

Signed-off-by: Riccardo Iaconelli <riccardo@kde.org>
 2021-01-27 18:13:16 +00:00