community
/
rclone
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
rclone/backend
History
Maxwell Calman 9cc8ff4dd4 chunker: partially implement no-rename transactions (#4675) 
Some storage providers e.g. S3 don't have an efficient rename operation.
Before this change, when chunker finished an upload, the server-side copy
and delete operations that renamed temporary chunks to their final names
could take a significant amount of time.
This PR records transaction identifier (versioning) in the metadata of
chunker composite objects striving to remove the need for rename
operations on such backends.
This approach will be triggered be the new "transactions" configuration
option, which can be "rename" (the default) or "norename".
We implement the new approach for uploads (Put operations).
The chunker Move operation still uses the rename operation of
underlying backend. Filling this gap is left for a later PR.

Co-authored-by: Ivan Andreev <ivandeex@gmail.com>
 		2021-02-28 10:49:17 +00:00
..
alias alias: fix tests after parsing of ... change #4862 2020-12-21 18:23:16 +00:00
all HDFS (Hadoop Distributed File System) implementation - #42 2021-01-07 09:48:51 +00:00
amazonclouddrive fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
azureblob azureblob: require go1.14+ to compile due to SDK changes 2021-01-30 18:01:12 +00:00
b2 b2: fix failed to create file system with application key limited to a prefix 2021-02-11 15:13:19 +00:00
box box: fix finding directories in a case insentive way #4830 2021-01-26 14:48:33 +00:00
cache fs: add Shutdown optional method for backends 2020-11-27 17:35:01 +00:00
chunker chunker: partially implement no-rename transactions (#4675) 2021-02-28 10:49:17 +00:00
compress compress: check type assertion in SetTier - fixes #4941 2021-01-20 22:59:14 +00:00
crypt crypt: add option to not encrypt data #1077 (#2981) 2021-02-17 03:40:37 +03:00
drive drive: refer to Shared Drives instead of Team Drives 2021-02-07 12:30:21 +00:00
dropbox dropbox: add polling support - fixes #2949 2021-02-24 09:33:31 +00:00
fichier fichier: implement copy & move 2021-02-24 21:05:41 +01:00
filefabric filefabric: fix finding directories in a case insensitive way #4830 2021-01-27 14:28:17 +00:00
ftp ftp: close idle connections after --ftp-idle-timeout (1m by default) 2021-02-16 12:39:05 +00:00
googlecloudstorage Use http.NewRequestWithContext where possible after go1.13 minimum version 2021-02-03 17:41:27 +00:00
googlephotos gphotos: new flag --gphotos-include-archived - fixes #4728 2021-01-28 16:51:31 +00:00
hdfs hdfs: support kerberos authentication #42 2021-01-27 18:16:58 +00:00
http Use http.NewRequestWithContext where possible after go1.13 minimum version 2021-02-03 17:41:27 +00:00
hubic swift: update github.com/ncw/swift to v2.0.0 2021-02-03 20:23:37 +00:00
jottacloud jottacloud: Add support for Telia Cloud (#4930) 2021-01-17 02:38:57 +01:00
koofr fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
local local: Add flag --no-preallocate - #3207 2021-02-11 17:25:28 +00:00
mailru fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
mega fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
memory azureblob,memory,pcloud: fix setting of mime types 2020-11-27 14:40:05 +00:00
onedrive onedrive: add support for china region operated by 21vianet #4963 (#4963) 2021-01-29 23:04:21 +03:00
opendrive opendrive: fix finding directories in a case insensitive way #4830 2021-01-27 14:28:17 +00:00
pcloud azureblob,memory,pcloud: fix setting of mime types 2020-11-27 14:40:05 +00:00
premiumizeme premiumizeme: fix finding directories in a case insensitive way #4830 2021-01-27 14:28:17 +00:00
putio Use http.NewRequestWithContext where possible after go1.13 minimum version 2021-02-03 17:41:27 +00:00
qingstor qingstor: fix rclone cleanup 2021-01-21 17:35:31 +00:00
s3 s3: fix failed to create file system with folder level permissions policy 2021-02-24 20:35:44 +00:00
seafile fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
sftp sftp: close idle connections after --sftp-idle-timeout (1m by default) 2021-02-16 12:39:05 +00:00
sharefile fs: deglobalise the config #4685 2020-11-26 16:40:12 +00:00
sugarsync sugarsync: fix finding directories in a case insentive way #4830 2021-01-26 14:48:33 +00:00
swift swift: update github.com/ncw/swift to v2.0.0 2021-02-03 20:23:37 +00:00
tardigrade build: remove go1.13 build constraints 2021-02-03 17:46:08 +00:00
union union: fix mkdir at root with remote:/ 2021-02-17 12:11:34 +00:00
webdav webdav: check that purged directory really exists (#2921) 2021-02-23 23:27:30 +00:00
yandex yandex: set Features.WriteMimeType=false as Yandex ignores mime types 2020-11-29 17:22:43 +00:00
zoho zoho: fix custom client id's 2021-02-23 11:27:05 +00:00