community
/
rclone
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
rclone/backend
History
Nick Craig-Wood 83f61a9cfb s3: GCS provider: fix server side copy of files bigger than 5G 
GCS gives NotImplemented errors for multi-part server side copies. The
threshold for these is currently set just below 5G so any files bigger
than 5G that rclone attempts to server side copy will fail.

This patch works around the problem by adding a quirk for GCS raising
--s3-copy-cutoff to the maximum. This means that rclone will never use
multi-part copies for files in GCS. This includes files bigger than
5GB which (according to AWS documentation) must be copied with
multi-part copy. However this seems to work with GCS.

See: https://forum.rclone.org/t/chunker-uploads-to-gcs-s3-fail-if-the-chunk-size-is-greater-than-the-max-part-size/44349/
See: https://issuetracker.google.com/issues/323465186
 		2024-02-08 14:53:30 +00:00
..
alias build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
all amazonclouddrive: remove Amazon Drive backend code and docs #7539 2024-01-04 17:05:54 +00:00
azureblob azureblob: fix data corruption bug #7590 2024-01-24 11:28:05 +00:00
azurefiles azure-files: fix storage base url 2023-12-18 14:15:13 +00:00
b2 b2: clarify exactly what --b2-download-auth-duration does in the docs 2024-02-08 09:39:53 +00:00
box backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
cache cache: fix integration tests by adding missing OpenChunkWriter exclude 2023-09-04 19:26:14 +01:00
chunker chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2023-12-10 22:29:57 +00:00
combine build: fix lint errors when re-enabling revive exported & package-comments 2023-08-29 13:03:13 +01:00
compress chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2023-12-10 22:29:57 +00:00
crypt chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2023-12-10 22:29:57 +00:00
drive drive: fix integration tests by enabling metadata support from the context 2023-11-19 12:48:27 +00:00
dropbox dropbox: fix used space on dropbox team accounts 2023-12-08 14:26:46 +00:00
fichier fichier: fix error code parsing 2023-08-11 00:47:01 +09:00
filefabric filefabric: use atomic types 2023-08-22 12:52:13 +01:00
ftp ftp: fix mkdir with rsftp which is returning the wrong code 2024-02-07 22:09:56 +00:00
googlecloudstorage gcs: fix 400 Bad request errors when using multi-thread copy 2023-11-20 18:07:05 +00:00
googlephotos googlephotos: fix nil pointer exception when batch failed 2024-01-03 10:57:59 +00:00
hasher hasher: fix invalid memory address error when MaxAge == 0 2024-01-02 18:14:01 +00:00
hdfs hdfs: added support for list of namenodes in hdfs remote config 2023-11-13 15:55:52 +00:00
hidrive backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
http http: implement set backend command to update running backend 2023-11-20 18:07:05 +00:00
imagekit imagekit: Added ImageKit backend 2023-11-24 18:18:01 +00:00
internetarchive fs: implement --metadata-mapper to transform metatadata with a user supplied program 2023-11-18 17:49:35 +00:00
jottacloud backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
koofr rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
linkbox linkbox: pre-merge fixes 2023-11-18 17:14:45 +00:00
local local: fix cleanRootPath on Windows after go1.21.4 stdlib update 2024-01-20 14:50:08 -05:00
mailru rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
mega docs: fix typos found by codespell in docs and code comments 2023-09-23 12:20:01 +01:00
memory Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
netstorage rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
onedrive onedrive: fix "unauthenticated: Unauthenticated" errors when uploading 2024-01-07 11:14:08 +00:00
opendrive opendrive: fix List on a just deleted and remade directory 2023-09-05 17:59:03 +01:00
oracleobjectstorage oracleobjectstorage: Support "backend restore" command - fixes #7371 2024-01-09 09:43:36 +00:00
pcloud backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
pikpak docs: fix typos found by codespell in docs and code comments 2023-09-23 12:20:01 +01:00
premiumizeme backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
protondrive protondrive: make cached keys rclone style and not show with `rclone config redacted` 2023-09-11 15:57:08 +01:00
putio rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
qingstor rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
quatrix quatrix: overwrite files on conflict during server-side move 2023-11-16 17:14:00 +00:00
s3 s3: GCS provider: fix server side copy of files bigger than 5G 2024-02-08 14:53:30 +00:00
seafile Seafile: Fix download/upload error when FILE_SERVER_ROOT is relative 2024-02-05 11:48:51 +00:00
sftp ssh: shorten wait delay for external ssh binaries now that we are using go1.20 2024-01-15 16:22:07 +00:00
sharefile backend: fs implements the `Shutdowner` interface 2023-12-09 11:44:50 +00:00
sia rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
smb smb: fix shares not listed by updating go-smb2 2023-11-29 15:39:27 +00:00
storj storj: update storj.io/uplink to v1.12.0 2023-09-14 14:01:35 +01:00
sugarsync rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
swift swift: fix HEADing 0-length objects when --swift-no-large-objects set 2023-08-03 08:38:39 +01:00
union chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2023-12-10 22:29:57 +00:00
uptobox rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
webdav owncloud: add config `owncloud_exclude_shares` which allows to exclude shared files and folders when listing remote resources 2024-01-31 14:47:24 +00:00
yandex yandex: fix 400 Bad Request on transfer failure 2023-06-11 11:11:43 +01:00
zoho zoho: remove Range requests workarounds to fix integration tests 2023-09-05 18:21:15 +01:00