community
/
rclone
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,162 Commits 2 Branches 109 Tags 198 MiB
Commit Graph

3162 Commits

Author SHA1 Message Date
Nick Craig-Wood 1fefa6adfd vendor: add github.com/mitchellh/go-homedir 2019-05-11 23:39:04 +01:00
Nick Craig-Wood af030f74f5 vfs: make WriteAt for non cached files work with non-sequential writes 
This makes WriteAt for non cached files wait a short time if it gets
an out of order write (which would normally cause an error) to see if
the gap will be filled with an in order write.

The makes the SFTP backend work fine with --vfs-cache-mode off
 2019-05-11 23:39:04 +01:00
Nick Craig-Wood ada8c22a97 sftp: send custom client version and debug server version 2019-05-11 23:39:04 +01:00
Nick Craig-Wood 610466c18c sftp: fix about parsing of df results so it can cope with -ve results 
This is useful when interacting with "serve sftp" which returns -ve
results when the corresponding value is unknown.
 2019-05-11 23:39:04 +01:00
Nick Craig-Wood 9950bb9b7c about: fix crash if backend returns a nil usage 2019-05-11 23:39:04 +01:00
Nick Craig-Wood 7d70e92664 operations: enable multi threaded downloads - Fixes #2252 
This implements the --multi-thread-cutoff and --multi-thread-streams
flags to control multi thread downloading to the local backend.
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood 687cbf3ded operations: if --ignore-checksum is in effect, don't calculate checksum 
Before this change we calculated the checksum which is potentially
time consuming and then ignored the result.  After the change we don't
calculate the checksum if we are about to ignore it.
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood c3af0a1eca local: only calculate the required hashes for big speedup 
Before this change we calculated all possible hashes for the file when
the `Hashes` method was called.

After we only calculate the Hash requested.

Almost all uses of `Hash` just need one checksum.  This will slow down
`rclone lsjson` with the `--hash` flag.  Perhaps lsjson should have a
`--hash-type` flag.

However it will speed up sync/copy/move/check/md5sum/sha1sum etc.

Before it took 12.4 seconds to md5sum a 1GB file, after it takes 3.1
seconds which is the same time the md5sum utility takes.
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood 822483aac5 accounting: enable accounting without passing through the stream #2252 
This is in preparation for multithreaded downloads
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood 2eb31a4f1d sync: move Transferring into operations.Copy 
This makes the code more consistent with the operations code setting
the transfer statistics up.
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood 0655738da6 operations: re-work reopen framework so it can take a RangeOption #2252 
This is in preparation for multipart downloads.
 2019-05-11 23:35:19 +01:00
Nick Craig-Wood 7c4fe3eb75 local: define OpenWriterAt interface and test and implement it #2252 
This will enable multipart downloads in future commits
 2019-05-11 23:35:19 +01:00
Stefan Breunig 72721f4c8d copyurl: honor --no-check-certificate 2019-05-11 17:44:58 +01:00
Nick Craig-Wood 0c60c00187 Add Peter Berbec to contributors 2019-05-11 17:40:17 +01:00
Peter Berbec 0d511b7878 cmd: implement --stats-one-line-date and --stats-one-line-date-format 2019-05-11 17:39:57 +01:00
Nick Craig-Wood bd2a7ffcf4 Add Jeff Quinn to contributors 2019-05-11 17:26:56 +01:00
Nick Craig-Wood 7a5ee968e7 Add Jon to contributors 2019-05-11 17:26:56 +01:00
Jeff Quinn c809334b3d ftp: add FTP List timeout, fixes #3086 
The timeout is controlled by the --timeout flag
 2019-05-11 17:26:23 +01:00
Animosity022 b88e50cc36 docs: Typo fixes with "a existing" 
Fixed a typo with "a existing" to "an existing"
 2019-05-11 16:49:48 +01:00
Jon bbe28df800 docs: Fix typo: Dump HTTP bodies -> Dump HTTP headers 2019-05-11 16:40:40 +01:00
calistri f865280afa Adds a public IP flag for ftp. Closes #3158 
Fixed variable names
 2019-05-09 22:52:21 +01:00
Nick Craig-Wood 8beab1aaf2 build: more pre go1.8 workarounds removed 2019-05-08 15:14:51 +01:00
Fionera b9e16b36e5 Fix Multipart upload check 
In the Documentation it states:
// If (opts.MultipartParams or opts.MultipartContentName) and
// opts.Body are set then CallJSON will do a multipart upload with a
// file attached.
 2019-05-02 16:22:17 +01:00
Nick Craig-Wood b68c3ce74d s3: suppport S3 Accelerated endpoints with --s3-use-accelerate-endpoint 
Fixes #3123
 2019-05-02 14:00:00 +01:00
Fabian Möller d04b0b856a fserrors: use errors.Walk for the wrapped error types 2019-05-01 16:56:08 +01:00
Gary Kim d0ff07bdb0 mega: add cleanup support 
Fixes #3138
 2019-05-01 16:32:34 +01:00
Nick Craig-Wood 577fda059d rc: fix race in tests 2019-05-01 16:09:50 +01:00
Nick Craig-Wood 49d2ab512d test_all: run restic integration tests against local backend 2019-05-01 16:09:50 +01:00
Nick Craig-Wood 9df322e889 tests: make test servers choose a random port to make more reliable 
Tests have been randomly failing with messages like

    listen tcp 127.0.0.1:51778: bind: address already in use

Rework all the test servers so they choose a random free port on
startup and use that for the tests to avoid.
 2019-05-01 16:09:50 +01:00
Nick Craig-Wood 8f89b03d7b vendor: update github.com/t3rm1n4l/go-mega and dependencies 
This is to fix a crash reported in #3140
 2019-05-01 16:09:50 +01:00
Fabian Möller 48c09608ea
fix spelling 2019-04-30 14:12:18 +02:00
Nick Craig-Wood 7963320a29 lib/errors: don't panic on uncomparable errors #3123 
Same fix as c5775cf73d
 2019-04-26 09:56:20 +01:00
Nick Craig-Wood 81f8a5e0d9 Use golangci-lint to check everything 
Now that this issue is fixed: https://github.com/golangci/golangci-lint/issues/204

We can use golangci-lint to check the printfuncs too.
 2019-04-25 15:58:49 +01:00
Nick Craig-Wood 2763598bd1 Add Gary Kim to contributors 2019-04-25 10:51:47 +01:00
Gary Kim 49d7b0d278 sftp: add About support - fixes #3107 
This adds support for using About with SFTP remotes. This works by calling the df command remotely.
 2019-04-25 10:51:15 +01:00
Animosity022 3d475dc0ee mount: Fix poll interval documentation 2019-04-24 18:21:04 +01:00
Fionera 2657d70567 drive: fix move and copy from TeamDrive to GDrive 2019-04-24 18:11:34 +01:00
Nick Craig-Wood 45df37f55f Add Kyle E. Mitchell to contributors 2019-04-24 18:09:40 +01:00
Kyle E. Mitchell 81007c10cb doc: Fix "conververt" typo 2019-04-24 18:09:24 +01:00
Nick Craig-Wood aba15f11d8 cache: note unsupported optional methods 2019-04-16 13:34:06 +01:00
Nick Craig-Wood a57756a05c lsjson, lsf: support showing the Tier of the object 2019-04-16 13:34:06 +01:00
Nick Craig-Wood eeab7a0a43 crypt: Implement Optional methods SetTier, GetTier - fixes #2895 
This implements optional methods on Object
- ID
- SetTier
- GetTier

And declares that it will not implement MimeType for the FsCheck test.
 2019-04-16 13:33:10 +01:00
Nick Craig-Wood ac8d1db8d3 crypt: support PublicLink (rclone link) of underlying backend - fixes #3042 2019-04-16 13:33:10 +01:00
Nick Craig-Wood cd0d43fffb fs: add missing PublicLink to mask 
The enables wrapping file systems to declare that they don't support
PublicLink if the underlying fs doesn't.
 2019-04-16 13:33:10 +01:00
Nick Craig-Wood cdf12b1fc8 crypt: Fix wrapping of ChangeNotify to decrypt directories properly 
Also change the way it is added to make the FsCheckWrap test pass
 2019-04-16 13:33:10 +01:00
Nick Craig-Wood 7981e450a4 crypt: make rclone dedupe work through crypt 
Implement these optional methods:

- WrapFs
- SetWrapper
- MergeDirs
- DirCacheFlush

Fixes #2233 Fixes #2689
 2019-04-16 13:33:10 +01:00
Nick Craig-Wood e7fc3dcd31 fs: copy the ID too when we copy a Directory object 
This means that crypt which wraps directory objects will retain the ID
of the underlying object.
 2019-04-16 13:33:10 +01:00
Nick Craig-Wood 2386c5adc1 hubic: fix tests for optional methods 2019-04-16 13:33:10 +01:00
Nick Craig-Wood 2f21aa86b4 fstest: add tests for coverage of optional methods for wrapping Fs 2019-04-16 13:33:10 +01:00
Nick Craig-Wood 16d8014cbb build: drop support for go1.8 2019-04-15 21:49:58 +01:00