Commit Graph

1289 Commits

Author SHA1 Message Date
Nick Craig-Wood a288c2b3a3 Make a retry error wrapper for a plain error 2015-09-09 23:22:41 +01:00
Nick Craig-Wood fec16b0ac8 acd: Skip test on FS which don't support ModifiedTime 2015-09-09 23:21:50 +01:00
Nick Craig-Wood dd8717797e Implement --dump-headers and --dump-bodies debug flags 2015-09-08 21:02:48 +01:00
Nick Craig-Wood 8c3df224ef Implement Amazon Cloud Drive - fixes #45
* Optional interfaces Copier, Mover, DirMover not done
2015-09-08 21:02:48 +01:00
Nick Craig-Wood c98a51b26c Lightly obscure secrets 2015-09-01 22:33:34 +01:00
Nick Craig-Wood 8d33ce0154 Check for source and dest being the same in sync/copy/move 2015-09-01 21:49:13 +01:00
Nick Craig-Wood 92745aa950 Add Root() to Fs interface 2015-09-01 21:49:13 +01:00
Nick Craig-Wood 0168f55f3e Switch to spf13 fork of pflag - fixes #116
This supports --long value as well as --long=value which is as
expected for a unix utility.
2015-08-29 18:14:24 +01:00
Nick Craig-Wood 9ed2de3d6e Version v1.19 2015-08-28 09:47:13 +01:00
Nick Craig-Wood 59ba8f28c8 Implement move command - fixes #35
* Define Mover interface to move a single object
  * Define DirMover interface to move a directory
  * Implement DirMove operation
  * Add `rclone move` command
  * Tests for Dir Move

To Do
  * Implement Move, DirMover in local, drive, dropbox
  * unit test for Mover
  * unit test for DirMover
2015-08-28 08:49:16 +01:00
Nick Craig-Wood 5c226e91c0 Ignore rmdir in limited fs rather than throwing error - fixes #112 2015-08-25 19:16:25 +01:00
Nick Craig-Wood a96b522958 Implement server side copies if possible - fixes #99
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Nick Craig-Wood 0c6f816a49 Implement --retries flag - fixes #109 2015-08-20 21:07:00 +01:00
Nick Craig-Wood dfe771fb0c Correct log messages for remotes which don't support modtime/md5sum 2015-08-20 20:48:58 +01:00
Nick Craig-Wood 2fcc18779b Version v1.18 2015-08-17 17:59:37 +01:00
Nick Craig-Wood 1d9e76bb0f dropbox: remove datastore - Fixes #55 #84
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
2015-08-17 17:26:36 +01:00
Nick Craig-Wood 771e60bd07 Show errors when reading the config file 2015-08-15 17:15:02 +01:00
Leonid Shalupov 3fcff32524 do not print stats in quiet mode - fixes #70
...unless had some errors or stats interval requested.

Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood 9d9a17547a Version v1.17 2015-06-14 15:36:16 +01:00
Leonid Shalupov bd5f685d0a fix TestLsLong on non-UTC timezones 2015-06-14 15:23:56 +01:00
Nick Craig-Wood 539ad44757 Version v1.16 2015-06-09 18:00:33 +01:00
Nick Craig-Wood 74994a2ec1 Fix uploading big files which was causing timeouts or panics
The symtom was one of these two on upload of files only
  * panic: d.nx != 0 in crypto/md5.(*digest).checkSum
  * read tcp: i/o timeout

It turned out to be a combination of two upstream bugs

  * 5a2187309e
  * https://groups.google.com/forum/#!topic/golang-dev/0Nl6k5Sj6UU

This commit contains a work-around for the second problem, I've fixed
the first and had the change accepted upstream.
2015-06-09 17:32:45 +01:00
Nick Craig-Wood 97dced6a0b Don't check md5sum after download with --size-only - fixes #75 2015-06-09 13:18:40 +01:00
Nick Craig-Wood e04acb09ce Version v1.15 2015-06-06 15:45:00 +01:00
Nick Craig-Wood 90744301d3 Fix package docs so they appear in godoc correctly 2015-06-06 14:24:30 +01:00
Nick Craig-Wood e22b445cff Implement --size-only flag to sync on size not checksum & modtime - fixes #75 2015-06-06 08:49:01 +01:00
Alex Couper 7af1282375 Add --checksum flag to only discard transfers by MD5SUM - fixes #61
Useful for copying between backends where checksum fetching is fast,
ie any of s3, swift, drive or googlecloudstorage.
2015-06-05 19:46:03 +01:00
Nick Craig-Wood d9fcc32f70 Version v1.14 2015-05-21 20:13:40 +01:00
Nick Craig-Wood 88293a4b8a Version v1.13 2015-05-10 12:39:06 +01:00
Nick Craig-Wood 1d254a3674 Implement --timeout and --conntimeout - fixes #54
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood ba2091725e Version v1.12 2015-03-15 15:55:38 +00:00
Nick Craig-Wood 5cc5429f99 drive: switch to insert from update when a failed copy deletes the upload 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 09d71239b6 Make file size render more neatly and prevent from being < 0 2015-03-15 15:27:55 +00:00
Nick Craig-Wood c643e4585e core: Log duplicate files if they are detected 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 873db29391 Log all objects more informatively 2015-03-15 15:27:55 +00:00
Nick Craig-Wood ecb3c7bcef drive, googlecloudstorage: remove SeekWrapper after googleapi fix 2015-03-04 20:47:59 +00:00
Nick Craig-Wood 80000b904c Version v1.11 2015-03-04 17:59:31 +00:00
Nick Craig-Wood b4a0941d4c In remote paths, change native directory separators to / - fixes #37 2015-03-02 17:04:34 +00:00
Nick Craig-Wood 46d39ebaf7 Factor Mime Type guessing into fs.MimeType() 2015-03-02 09:21:15 +00:00
Nick Craig-Wood fe68737268 Fix niggles found by go vet 2015-02-28 15:35:54 +00:00
Nick Craig-Wood 2360bf907a Add synchronization to list output to stop corruptions - fixes #29 2015-02-28 15:30:40 +00:00
Nick Craig-Wood a287e3ced7 Implement --bwlimit to limit data transfer bandwidth 2015-02-27 15:03:47 +00:00
Nick Craig-Wood cf208ad21b Version v1.10 2015-02-12 18:00:20 +00:00
Nick Craig-Wood 8d1c0ad07c Fix config loop - fixes #25 2015-02-10 16:48:04 +00:00
Nick Craig-Wood 165e89c266 Version v1.09 2015-02-07 22:44:23 +00:00
Nick Craig-Wood b4e19cfd62 windows: make tests work properly 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 20ad96f3cd windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:)
This was done by stopping the user configuring single letter remotes
and making sure we don't treat single letter remotes as a remote name,
but as a drive letter.
2015-02-07 22:32:51 +00:00
Nick Craig-Wood 5fb6f94579 drive: fix rate limit exceeded errors - fixes #20
This is done by pacing the requests to drive and backing them off
using an exponential increase.  Put and Modify operations can now be
retried also.
2015-02-07 22:32:51 +00:00
Nick Craig-Wood 18a8a61cc5 Release v1.08 2015-02-04 22:31:56 +00:00
Nick Craig-Wood cfdb48c864 Version v1.07 2014-12-23 11:26:32 +00:00
Nick Craig-Wood 14567952b3 google cloud storage: Fix memory leak - fixes #17
This was the same problem as issue #5 (which affected google drive)
2014-12-23 11:03:34 +00:00
Nick Craig-Wood 439a126af6 Version v1.06 2014-12-12 20:13:03 +00:00
Nick Craig-Wood af9c447146 Fix "Couldn't find home directory" on OSX - fixes #15 2014-12-12 19:18:23 +00:00
Nick Craig-Wood 839133c5e1 Version v1.05 2014-08-09 17:22:17 +01:00
Nick Craig-Wood 18439cf2d7 Move rclonetest into go tests for fs module 2014-08-03 11:18:25 +01:00
Nick Craig-Wood 3e27ff1b95 Add Root List test and fs.Limited tests for single files 2014-07-31 21:35:29 +01:00
Nick Craig-Wood ff91698fb5 Skip tests if test remote not configured 2014-07-31 08:51:39 +01:00
Nick Craig-Wood 2f9f9afac2 fs: Document that Purger returns error on empty directory, test and fix 2014-07-29 17:18:22 +01:00
Nick Craig-Wood 17ffb0855f Fixes after running errcheck 2014-07-25 18:19:49 +01:00
Nick Craig-Wood a5b0d88608 Make tests for each Fs
Factor tests out of rclonetest
2014-07-24 23:13:32 +01:00
Nick Craig-Wood e09ef62d5b core: Fix race detected by go race detector 2014-07-22 23:03:14 +01:00
Nick Craig-Wood a75bc0703f Version 1.04 2014-07-21 21:32:37 +01:00
Nick Craig-Wood 54cd46372a Version 1.03 2014-07-20 11:28:50 +01:00
Nick Craig-Wood 9aa4b6bd9b Version 1.02 2014-07-19 13:24:48 +01:00
Nick Craig-Wood e559194fb2 fs: Verify sizes are the same after transfer in Copy() 2014-07-19 13:05:07 +01:00
Nick Craig-Wood 8a76568ea8 core: Verify MD5 sums after each transfer 2014-07-15 19:27:05 +01:00
Nick Craig-Wood c1a245d1c8 Factor UserAgent to fs and move Version to fs 2014-07-13 19:19:58 +01:00
Nick Craig-Wood eb2b4ea8aa rclone: Don't purge if --dry-run set 2014-07-13 10:54:30 +01:00
Nick Craig-Wood e055ed0489 rclone: change "ls" and add "lsl" and "md5sum" commands
Changed "ls" command not to show modification time by default only
size and path.  That is because it is slow for nearly all the remotes
as it requires extra metadata lookup.  All remotes can look up files
and sizes without extra operations.

Added "lsl" which does what "ls" used to - namely show modification
time, size and path.

Added "md5sum" which produces the same output as the md5sum command -
md5sums and paths that is.
2014-07-12 12:09:20 +01:00
Nick Craig-Wood 7ebf48ef42 Fix --dry-run not working and add tests for it - fixes #3 2014-06-26 15:33:06 +01:00
Nick Craig-Wood 1d67b014cb Make compatible with go 1.1 - fixes #1 2014-06-26 15:18:48 +01:00
Nick Craig-Wood 99695d57ab Implement single file operations for all file systems 2014-05-05 22:17:57 +01:00
Nick Craig-Wood 8fd59f2e7d drive: Use o.Update and fs.Put to optimise transfers 2014-04-18 17:49:01 +01:00
Nick Craig-Wood 02afcb00e9 Factor Fs.Put into Object.Update and call Update rather than Put if possible 2014-04-18 17:49:01 +01:00
Nick Craig-Wood d6a5bfe2d4 Get rid of fs.CopyFs and replace with fs.Sync in preparation for Object.Update 2014-04-18 17:48:46 +01:00
Nick Craig-Wood 92ec29fe3f Factor the generic code into fs and add some more intefaces 2014-03-28 22:26:42 +00:00
Nick Craig-Wood b0064e6405 Change remote format one last time 2014-03-27 17:31:57 +00:00
Nick Craig-Wood 3eda36f6da Factor rclone main() in preparation for tests 2014-03-27 17:26:24 +00:00
Nick Craig-Wood a628bef9c2 Use pflag for gnu style flags 2014-03-27 16:55:29 +00:00
Nick Craig-Wood 1b3a49929b Add ability for FS to have a Config helper function run 2014-03-16 13:54:43 +00:00
Nick Craig-Wood 1ea9972be7 Set strict permissions on the config file 2014-03-16 13:53:51 +00:00
Nick Craig-Wood 7d786204b4 Add -config option to specify a config file 2014-03-15 17:01:13 +00:00
Nick Craig-Wood b3f1a45bbf Config fixes
* Fix empty config configuration
  * Alter menus when no remotes
  * Save config file after delete remote
2014-03-15 16:52:51 +00:00
Nick Craig-Wood 0a108832e2 Remove filesystem flags and put in config file with editor 2014-03-15 16:06:11 +00:00
Nick Craig-Wood 754c54b64f Fix differences print out on check 2013-07-02 08:33:17 +01:00
Nick Craig-Wood 43be26d1f8 Make local filesystem go last 2013-06-29 12:15:55 +01:00
Nick Craig-Wood b1f681b912 Rename fs.FsX to fs.X 2013-06-28 08:57:32 +01:00
Nick Craig-Wood 6085dc1b5a Refactor into sub filesystems 2013-06-27 22:17:51 +01:00