fce2af7934
go fmt 1.1 style
2013-06-27 20:06:44 +01:00
b8c191b51c
swift: Store root for later when we support directories
2013-06-27 19:49:35 +01:00
717a0d268b
Fix after goamz interface change
2013-06-27 19:46:31 +01:00
4cda51b77a
Update after API changes in goamz/s3
2013-02-06 23:31:42 +00:00
4cce906f8b
Implement fs.ListDir / lsd command
2013-01-23 22:43:20 +00:00
8404290499
drive: add full directory listing option for speed
...
* Add -drive-full-list flag to choose between recursive and full directory list
* Full list is the default - much quicker if there are lots of directories
2013-01-23 21:19:26 +00:00
351829e9fe
drive: refactor listDir to take a client function
2013-01-20 11:56:56 +00:00
98108caac2
drive: remove files.Info from FsObjectDrive and copy data out to save memory
2013-01-19 10:16:51 +00:00
f8246b5a7e
Implement Precision interface in all filesystems
...
* drive: add mime type on upload
* drive: correct info on upload to fix crash
* local: measure time precision by making a local file and using Chtimes
* swift: move snet parameter here
* core: add -modify-window and compute the optimum one
* core: use modify window when checking times
2013-01-19 10:16:51 +00:00
b41367856b
Update docs
2013-01-18 18:54:19 +00:00
6e732f3dc0
Implement Purger interface and drive.Purge
2013-01-18 17:11:44 +00:00
a0a789ff0f
Fix after go vet
2013-01-18 17:11:44 +00:00
fdb1ec4d53
Implement Google Drive interface
...
* needs writeup of rather complicated api key setup
* not implemented yet
* fix for slow directory listing
* mimetypes for files
2013-01-18 17:11:22 +00:00
90a397c533
Stop sync creating FsObjects unecessarily when they have been created by List already
2013-01-14 23:08:10 +00:00
8b066ba77c
Don't delete files if errors in copy phase of sync
...
Make check return errors on differences
2013-01-10 22:14:23 +00:00
555ab5001e
Factor Put() methods into generic Copy() function
2013-01-10 21:58:46 +00:00
38ce4c3629
S3: Fix mkdir not to error if bucket exists and implement SetModTime
2013-01-08 22:31:16 +00:00
b9e468ebbd
Add initial S3 support
...
* Requires a patched goamz to enable metadata reading and writing
* Still todo: updating metadata
* EU region not working yet
2013-01-08 18:59:38 +00:00
f7197c30d7
Implement stats counting and reporting and return errors in return code
2013-01-03 22:50:00 +00:00
12015b0007
Run ls in parallel so can parallelise the slow ModTime operations
2013-01-02 15:34:25 +00:00
ecb863dd4f
Make ModTime fall back to LastModified header
...
Which means it no longer needs to report an error which simplifies the code
2013-01-02 15:21:55 +00:00
90a2c86eb3
Implement check and help commands
2012-12-31 17:32:11 +00:00
335667fdcb
Implement sync, -dry-run and fix logging
...
* Implement sync command
* Implement String() interface for Fs
* Sort out logging of FsObject~s
* Implement -dry-run, -verbose and -quiet
2012-12-31 16:40:34 +00:00
c15ae179ee
Detect Fs from path to make operations consistent everywhere
2012-12-29 11:35:41 +00:00
4028a4192f
Use swift.ObjectsWalk
2012-12-28 16:38:51 +00:00
555e545b46
Fix after go vet
2012-12-26 12:27:26 +00:00
3bf6348f57
Factor local filesystem and remote swift into Fs and FsObject interfaces
...
This will enable
* local -> local and remote -> remote copies
* a much more uniform style
* could do s3 as well
2012-12-26 12:23:58 +00:00
12658efef2
gofmt
2012-12-24 00:03:18 +00:00
c3671907cb
Update notes
2012-12-23 17:46:03 +00:00
720d44f8e9
Delete many files at once with purge
2012-12-23 09:49:49 +00:00
a5ecd66388
Remove commented out code
2012-12-23 09:34:29 +00:00
9a066460f1
Make downloads work in parallel
2012-12-23 09:32:33 +00:00
9fb8a1afa2
Implement purge and make upload make the container first
2012-12-04 00:09:22 +00:00
664dbdc9e7
Make the commands data driven
2012-12-03 23:58:17 +00:00
19668ac18f
Fix loss of precision for setting timestamps so roundtrip happens properly
2012-12-01 12:05:17 +00:00
25a1b96537
Make download work properly skipping not changed files
...
Reorganise FsObject and factor more stuff into methods
Note that os.Chtimes() isn't setting the same precision (us) as we are
reading from os.Lstat() (ns) so the modification times don't roundtrip
properly yet.
2012-12-01 10:53:48 +00:00
a3e28e6b97
Fix downloads - missing is the modified check so it downloads everything
2012-12-01 08:18:58 +00:00
e41a8b9a02
First attempt at download
...
Working but modification times are messed up
2012-11-29 23:07:29 +00:00
4d23e29274
Make checking and uploading run in parallel using channels
...
Runs much faster and uses much less memory (thanks to walking the
filesystem into a channel too).
2012-11-29 22:13:58 +00:00
ba24b84396
Use ObjectsAll and ContainersAll to see everything
2012-11-28 23:40:09 +00:00
31745ac13b
Update the mtime of the remote object when size and md5sum match but mtime doesn't
2012-11-28 17:42:12 +00:00
dbb039af9f
Check the MD5SUM of an object if the mtime is different before uploading
2012-11-28 17:24:00 +00:00
5ebbe342b8
Use os.Lstat instead of os.Stat so we ignore symlinks completely
2012-11-28 17:12:54 +00:00
c42bafc4e2
Make logging more consistent
2012-11-28 17:12:22 +00:00
a49b8ca0b6
Use filesystem walk ordering rather than hash order
2012-11-28 11:38:14 +00:00
3e4f75b558
Fix relative vs absolute path confusion in logging
2012-11-28 11:32:04 +00:00
e3096508dd
If the remote object doesn't have mtime then check the md5sum to see if changed
2012-11-28 11:17:31 +00:00
81b7f41dfb
More notes
2012-11-28 10:47:35 +00:00
27ed023c30
Set the mtime when uploading and skip uploads if mtime and size the same
2012-11-27 22:28:46 +00:00
73844ae022
Factor mtime functions info github.com/ncw/swift
2012-11-27 22:04:04 +00:00
Nick Craig-Wood