Commit Graph

996 Commits

Author SHA1 Message Date
Nick Craig-Wood fa500e6d21 lib/atexit: factor from cmd so it can be used by backend/cache #1946 2018-01-25 10:33:00 +00:00
Jody Frankowski 5e73acd40a Clean up mount.go and vfs/help.go docs
* Title cleanups
* Typos
* `rclone mount vs rclone sync/copy` update with `File Caching`
2018-01-20 18:14:20 +00:00
Nick Craig-Wood a80d8a21dc vfs: add flags parameter to Dir.Create 2018-01-19 13:18:40 +00:00
Nick Craig-Wood 517bdc719b vfs: make specialized file Open functions private 2018-01-19 11:46:01 +00:00
Nick Craig-Wood 5ad226ab54 fs: Add dir option to fs.Purge #1891
Purge optional interface needs fixing too.
2018-01-19 11:45:50 +00:00
Nick Craig-Wood 97c414f025 config/hash: rename more symbols after factoring into own package 2018-01-18 20:27:52 +00:00
Nick Craig-Wood 71722b5b95 config: factor Obscure and Reveal into its own package 2018-01-18 20:19:55 +00:00
Nick Craig-Wood 821be5ebed ncdu: add link to asciinema demo of it in action 2018-01-18 14:22:43 +00:00
Ernest Borowski 5cce74d630 flags: remove --no-traverse flag because it is obsolete - fixes #1813
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
2018-01-18 11:00:25 +00:00
Nick Craig-Wood 11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood 60afda007b Move dircache, oauthutil, rest and pacer modules into lib 2018-01-12 17:07:38 +00:00
Nick Craig-Wood b8b620f5c2 Move all backends into backend directory 2018-01-12 17:07:38 +00:00
ishuah 0a7731cf0d cryptdecode: added option to return encrypted file names. Fixes #1923 2018-01-11 19:22:40 +03:00
Nick Craig-Wood 0eba37d8f3 lsf: add --files-only and --dirs-only flags 2018-01-06 18:04:24 +00:00
Nick Craig-Wood c74c3b37da lsf: add option to print hashes 2018-01-06 17:53:37 +00:00
Nick Craig-Wood ed20fa5ee7 ls* commands: update docs and add defaults into options for lsf 2018-01-06 17:00:20 +00:00
Nick Craig-Wood 54a9fdf421 ls2: remove in favour of lsf 2018-01-06 14:41:36 +00:00
Jakub Tasiemski 0d041602cf cmd: new command lsf 2018-01-06 14:39:31 +00:00
Nick Craig-Wood 65618afd8c serve/http: fix serving files with : in - fixes #1939 2018-01-05 17:25:05 +00:00
Nick Craig-Wood 2a01fa9fa0 moveto,copyto: clarify error message if source doesn't exist - fixes #1022 2017-12-15 11:37:31 +00:00
lewapm 9c242edc10 rmdirs: add --leave-root flag - fixes #1874 2017-12-13 10:23:54 +00:00
Nick Craig-Wood 2cf808c825 ncdu: fix crashes on empty directories - fixes #1910
Up arrow or right arrow in an empty directory would crash ncdu
2017-12-12 13:54:15 +00:00
remusb b48b537325
cache: plex integration, refactor chunk storage and worker retries (#1899) 2017-12-09 23:54:26 +02:00
Laurence e767c9ac9f Fix typo in dbhashsum description 2017-12-03 10:24:33 +00:00
Nick Craig-Wood 6d3df6f172 cmount: make tests more reliable on Windows 2017-11-28 20:39:24 +00:00
ishuah aab8051f50 move: add --delete-empty-src-dirs flag - fixes #1854 2017-11-28 11:38:19 +03:00
Nick Craig-Wood 1248beb0b2 cachestats: Fix nil pointer if not a cache remote - fixes #1855
Also don't retry or show stats
2017-11-24 10:22:23 +00:00
Nick Craig-Wood dc1c679c65 mount: support truncate properly 2017-11-20 19:42:35 +00:00
Nick Craig-Wood af50f31f7d mounttest: wait for Release after every Read to stop using in use files under Windows 2017-11-20 12:46:24 +00:00
Nick Craig-Wood 2b666187a6 cmount: disable tests on windows + race detector
These either hang or produce incorrect results for reasons I haven't
worked out yet.
2017-11-20 12:46:24 +00:00
Nick Craig-Wood 00b46a8b96 mounttest: wait for files to disappear from the directory listing 2017-11-20 12:46:24 +00:00
Nick Craig-Wood b21f227bd3 mounttest: fix crash when FUSE not present 2017-11-20 12:46:24 +00:00
Nick Craig-Wood e98e550021 mounttest: wait for all background Close/Release after writing a file
The filesystem does a certain amount of things asynchronously waiting
for the file to be released after writing it means everything should
be in a consistent state.
2017-11-20 12:46:23 +00:00
Nick Craig-Wood b4083b4371 vfs: rename Fsync to Sync and implement Sync on Node and Handle 2017-11-20 12:46:23 +00:00
Nick Craig-Wood eb3415db50 cmount: enable more tests for Windows 2017-11-20 12:46:23 +00:00
Nick Craig-Wood 9fbd8a6419 mounttest: fixes for running under Windows
* don't mount and unmount between cache runs - WinFSP doesn't suport it
  * use OS paths for opening things
2017-11-20 12:46:23 +00:00
Nick Craig-Wood 61b6159a05 mount, cmount: add O_CREATE to Open calls since fuse doesn't seem to supply it 2017-11-20 12:46:22 +00:00
Nick Craig-Wood c560017934 vfs: add Path method to Node and use it to stop reading nil DirEntry
All DirEntry calls now have been checked for nil or converted to use Path.
2017-11-20 12:46:22 +00:00
Nick Craig-Wood 7c3584f4e6 mountlib: wait for mountpoint to disappear under Windows 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 981cfb1bec mounttest: retry directory listings to account for slow updates on Windows 2017-11-20 12:46:22 +00:00
Nick Craig-Wood dec21ccf63 vfs, cmount: make truncate work properly in the presence or otherwise of open files 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 94adf4f43b cmount: translate FUSE open flags into OS flags
On Windows the fuse.O_* flags do not have the same values as the
os.O_* flags so translate between the two representations.  They are
mostly the same which is why this hasn't caused a problem before.
2017-11-20 12:46:22 +00:00
Nick Craig-Wood f5f8c0c438 cmount: make Truncate call the correct Handle or Node method 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 60cdcf784c cmount: use -o atomic_o_trunc to make sure O_TRUNC is supplied to Open() 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 57a5c67729 mounttest: run the tests for all 4 VFS cache modes 2017-11-20 12:46:21 +00:00
Nick Craig-Wood d7908c06c9 mountlib: ensure we don't open files with read and write intent 2017-11-20 12:46:21 +00:00
Nick Craig-Wood 8951875c21 vfs,mount,cmount,mountlib: allow flags to be overriden by environment variables 2017-11-20 12:46:21 +00:00
Nick Craig-Wood 05a1e1532b vfs,mount,cmount,serve: Add documentation for vfs caching modes 2017-11-20 12:46:21 +00:00
Nick Craig-Wood 3fb1e96988 vfs: factor Open logic from Dir.Create into vfs.OpenFile 2017-11-20 12:00:32 +00:00
Jakub Tasiemski 3cf7c61aa0 Add touch command - fixes #1594 2017-11-20 11:16:05 +00:00
Fabian Möller d8b3bf014d mount: use sdnotify to signal systemd the mount is ready
When the NOTIFY_SOCKET environment variable is set notify systemd after
the mount is ready.
2017-11-20 11:03:10 +00:00
ishuah 31d2fb4e11 mount: Fix mount breaking on Windows - fixes #1827 2017-11-16 15:20:53 +03:00
Remus Bunduc 016abf825e cache: first version 2017-11-15 15:23:21 +00:00
ishuah aec2265be0 rclone: implement exit codes - #1136 2017-11-15 17:48:37 +03:00
Substantiel 2423fa40e2 config: add password sub command for setting obscured passwords 2017-11-15 14:44:45 +00:00
Ernest Borowski 9fbff7bcab mountlib: check if directory is not empty before mounting - fixes #1386
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
2017-11-14 21:24:31 +00:00
Nick Craig-Wood 68d0b5adbb serve webdav: this implements a webdav server for any rclone remote. 2017-11-04 10:24:11 +00:00
Nick Craig-Wood 16e16bc220 serve http: use vfs to cache the directories and support Range header 2017-11-04 10:24:11 +00:00
Nick Craig-Wood 2083ac6e2a vfs: add ECLOSED and tidy errors 2017-11-04 10:24:10 +00:00
Nick Craig-Wood 22ee839d05 cmount,vfs: unify Read and Write handles and File and Dir where possible 2017-11-04 10:24:10 +00:00
Nick Craig-Wood 5634659ea3 mount,vfs: unify Read and Write handles in preparation for ReadWrite handles 2017-11-04 10:24:10 +00:00
Nick Craig-Wood e18122e88b vfs: add tests and subsequent fixes
* Tests for VFS layer
  * Small fixes found during testing
  * Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
  * Fix nil object bugs on File
2017-11-04 10:24:10 +00:00
Nick Craig-Wood 07ec8073fe mount: remove unused DirEntry struct 2017-11-03 13:00:00 +00:00
Nick Craig-Wood 8184ec4b70 vfs: add EPERM to errors 2017-11-03 13:00:00 +00:00
Nick Craig-Wood 190367d917 vfs: factor duplicated Open code into vfs from mount/cmount 2017-11-03 13:00:00 +00:00
Nick Craig-Wood 7e065440fb vfs: rename Lookup to Stat to be more in keeping with os 2017-11-03 12:59:59 +00:00
Nick Craig-Wood e8883e9fdb vfs: factor flags into vfsflags and remove global variables 2017-11-03 12:59:59 +00:00
Nick Craig-Wood 1a8f824bad vfs: use os package errors where possible 2017-11-03 12:59:59 +00:00
Nick Craig-Wood c1aaff220d Factor new vfs module out of cmd/mountlib
This is an OS style file system abstraction with directory caching
used in mount, cmount, serve webdav and serve http.
2017-11-03 12:59:59 +00:00
Nick Craig-Wood 6da6b2556b mountlib: make directory entries be returned in sorted order 2017-11-03 12:59:59 +00:00
Nick Craig-Wood ca19fd2d7e mountlib: Make read/write file handles support more standard interfaces
Including Read, ReadAt, Seek, Close for read handles and Write,
WriteAt, Close for read handles.
2017-11-03 12:59:59 +00:00
Nick Craig-Wood 2fac74b517 mountlib: store only Node in *Dir removing DirEntry struct 2017-11-03 12:59:59 +00:00
Nick Craig-Wood 8b6daaa877 mountlib: add DirEntry() to Node interface 2017-11-03 12:59:59 +00:00
Nick Craig-Wood 3af9d63261 mountlib: add Remove and RemoveAll methods to Node 2017-11-03 12:59:59 +00:00
Nick Craig-Wood c6cd2a5280 mountlib: add parent and entry to Dir 2017-11-03 12:59:59 +00:00
Nick Craig-Wood 0bb84efe75 mountlib: Rename Remove to RemoveName 2017-11-03 12:59:59 +00:00
Nick Craig-Wood 3ec15ac2bd mountlib: make sure Node is always set in DirEntry
This simplifies the code and makes using the DirEntry.Node usable when
using ReadDir.
2017-11-03 12:59:58 +00:00
Nick Craig-Wood 750690503e mountlib: make Node satisfy os.FileInfo interface 2017-11-03 12:59:58 +00:00
Nick Craig-Wood 54950d3423 mountlib: make more useful as a general purpose file system adaptor 2017-11-03 12:59:58 +00:00
Nick Craig-Wood 96665c16cb serve http: make it compile on go1.6 and go1.7 2017-10-26 21:52:29 +01:00
Nick Craig-Wood cd146415d1 serve http: error if Range supplied (not supported yet)
Also add Server header
2017-10-24 23:18:36 +01:00
Nick Craig-Wood 2740c965c0 serve http: Fix timeouts 2017-10-24 23:07:46 +01:00
Nick Craig-Wood 6669165b6b serve http command to serve a remote over HTTP
This implements a basic webserver to serve an rclone remote over HTTP.

It also sets up the framework for adding more types of server later.
2017-10-24 13:25:49 +01:00
Dan Dascalescu 9417194751 Fix dedupe description typo 2017-10-16 21:51:31 +01:00
Nick Craig-Wood edfab09eb9 config: add sub commands for full config file management
Previously config sub commands were manually parsed rather than using
cobra.

Make config command have the following sub commands:

 * create    Create a new remote with name, type and options.
 * delete    Delete an existing remote <name>.
 * dump      Dump the config file as JSON.
 * edit      Enter an interactive configuration session.
 * file      Show path of configuration file in use.
 * providers List in JSON format all the providers and options.
 * show      Print (decrypted) config file, or the config for a single remote.
 * update    Update options in an existing remote.

The following changes were made to existing commands

 * listproviders was renamed to providers
 * listoptions was removed in favour of providing the output in providers
 * jsonconfig was renamed to create
 * an optional parameter was added to the show command
2017-10-14 11:50:41 +01:00
thierry 0575623dff Add config listproviders, listoptions, jsonconfig for automated config
Addition of a method listing the providers, a method listing the
options of a provider and method of manual configuration.
2017-10-13 17:17:36 +01:00
Nick Craig-Wood 7f3dc9b5c4 Implement WebDAV remote #580
This has special knowledge of Owncloud and Nextcloud to enable more
functionality such as mod times.
2017-10-09 16:19:37 +01:00
Nick Craig-Wood 9e9297838f Implement pcloud remote - #418 2017-10-01 11:37:35 +01:00
Nick Craig-Wood 63d0734c71 tree: remove workaround for tree library bug now it is fixed 2017-09-30 15:51:14 +01:00
ishuah 7195e44dce crypt: added cryptdecode command - #1129 2017-09-24 20:02:59 +01:00
Stefan 85877f3adc config: add show/file subcommands which print the config/its path (fixes #1086) 2017-09-19 17:59:19 +02:00
Stefan Breunig 12405f9f41 fuse: re-use rcat to support uploads for all remotes (fixes #1672) 2017-09-16 22:49:08 +02:00
Stefan Breunig 80b1f2a494 rcat: configurable small files cutoff and implement proper upload verification 2017-09-11 08:26:53 +02:00
bpicode f1f7e0e6f9 support for zsh auto-completion - #983 2017-08-31 16:21:28 +01:00
Stefan Breunig 0158ab6926 info: add check to stream files with unknown size 2017-08-22 08:00:10 +02:00
Nick Craig-Wood bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood 92d2e1f8d7 azureblob: rework and complete #801
* Fixup bitrot (rclone and Azure library)
  * Implement Copy
  * Add modtime to metadata under mtime key as RFC3339Nano
  * Make multipart upload work
  * Make it pass the integration tests
  * Fix uploading of zero length blobs
  * Rename to azureblob as it seems likely we will do azurefile
  * Add docs
2017-08-04 22:56:16 +01:00
wuyu ec5b72f8d5 Add new QingStor remote
Add new package qingstor to support QingStor API.

Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.

Add new docs for it.
2017-08-04 17:25:47 +01:00
Nick Craig-Wood 7d34caac83 cmd: add os and go version to rclone version output 2017-08-04 14:25:55 +01:00
Stefan Breunig 28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood 21aca68680 tree: fix when running under Windows 2017-08-01 14:46:21 +01:00
Andy Pilate ef328c5497 Fixes typo in command dedupe definition 2017-07-27 21:17:57 +01:00
Nick Craig-Wood f3060caf04 Implement tree command - fixes #1528 2017-07-26 23:06:48 +01:00
Nick Craig-Wood ebc8361933 mount: Add notes on Windows limitations from Bill Zissimopoulos 2017-07-26 21:08:24 +01:00
Nick Craig-Wood cd4895690a fstest: Factor test initialisation into Initialise() 2017-07-26 16:38:33 +01:00
Nick Craig-Wood de54fd4c64 mount: add docs for windows install 2017-07-23 13:05:02 +01:00
Nick Craig-Wood 7b81f12dad box: add docs
* reorder remotes so they are in alphabetical order by full name everywhere
  * update CONTRIBUTING doc
2017-07-23 11:32:34 +01:00
Nick Craig-Wood c223464cd0 mount: fix panic on renames - fixes #1533
Make sure d.items is not nil and improve locking
2017-07-22 11:00:51 +01:00
Yaroslav Halchenko 3ddcbce989 DOC: any empty directoryies -> empty directories (fixes #1546) 2017-07-22 10:24:41 +02:00
Nick Craig-Wood e86f62c3e8 Add rclone info internal command for testing out limits of remotes 2017-07-03 15:05:27 +01:00
Nick Craig-Wood e7d04fc103 Create fs.Directory interface and use it everywhere 2017-06-30 14:26:59 +01:00
Nick Craig-Wood e2d7d413ef fs: rename BasicInfo to DirEntry 2017-06-30 14:26:58 +01:00
Nick Craig-Wood aa20486485 Add --stats-log-level so can see --stats without -v - fixes #1180
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
2017-06-26 22:50:37 +01:00
sainaen 1817d8f631 crypt: Fix typo in cryptcheck's short description 2017-06-26 12:35:20 +01:00
Nick Craig-Wood b360527931 mount: fix hang on errored upload
In certain circumstances if an upload failed then the mount could hang
indefinitely. This was fixed by closing the read pipe after the Put
completed.  This will cause the write side to return a pipe closed
error fixing the hang.

Fixes #1498
2017-06-26 12:08:51 +01:00
Nick Craig-Wood b22c4c4307 http: fix, tidy and rework ready for release
* Fix remaining problems
  * Refactor to make testing easier and add a test suite
  * Make path parsing more robust.
  * Add single file operations
  * Add MimeType reading for objects
  * Add documentation
  * Note go1.7+ is required to build
2017-06-20 21:27:14 +01:00
Nick Craig-Wood dcce65b2b3 mount/cmount: factor duplicated code into mountlib 2017-06-19 14:36:51 +01:00
Nick Craig-Wood 97364fd0b6 ncdu: disable on plan9 and solaris as termbox isn't supported there 2017-06-15 20:10:54 +01:00
Nick Craig-Wood c34f11a92f rclone ncdu for exploring a remote with a text based user interface. 2017-06-15 17:44:17 +01:00
Nick Craig-Wood 7be49249d3 Add lsjson command - fixes #1063 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 7e20e16cff core: Implement Walk directory listing and use in place of Lister
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
2017-06-14 16:49:40 +01:00
Nick Craig-Wood 68333d34a1 dropbox: make setting mod time on existing files work properly
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
2017-06-13 13:58:39 +01:00
Nick Craig-Wood 740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood 28fcc53e45 mount test: retry umount as it fails occasionally
This is because of the background releasing of files which happens
after all the files are closed.
2017-06-13 10:52:10 +01:00
Nick Craig-Wood d30cc1e119 Factor RemoteSplit into fs 2017-06-07 12:27:33 +01:00
Stefan Breunig 9782c264e9 hand dirCacheTime through again 2017-06-01 09:02:22 +01:00
Nick Craig-Wood 52e96bc0e2 dropbox: add missing dbhashsum command
This was missed from 6381959850
2017-05-30 19:26:06 +01:00
Nick Craig-Wood 6381959850 dropbox: support Dropbox content hashing scheme - fixes #1302
* add support to hashing module
  * add dbhashsum to list the hashes
  * add support to dropbox module

This means objects up and downloaded to/from Dropbox will have their
hashes checked.

Note after this change local objects are calculating MD5, SHA1 and
DBHASH which is excessive and needs to be fixed.
2017-05-29 12:04:44 +01:00
Stefan Breunig a2e3af0523 poll for Google Drive changes when mounted 2017-05-28 17:54:52 +01:00
Sjur Fredriksen ade61fa756 Updated FTP to follow SFTP standards, updated documentation 2017-05-25 09:30:15 +01:00
Nick Craig-Wood 8f54dc06a2 Use build tags to control when and where cmount is built 2017-05-19 17:08:04 +01:00
Nick Craig-Wood 2cae017738 mountlib: fix race condition in cache clear 2017-05-19 15:47:52 +01:00
Nick Craig-Wood ecedcd0e7f cmount: stop failing tests on Windows 2017-05-15 17:40:44 +01:00
Nick Craig-Wood 3dff91d691 mount: add missing build constraint to fix Windows build 2017-05-15 17:40:15 +01:00
Nick Craig-Wood b259f8b752 cmount, mount, mountlib: make --read-only reject modify operations
Normally mount/cmount use `-o ro` to get the kernel to mark the fs as
read only.  However this is ignored by WinFsp, so in addition if
`--read-only` is in effect then return EROFS ("Read only File System")
from all methods which attempt to modify something.
2017-05-15 16:56:47 +01:00
Nick Craig-Wood 8be8a8e41b mountlib: on read only open of file, make open pending until first read
This fixes a problem with Windows which seems fond of opening files
just to read their attributes and closing them again.
2017-05-15 16:56:47 +01:00
Nick Craig-Wood f9500729b7 mountlib: fix cross platform tests 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 204a19e67f cmount: Wait for mountpoint to appear on Windows before declaring mounted 2017-05-15 16:56:46 +01:00
Nick Craig-Wood e6ffe3464c cmount: check for filesystem blowing up before Init is called 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 0384364c3e cmount: pass --FileSystemName under windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 763facfd78 cmount: implement --fuse-flag to pass commands to fuse library directly
Useful for `--fuse-flag -h` to see exactly which options the library supports.
2017-05-15 16:56:46 +01:00
Nick Craig-Wood bc88f1dafa cmount: fix openFile leak 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 0c055a1215 cmount: Statfs: reduce max size of volume for Windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 938d7951ab cmount: allow extra options to pass to fuse with -o 2017-05-15 16:56:45 +01:00
Nick Craig-Wood b4466bd9b1 Add -o uid=-1 -o gid=-1 for Windows/WinFsp 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 31f76aa464 cmount: implement no-ops for Fsync, Chmod, Chown, Access, Fsyncdir and stop using fuse.FileSystemBase 2017-05-15 16:56:45 +01:00
Nick Craig-Wood c887c164dc cmount: add function tracing 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 115ac00222 mount, mountlib: move function tracing into mount 2017-05-15 16:56:45 +01:00
Nick Craig-Wood abda616f84 mountlib: make Nodes also be fmt.Stringer so they debug nicely 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 9c3048580a cmount: fix code quality warnings 2017-05-15 16:56:45 +01:00
Nick Craig-Wood c1d5faa32a mountlib: fix code quality warnings 2017-05-15 16:56:45 +01:00
Nick Craig-Wood d127d8686a mountlib: pass options in fsys not as args 2017-05-15 16:56:44 +01:00
Nick Craig-Wood bc9856b570 Forward port 930ff266f2 to cmount branch
compare checksums on upload/download via FUSE
2017-05-15 16:56:44 +01:00
Nick Craig-Wood 855071cc19 cmount: name the command mount under windows and cmount under linux 2017-05-15 16:56:44 +01:00
Nick Craig-Wood b179540e80 cmount: fix Getattr to work on directories 2017-05-15 16:56:44 +01:00
Nick Craig-Wood 6a8e4690d3 mountlib: windows fixes for drive letter and timing 2017-05-15 16:56:44 +01:00
Nick Craig-Wood 917ea6ac57 mountlib: make tests work under all platforms 2017-05-15 16:56:44 +01:00
Nick Craig-Wood 7b47a1e842 cmount: set the correct values for uid, gid under Windows 2017-05-15 16:56:44 +01:00
Nick Craig-Wood bcd87009e2 Fix docs typo 2017-05-15 16:56:44 +01:00
Nick Craig-Wood caf85737c3 cmount: fix Windows compile (thanks Bill Zissimopoulos) 2017-05-15 16:56:44 +01:00
Nick Craig-Wood e1516e0159 Forward port 58a82cd578 into cmount branch
allow the fuse directory cached to be cleaned manually
2017-05-15 16:56:43 +01:00
Nick Craig-Wood ee1111e4c9 cmount: a new mount option based on cgofuse.
This with the aid of WinFSP should work on Windows.

Unfinished bits
  * 1 test doesn't pass
  * docs
  * build
2017-05-15 16:56:43 +01:00
Nick Craig-Wood 268fe0004c mount: factor filesystem code into mountlib and mounttest 2017-05-12 21:24:24 +01:00
Stefan Breunig 58a82cd578 allow the fuse directory cached to be cleaned manually (fixes #803) 2017-05-07 12:08:59 +01:00
Stefan Breunig 930ff266f2 compare checksums on upload/download via FUSE 2017-05-02 22:27:38 +01:00
Stefan Breunig 2fd86c93fc allow modTime to be changed even before all writers are closed (fixes #1197 -- again) 2017-03-31 01:28:08 +02:00
Ihor Dvoretskyi a54692d165 OneDrive vs One Drive
It's better to call this service as it's officially named.
2017-03-29 18:11:33 +01:00
marvwatson 4810aa65a4 Update references from HTTP to HTTPS where possible 2017-03-29 05:38:34 -07:00
Stefan Breunig 4dc030d081 implement ModTime via FUSE for remotes that support it (fixes #1197) 2017-03-24 09:23:04 +01:00
Jérôme Vizcaino 788b6ce821 mount: umount dir when program ends with SIGINT (Ctrl+C) or SIGTERM 2017-03-18 21:24:05 +00:00
Nick Craig-Wood 1cc58e4e09 mount: fix logging for unimplemented file open modes #1195 2017-03-02 22:07:01 +00:00
Nick Craig-Wood 30e97ad9ec Fix parsing of remotes in moveto and copyto - fixes #1079 2017-02-22 22:09:33 +00:00
Nick Craig-Wood 07dc76eff0 Remove unused test file 2017-02-22 20:58:24 +00:00
Nick Craig-Wood 31745320c8 Log the rclone version at the end of the run - fixes #847 2017-02-20 16:36:25 +00:00
Nick Craig-Wood 2da6cd7f84 Introduce AtExit to fix --cpuprofile and --memprofile to write profiles at end of run 2017-02-20 16:33:45 +00:00
Nick Craig-Wood 3ef9f6f016 mount: add test scripts 2017-02-17 11:37:19 +00:00
Nick Craig-Wood 033d1eb7af Refactor Account interface 2017-02-17 09:15:24 +00:00
Nick Craig-Wood 928be0f1fd mount: fix seek with buffering to use correct interface
Stop pre-cache before seeking which stops lots of excess data transfer
2017-02-17 08:55:24 +00:00
Nick Craig-Wood ef604f6100 mount: implement renaming directories - fixes #954
This also fixes various caching issues renaming files.
2017-02-16 17:42:38 +00:00
Nick Craig-Wood dac4bb22d3 mount: Make include and exclude filters apply to mount - fixes #1060 2017-02-15 23:28:53 +00:00
Nick Craig-Wood 9a9d09845c mount: put read and write async buffers back - control with --buffer-size #1043 2017-02-14 22:59:52 +00:00
Nick Craig-Wood 541929258b check: Add --download flag to check all the data, not just hashes 2017-02-13 10:48:26 +00:00
Nick Craig-Wood 01c747e7db Add cryptcheck command to check integrity of crypt remotes #1102 2017-02-12 16:30:18 +00:00
Nick Craig-Wood ca0e25b1a1 Remove spurious comment 2017-02-12 10:56:52 +00:00
Nick Craig-Wood 666dae4229 Add --syslog flag to optionally log to syslog on capable platforms 2017-02-11 20:27:46 +00:00
Nick Craig-Wood ac1c041377 Redo log level flags
* -vv or --log-level DEBUG
  * -v or --log-level INFO
  * --log-level NOTICE (default)
  * -q --log-level ERROR

Replace Config.Verbose and Config.Quiet with Config.LogLevel

Fixes #739 Fixes #1108 Fixes #1000
2017-02-11 20:22:42 +00:00
Nick Craig-Wood 0366ea39c5 Reassign some logging levels 2017-02-11 17:56:05 +00:00
Nick Craig-Wood 80f53176d9 Rename log functions and factor into own file 2017-02-11 17:54:50 +00:00
Nick Craig-Wood d091d4a8bb rclone cat: add --head, --tail, --offset, --count and --discard
Fixes #819
2017-02-08 08:09:41 +00:00
Nick Craig-Wood 288302c2cf Make fallback purge delete empty directories too.
This was implemented to make the SFTP unit tests pass.
2017-02-04 10:29:46 +00:00
Nick Craig-Wood 8a11da4e14 mount: Make fsync be a no-op for direectories too #1045 2017-02-02 21:31:41 +00:00
Nick Craig-Wood 37fe6d56e5 mount: fix docs for umount flags - fixes #1036 2017-01-30 18:17:16 +00:00
Nick Craig-Wood 29c6e22024 mount: Make fsync be a no-op rather than returning an error - fixes #1045 2017-01-29 11:29:42 +00:00
Nick Craig-Wood a6b4065e13 mount: fix retry on network failure when reading off crypt - fixes #1042 2017-01-17 16:32:04 +00:00
T.C. Ferguson d4c923a5cc Add obscure command for generating encrypted passwords for rclone's config 2017-01-10 13:18:09 +00:00
Nick Craig-Wood 35a6436983 mount: implement proper directory handling (mkdir, rmdir)
Before this change mount only simulated rmdir & mkdir, now it actually
runs mkdir & rmdir on the underlying remote, using the new parmaeters
to fs.Mkdir and fs.Rmdir.

Fixes #956
2017-01-06 11:24:22 +00:00
Nick Craig-Wood f73ee5eade Make all config file variables be settable in the environment
These are set in the form RCLONE_CONFIG_remote_option where remote is
the uppercased remote name and option is the uppercased config file
option name.  Note that RCLONE_CONFIG_remote_TYPE must be set if
defining a new remote.

Fixes #616
2017-01-03 22:42:47 +00:00
Nick Craig-Wood 0d75d2585f Allow all options to be set from environment variables
The option names are munged changing - to _ making upper case and
prepending RCLONE_.  The values are as parsed by pflag.
2017-01-03 22:42:47 +00:00
Lukas Loesche 5894c02a34 Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
Nick Craig-Wood 43c530922a Restore ability for any command to show stats by adding --stats flag
Make default for `mount` command not to show stats - this can be
re-enabled by adding a `--stats` parameter.
2016-12-15 17:40:17 +00:00
Nick Craig-Wood dd60f088ed mount: retry reads on error #873 2016-12-15 17:16:55 +00:00
Nick Craig-Wood 0117aeafbf mount: this removes the async buffering as it was killing seek performance 2016-12-15 17:08:52 +00:00
Nick Craig-Wood 13b705e227 mount: report the modification times for directories from the remote #940 #950
This stops the modification times for directories just being the
current time and reads them from the remote instead.  This doesn't
take any extra transactions.
2016-12-14 15:26:04 +00:00
Nick Craig-Wood c24da0b886 fuse: add stats printing and note which files are transferring 2016-12-04 16:59:46 +00:00
Nick Craig-Wood be4fd51289 fuse: Add bandwidth accounting and buffering
This fixes rclone mount ignoring bwlimit and increases buffering which
should speed up transfers greatly.

Fixes #796
Fixes #690
2016-12-04 16:57:47 +00:00
Nick Craig-Wood 2cbdb95ce5 Only show transfer stats on commands which transfer stuff - fixes #849 2016-12-04 16:52:24 +00:00
Nick Craig-Wood c265f451f2 Implement moveto and copyto commands for choosing a destination name on copy/move
Fixes #227
Fixes #476
2016-12-03 23:43:52 +00:00
Nick Craig-Wood 2058652fa4 Allow overlapping remotes in move when DirMove is supported 2016-12-03 09:08:40 +00:00
Nick Craig-Wood 5e35aeca9e Regularize the command definition names 2016-12-03 09:08:40 +00:00
Nick Craig-Wood 7929b6e756 fuse: support R/W files only if truncate is set.
Any reads on the file handle will return an error.  This is to support
windows/samba writes.
2016-11-28 17:56:54 +00:00
Scott McGillivray f9df545e3c add --stats-unit option and improve alignment for --stats output 2016-11-28 12:18:30 +00:00
Scott McGillivray 5e62ede8d0 make the parameter format for --stats flag more obvious 2016-11-27 18:57:23 +00:00
Nick Craig-Wood 8aae166a5b Add missing rmdirs command 2016-11-27 18:36:13 +00:00
Nick Craig-Wood f3365dd251 Make rclone rmdirs command to delete empty directories - fixes #831 2016-11-27 11:49:31 +00:00
Nick Craig-Wood aaa1370a36 Add directory parameter to Rmdir and Mkdir #100 #831
This will enable rclone to manage directories properly in the future.
2016-11-26 12:02:53 +00:00
Nick Craig-Wood c41b67ea08 mount: Implement statfs interface so df works - fixes #894
The data returned is not related to the files on the remote, but
apparently samba needs it.
2016-11-20 22:54:03 +00:00
Nick Craig-Wood 0b562bcabc mount: Note that write is now supported on ACD 2016-11-19 10:54:37 +00:00
Stefan Breunig bf243f30d3 report number of blocks in fuse 2016-11-12 14:10:36 +01:00
Nick Craig-Wood 64b5a76bec mount: detect and deal with seeking beyond end of file - fixes #828 2016-11-05 09:59:36 +00:00
Nick Craig-Wood 7cfb1bdc70 fuse: tests: create the directory before starting tests 2016-11-05 09:57:45 +00:00
Nick Craig-Wood 062616e4dd mount: update code comments 2016-10-30 17:46:00 +00:00
Nick Craig-Wood 87107413f5 fuse: add missing locking on filehandle read #823 #802 2016-10-27 09:57:52 +01:00
Nick Craig-Wood 77b975d16f Note Amazon Drive doesn't support uploads via FUSE yet 2016-10-23 21:46:48 +01:00
Nick Craig-Wood c464cc6376 mount: fix alignment of 64 bit counter on ARM #813 2016-10-23 17:36:35 +01:00
Nick Craig-Wood 93e84403bb Remove io.SeekStart and replace with 0 as it is go 1.7 only 2016-10-22 12:07:51 +01:00
Nick Craig-Wood eba0a3633b crypt: speed up repeated seeking - fixes #804 2016-10-21 10:03:16 +01:00
Nick Craig-Wood de73063977 Fix output of crypt objects in logs 2016-10-20 17:46:51 +01:00
Nick Craig-Wood a02edb9e69 Add rclone mount --dir-cache-time to control caching of directory entries - fixes #680 2016-10-18 17:23:57 +01:00
Nick Craig-Wood d8d11023d3 mount: update internal position on seek - fixes #774 2016-10-17 20:20:07 +01:00
Nick Craig-Wood ace1e21894 Add listremotes command - fixes #558 2016-10-08 14:24:37 +01:00
Nick Craig-Wood f45b3c87bf mount: add --no-seek flag to disable seeking 2016-10-06 13:37:45 +01:00
Nick Craig-Wood de80a540a7 mount: attempt to speed up 2016-10-05 21:04:57 +01:00
Nick Craig-Wood 392a86f585 mount: Fix read flushing - fixes #638 2016-10-05 21:03:56 +01:00
Nick Craig-Wood 265f5b77a7 mount: make files opened for read seekable - fixes #707 2016-10-05 21:03:56 +01:00
Nick Craig-Wood bfe6f299d0 Revise list of OSes which can redirect stderr - fixes #698 2016-09-19 17:13:41 +01:00
Nick Craig-Wood 4001e21624 Make sure high level retries show with -q - fixes #648
Also update the exit code documentation describing that.
2016-09-12 17:50:19 +01:00
Nick Craig-Wood bbf819e2d1 Move versioncheck so it happens earlier in the compile process. 2016-09-12 17:50:19 +01:00
Nick Craig-Wood 5c91623148 mount: Implement FUSE mount options - fixes #653 2016-09-10 09:50:46 +01:00
Fredrik Fornwall 87db3cfad3 Use Dup2 library function instead of raw syscall
The Dup2 syscall does not exist on 64-bit arm Linux while its
replacement Dup3 does not exist on non-Linux systems.

Using the unix.Dup2 library function instead of raw syscalls
improves the portability across more platforms.
2016-09-08 08:17:31 +01:00
Nick Craig-Wood ca841c56a8 Disable smart dashes so `--flag` shows properly in the docs - fixes #632 2016-08-22 16:46:08 +01:00
Nick Craig-Wood 9e7ddd5efc Fix tests when FUSE isn't present 2016-08-20 14:11:21 +01:00
Nick Craig-Wood 84eb7031bb Implement the rclone cat command 2016-08-18 22:45:32 +01:00
Nick Craig-Wood f22029bf3d Add mount command to implement FUSE mounting of remotes #494
This enables any rclone remote to be mounted and used as a filesystem
with some limitations.

Only supported for Linux, FreeBSD and OS X
2016-08-18 21:54:54 +01:00
Nick Craig-Wood ae6018355c Correct parameter order for copy/sync etc 2016-08-06 00:07:36 +01:00
Nick Craig-Wood e27b91ffb8 Factor each commmand into its own package 2016-08-05 17:13:54 +01:00
Nick Craig-Wood 0a7b34eefc Move internals of rclone command into cmd so it can be imported externally 2016-08-04 22:33:46 +01:00