From 7ae40cb3527e982e6fd35d48f71d085b7d87495f Mon Sep 17 00:00:00 2001 From: klauspost Date: Wed, 13 Jan 2016 11:29:43 +0100 Subject: [PATCH] Update information on revised hash functionality. --- docs/content/about.md | 8 ++++---- docs/content/b2.md | 5 ++--- docs/content/docs.md | 10 +++++++--- docs/content/onedrive.md | 6 +++--- docs/content/overview.md | 35 +++++++++++++++++++---------------- 5 files changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/content/about.md b/docs/content/about.md index 641276252..136c1f7d5 100644 --- a/docs/content/about.md +++ b/docs/content/about.md @@ -27,13 +27,13 @@ Rclone is a command line program to sync files and directories to and from Features - * MD5SUMs checked at all times for file integrity + * MD5/SHA1 hashes checked at all times for file integrity * Timestamps preserved on files * Partial syncs supported on a whole file basis * Copy mode to just copy new/changed files - * Sync mode to make a directory identical - * Check mode to check all MD5SUMs - * Can sync to and from network, eg two different Drive accounts + * Sync (one way) mode to make a directory identical + * Check mode to check for file hash equality + * Can sync to and from network, eg two different cloud accounts Links diff --git a/docs/content/b2.md b/docs/content/b2.md index 948f34771..2fc8b3aec 100644 --- a/docs/content/b2.md +++ b/docs/content/b2.md @@ -90,9 +90,8 @@ have an API method to set them independently of doing an upload. ### SHA1 checksums ### -The SHA1 checksums of the files are checked on upload and download, -but they aren't used for sync as they aren't compatible with the MD5 -checksums rclone normally uses. +The SHA1 checksums of the files are checked on upload and download and +will be used in the syncing process. You can use the `--checksum` flag. ### Versions ### diff --git a/docs/content/docs.md b/docs/content/docs.md index 251782638..d5032e08c 100644 --- a/docs/content/docs.md +++ b/docs/content/docs.md @@ -189,11 +189,15 @@ The default is to run 8 checkers in parallel. Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check -MD5SUM and size to determine if files are equal. +the file hash and size to determine if files are equal. + +This is useful when the remote doesn't support setting modified time +and a more accurate sync is desired than just checking the file size. This is very useful when transferring between remotes which store the -MD5SUM on the object which include swift, s3, drive, and google cloud -storage. +same hash type on the object, eg Drive and Swift. For details of which +remotes support which hash type see the table in the [overview +section](/overview/). Eg `rclone --checksum sync s3:/bucket swift:/bucket` would run much quicker than without the `--checksum` flag. diff --git a/docs/content/onedrive.md b/docs/content/onedrive.md index 4e3dc0de9..7ace844f8 100644 --- a/docs/content/onedrive.md +++ b/docs/content/onedrive.md @@ -82,14 +82,14 @@ To copy a local directory to an One Drive directory called backup rclone copy /home/source remote:backup -### Modified time and MD5SUMs ### +### Modified time and hashes ### One Drive allows modification times to be set on objects accurate to 1 second. These will be used to detect whether objects need syncing or not. -One drive does not support MD5SUMs. This means the `--checksum` flag -will be equivalent to the `--size-only` flag. +One drive supports SHA1 type hashes, so you can use `--checksum` flag. + ### Deleting files ### diff --git a/docs/content/overview.md b/docs/content/overview.md index 882db1295..ea3b78063 100644 --- a/docs/content/overview.md +++ b/docs/content/overview.md @@ -15,27 +15,30 @@ show through. Here is an overview of the major features of each cloud storage system. -| Name | MD5SUM | ModTime | Case Insensitive | Duplicate Files | +| Name | Hash | ModTime | Case Insensitive | Duplicate Files | | ---------------------- |:-------:|:-------:|:----------------:|:---------------:| -| Google Drive | Yes | Yes | No | Yes | -| Amazon S3 | Yes | Yes | No | No | -| Openstack Swift | Yes | Yes | No | No | -| Dropbox | No | No | Yes | No | -| Google Cloud Storage | Yes | Yes | No | No | -| Amazon Cloud Drive | Yes | No | Yes | No | -| Microsoft One Drive | No | Yes | Yes | No | -| Hubic | Yes | Yes | No | No | -| Backblaze B2 | No | Partial | No | No | -| Yandex Disk | Yes | Yes | No | No | -| The local filesystem | Yes | Yes | Depends | No | +| Google Drive | MD5 | Yes | No | Yes | +| Amazon S3 | MD5 | Yes | No | No | +| Openstack Swift | MD5 | Yes | No | No | +| Dropbox | - | No | Yes | No | +| Google Cloud Storage | MD5 | Yes | No | No | +| Amazon Cloud Drive | MD5 | No | Yes | No | +| Microsoft One Drive | SHA1 | Yes | Yes | No | +| Hubic | MD5 | Yes | No | No | +| Backblaze B2 | SHA1 | Partial | No | No | +| Yandex Disk | MD5 | Yes | No | No | +| The local filesystem | All | Yes | Depends | No | -### MD5SUM ### +### Hash ### -The cloud storage system supports MD5SUMs of the objects. This -is used if available when transferring data as an integrity check and +The cloud storage system supports various hash types of the objects. +The hashes are used when transferring data as an integrity check and can be specifically used with the `--checksum` flag in syncs and in the `check` command. +To use the checksum checks between filesystems they must support a +common hash type. + ### ModTime ### The cloud storage system supports setting modification times on @@ -62,7 +65,7 @@ matter how many times you run the sync it never completes fully. The local filesystem may or may not be case sensitive depending on OS. - * Windows - usually case insensitive + * Windows - usually case insensitive, though case is preserved * OSX - usually case insensitive, though it is possible to format case sensitive * Linux - usually case sensitive, but there are case insensitive file systems (eg FAT formatted USB keys)