Reduce the number of nodes purged from the dir-cache when ForgetPath is
called. This is done by only forgetting the cache of the received path
and invalidating the parent folder cache by resetting *Dir.read.
The parent will read the listing on the next access and reuse the
dir-cache of entries in *Dir.items.
Before this change remotes without server side Move (eg swift, s3,
gcs) would not be able to rename files.
After it means nearly all remotes will be able to rename files on
rclone mount with the notable exceptions of b2 and yandex.
This changes checks to see if the remote can do Move or Copy then
calls `operations.Move` to do the actual move. This will do a server
side Move or Copy but won't download and re-upload the file.
It also checks to see if the destination exists first which avoids
conflicts or duplicates.
Fixes#1965Fixes#2569
vfs/refresh will walk the directory tree for the given paths and
freshen the directory cache. It will use the fast-list capability
of the remote when enabled.
This stops the cache cleaner running unnecessarily and saves
resources.
This also helps with issue #2227 which was caused by a second mount
deleting objects in the first mounts cache.
Without this fix the cached file can be removed as the file is being
uploaded or downloaded. This can cause the directory listings to
become inconsistent (this issue) or data loss (if a retry was needed
in the Copy).
Remove file needs to be excluded from running at the same time as both
openPending and close so it makes sense to unify the locking between
all 3.
This was caused by this sequence of calls
1> file.Release
1> file.close -> takes the file lock
2> vfs.waitforWriters
2> dir.walk -> takes the dir lock
1> file.setObject
1> dir.addObject -> attempts to take the dir lock - BLOCKS
2> file.activeWriters -> tries to take file lock - BLOCKS - DEADLOCK
The fix is to make activeWriters not take the file lock and use atomic
operations to read the number of writers instead.
This no longer needs to deal with O_RDONLY and O_TRUNC since we
disallow this earlier. This also fixes the code to just do it for
O_APPEND, not for everything.
Before this change Open("name", os.O_RDONLY|os.O_TRUNC) would have
truncated the file. This is what Linux does, but is counterintuitive.
POSIX states this is undefined, so return an error in this case
instead. This preserves the invariant O_RDONLY => file is not
changed.
Background: cmd/mount/file.go Open() function does a Seek(0, 1) to see
if the file handle is seekable to set a FUSE hint. Before this change
the file was downloaded before it needed to be which was inefficient
(and broke beta.rclone.org because HEAD requests caused downloads!).
Nick Craig-Wood
Fabian Möller
Oleg Kovalov
Stefan
Mateusz Pabian
Remus Bunduc