Commit Graph

209 Commits

Author SHA1 Message Date
Ovidiu Victor Tatar 502226bfc8 pacer: add ZeroDelayCalculator 2022-07-08 18:24:54 +01:00
Ovidiu Victor Tatar 53400d7edc oauthlib: add method to set a token as expired
This can be used by backends to trigger a refresh of an access token if
they detect an invalid token.
2022-07-08 18:24:54 +01:00
albertony ad8c94e982 staticcheck: redundant return statement 2022-07-04 11:24:59 +02:00
albertony f18095b004 staticcheck: ignore deprecations that are not relevant 2022-07-04 11:24:59 +02:00
albertony 73e3bb09d7 http: fix missing response when using custom auth handler 2022-07-02 15:29:50 +02:00
Martin Czygan 5de9278650
fs/cache: make sure we call the Shutdown method on backends
This change ensures we call the Shutdown method on backends when
they drop out of the fs/cache and at program exit.

Some backends implement the optional fs.Shutdowner interface. Until now,
Shutdown is only checked and called, when a backend is wrapped (e.g.
crypt, compress, ...).

To have a general way to perform operations at the end of the backend
lifecycle with proper error handling, we can call Shutdown at cache
clear time.

We add a finalize hook to the cache which will be called when values
drop out of the cache.

Previous discussion: https://forum.rclone.org/t/31336
2022-06-28 12:51:59 +01:00
albertony fdd2f8e6d2 Error strings should not be capitalized
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-23 23:26:02 +02:00
albertony 53f831f40a docs: add missing code section formatting to commands and flags 2022-06-21 23:43:00 +02:00
albertony ec117593f1 Fix lint issues reported by staticcheck
Used staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-13 21:13:50 +02:00
Eng Zer Jun 4f0ddb60e7 refactor: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-17 11:08:37 +01:00
Nick Craig-Wood 50a0c3482d lib/readers: add FakeSeeker to adapt io.Reader to io.ReadSeeker #5422 2022-05-12 08:49:20 +01:00
Nick Craig-Wood 07481396e0 lib/encoder: add Semicolon encoding 2022-04-21 12:02:27 +01:00
Tobias Klauser 54c9c3156c fs/config, lib/terminal: use golang.org/x/term
golang.org/x/crypto/ssh/terminal is deprecated in favor of
golang.org/x/term, see https://pkg.go.dev/golang.org/x/crypto/ssh/terminal

The latter also supports ReadPassword on solaris, so enable the
respective functionality in fs/config for solaris as well.
2022-04-01 12:48:18 +01:00
Nick Craig-Wood 2738db22fb pacer: default the Google pacer to a burst of 100 to fix gcs pacing
Before this change the pacer defaulted to a burst of 1 which mean that
it kept being activated unecessarily.

This affected Google Cloud Storage and Google Photos.

See: https://forum.rclone.org/t/no-traverse-too-slow-with-lot-of-files/29886/12
2022-03-25 15:28:59 +00:00
Nil Alexandrov 47f9ab2f56 lib/rest: add support for setting trailers 2022-03-09 12:42:22 +00:00
Eng Zer Jun 8cf76f5e11 test: use `T.TempDir` to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-02-01 11:47:04 +00:00
albertony 2523dd6220 version: report correct friendly-name for windows 10/11 versions after 2004
Until Windows 10 version 2004 (May 2020) this can be found from registry entry
ReleaseID, after that we must use entry DisplayVersion (ReleaseId is stuck at 2009).
Source: https://ss64.com/nt/ver.html
2022-01-24 21:27:42 +01:00
Koopa 0681a5c86a lib/rest: process HTML entities within XML
MEGAcmd currently includes escaped HTML4 entites in its XML messages.
This behavior deviates from the XML standard, but currently it prevents
rclone from being able to use the remote.
2021-12-01 16:31:43 +00:00
Nick Craig-Wood 3425726c50 oauthutil: fix crash when webrowser requests /robots.txt - fixes #5836
Before this change the oauth webserver would crash if it received a
request to /robots.txt.

This patch makes it ignore (with 404 error) any paths it isn't
expecting.
2021-11-25 12:12:14 +00:00
Nick Craig-Wood e43b5ce5e5 Remove github.com/pkg/errors and replace with std library version
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.

This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
2021-11-07 11:53:30 +00:00
Atílio Antônio c08d48a50d
docs: improve grammar and fix typos (#5361)
This alters some comments in source files, but is interested mainly in documentation files and help messages.
2021-11-04 12:50:43 +01:00
Ivan Andreev e78e73eae7 lib/encoder: fix benchmarks
Some day in the past the Slash encode option was added to Onedrive
encoder so it began to encode slashes in file names rather then treat
them as path separators.
This patch adapts benchmark test cases accordingly.

Fixes #5659
2021-11-02 19:23:16 +03:00
Nick Craig-Wood 07fcba888c dircache: add SetRootIDAlias to update RootID from FindLeaf 2021-10-31 19:18:55 +00:00
Nolan Woods a074a2b983 lib/http: Fix handling of ssl credentials
Adds a test that makes an actual http and https request against the server
2021-10-27 14:46:10 +03:00
albertony e2f47ecdeb docs: punctuation cleanup
See #5538
2021-10-20 22:56:19 +02:00
Ivan Andreev 57c7fde864 lib/kv: add unit tests, fix races #5587
After testing concurrent calling of `kv.Start` and `db.Stop` I had to restrict
more parts of these under mutex to make results deterministic without Sleep's
in the test body. It's more safe but has potential to lock Start for up to
2 seconds due to `db.open`.
2021-10-20 19:11:54 +03:00
Ivan Andreev 50df8cec9c lib/kv: add key-value database api #5587
Add bolt-based key-value database support.

Quick API description:
https://github.com/rclone/rclone/pull/5587#issuecomment-942174768
2021-10-20 19:11:54 +03:00
Nick Craig-Wood a98e3ea6f1 build: replace the deprecated golint linter with revive
This fixes up a small number of new lint items also
2021-10-15 12:51:31 +01:00
Nolan Woods 023e32de05 lib/http: Factor password hash salt into options with default 2021-10-13 11:33:38 +01:00
Nolan Woods b1cb41f8da lib/http: Fix bad username check in single auth secret provider 2021-10-13 11:33:38 +01:00
Ivan Andreev a95c7a001e core: run rclone as mount helper - #5594 2021-10-03 01:44:08 +03:00
Ivan Andreev 8b8a943dd8 mountlib: correctly daemonize for compatibility with automount - #5593
This patch will:
- add --daemon-wait flag to control the time to wait for background mount
- remove dependency on sevlyar/go-daemon and implement backgrounding directly
- avoid setsid during backgrounding as it can result in race under Automount
- provide a fallback PATH to correctly run `fusermount` under systemd as it
  runs mount units without standard environment variables
- correctly handle ^C pressed while background process is being setting up
2021-10-02 23:45:16 +03:00
albertony fbc7f2e61b lib/file: improve error message when attempting to create dir on nonexistent drive on windows
This replaces built-in os.MkdirAll with a patched version that stops the recursion
when reaching the volume part of the path. The original version would continue recursion,
and for extended length paths end up with \\? as the top-level directory, and the error
message would then be something like:
mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
2021-10-01 23:18:39 +02:00
Ivan Andreev 48e7246163 lib/encoder: add encoding of square brackets 2021-10-01 10:09:57 +03:00
Ivan Andreev 24de896df2 build: apply gofmt from golang 1.17 2021-09-09 20:43:59 +03:00
Ivan Andreev aa0ceb6c5c cmd/version: add support for openbsd/386
After this patch the version command will be fully
supported on openbsd/amd64 and openbsd/386.
Remaining os/arch combinations stay as is.
2021-09-02 11:13:12 +03:00
Nick Craig-Wood 2cefae51a1 build: make go1.14 the minimum supported Go for building 2021-08-20 19:04:29 +01:00
albertony 9a2811f0b2 local: refactor default os encoding out from local backend into shared encoder lib 2021-08-19 20:14:50 +02:00
Ivan Andreev e45c23ab79 cmd/version: add support for openbsd/amd64
After this patch the version command will be
- fully supported on openbsd/amd64
- stay stub on openbsd/i386 until we deprecate go 1.17
Remaining os/arch combinations stay as is.
2021-08-16 11:39:34 +03:00
Cnly bc74f0621e http: fix serve http exits directly after starting 2021-07-25 14:06:43 +01:00
Michael Hanselmann 6b7d7d0441 atexit: Terminate with non-zero status after receiving signal
When rclone received a SIGINT (Ctrl+C) or SIGTERM signal while an atexit
function is registered it always terminated with status code 0. Unix
convention is to exit with a non-zero status code. Often it's
`128 + int(signum), but at least not zero.

With this change fatal signals handled by the `atexit` package cause
a non-zero exit code. On Unix systems it's `128 + int(signum)` while
on other systems, such as Windows, it's always 2 ("error not otherwise
categorised").

Resolves #5437.

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
2021-07-07 17:59:26 +01:00
Michael Hanselmann cf19073ac9 cmd: Move exit status codes to separate package
Signal handling by the `atexit` package needs acceess to
`exitCodeUncategorizedError`. With this change all exit status values
are moved to a dedicated package so that they can be reused.

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
2021-07-07 17:59:26 +01:00
Nick Craig-Wood 06f27384dd b2: fix versions and .files with no extension - fixes #5244 2021-05-17 16:20:29 +01:00
Nick Craig-Wood f122808d86 fs: add names to each config parameter so we can override them #3455 2021-05-14 14:07:44 +01:00
Nick Craig-Wood 94dbfa4ea6 fs: change Config callback into state based callback #3455
This is a very large change which turns the post Config function in
backends into a state based call and response system so that
alternative user interfaces can be added.

The existing config logic has been converted, but it is quite
complicated and folloup commits will likely be needed to fix it!

Follow up commits will add a command line and API based way of using
this configuration system.
2021-05-14 14:07:44 +01:00
Nolan Woods e489a101f6 lib/http: add default 404 handler 2021-05-02 11:30:02 +01:00
Nolan Woods 002d323c94 lib/http: Move HTTP object serialization logic to lib/http 2021-04-28 22:54:15 +01:00
Nolan Woods 4ad62ec016 lib/http: Add authentication middleware with basic auth implementation 2021-04-28 22:54:15 +01:00
Nolan Woods 95ee14bb2c feat: Add lib/http
lib/http provides an abstraction for a central http server that services can bind routes to
2021-04-28 22:54:15 +01:00
Cnly 386acaa110 oauthutil: fix #5265 old authorize result not recognised 2021-04-23 01:20:52 +08:00