Commit Graph

3264 Commits

Author SHA1 Message Date
zeripath e6d46eeb55
Make migrations SKIP_TLS_VERIFY apply to git too (#19132) (#19141)
Backport #19132

Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command.

Fix #18998

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-19 16:20:26 +00:00
Gusted c1e6be47d7
Update golang.org/x/crypto (#19097) (#19098)
Backport #19097

* Update golang.org/x/crypto (#19097)

- Backport #19097

* Fix deprecation notice

* Backport workaround removal
2022-03-19 12:16:15 +00:00
Lunny Xiao 79a5e68816
Handle email address not exist (#19089) (#19121)
Backport #19089

* Handle email address not exist. (#19089)

* Fix lint about strings.Title

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-03-19 11:35:23 +00:00
6543 99861e3e06
Fix lfs bug (#19072) (#19080)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-14 15:59:54 +01:00
zeripath 0fe99cc00c
If rendering has failed due to a net.OpError stop rendering (attempt 2) (#19049) (#19056)
Backport #19049

Unfortunately #18642 does not work because a `*net.OpError` does not implement
the `Is` interface to make `errors.Is` work correctly - thus leading to the
irritating conclusion that a `*net.OpError` is not a `*net.OpError`.

Here we keep the `errors.Is` because presumably this will be fixed at
some point in the golang main source code but also we add a simply type
cast to also check.

Fix #18629

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 22:13:55 +01:00
zeripath 7aa29720f0
Improve SyncMirrors logging (#19045) (#19050)
Backport #19045

Yet another issue has come up where the logging from SyncMirrors does not provide
enough context. This PR adds more context to these logging events.

Related #19038

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 16:06:35 +01:00
6543 3e5c844a77
fix pam authorization (#19040) (#19047)
Backport #19040 

The PAM module has previously only checked the results of the authentication module.

However, in normal PAM practice most users will expect account module authorization to also be checked. Without doing this check in almost every configuration expired accounts and accounts with expired passwords will still be able to login.

This is likely to represent a significant gotcha in most configurations and cause most users configurations to be potentially insecure. Therefore we should add in the account authorization check.

## ⚠️ **BREAKING** ⚠️ 

Users of the PAM module who rely on account modules not being checked will need to change their PAM configuration.

However, as it is likely that the vast majority of users of PAM will be expecting account authorization to be checked in addition to authentication we should make this breaking change to make the default behaviour correct for the majority.

---

I suggest we backport this despite the BREAKING nature because of the surprising nature of this.

Thanks to @ysf for bringing this to our attention.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: ysf <34326+ysf@users.noreply.github.com>
2022-03-10 08:15:35 +00:00
zeripath dd52c08b74
Don't show context cancelled errors in attribute reader (#19006) (#19027)
Backport #19006

Fix #18997

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-08 19:20:37 +08:00
6543 87f8d37be5
git backend ignore replace objects (#18979) (#18980)
Co-authored-by: zeripath <art27@cantab.net>
2022-03-02 21:31:50 +00:00
Otto Richter (fnetX) a3f72303d1
Fix problem when self-assign notification (#18797) (#18976)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-02 20:11:55 +00:00
Otto Richter (fnetX) 578f19a682
Refactor admin user filter query parameters (#18965) (#18975)
Only pass `status_filter` on admin page
Use a more general method to pass query parameters, remove hard-coded keys

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-02 19:57:18 +01:00
Otto Richter (fnetX) 36e96e3481
Fix admin user list pagination (#18957) (#18960) 2022-03-01 23:00:03 +00:00
zeripath 5f9c18b2b3
Adjust error for already locked db and prevent level db lock on malformed connstr (#18923) (#18938)
Backport #18923

This PR adjusts the error returned when there is failure to lock the level db, and
permits a connections to the same leveldb where there is a different connection string.

Reference #18921
Reference #18917

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-28 15:45:38 +00:00
Gusted 4fb718d405
Don't treat BOM escape sequence as hidden character. (#18909) (#18910)
* Don't treat BOM escape sequence as hidden character. (#18909)

Backport #18909
2022-02-26 23:15:04 +01:00
Gusted df35049196
Allow adminstrator teams members to see other teams (#18918) (#18919)
Allow adminstrator teams members to see other teams (#18918)
2022-02-26 22:45:34 +01:00
silverwind ce75461380
Correctly link URLs to users/repos with dashes, dots or underscores (#18890) (#18908)
* Add tests for references with dashes

This commit adds tests for full URLs referencing repos names and user
names containing a dash.

* Extend regex to match URLs to repos/users with dashes

Co-authored-by: Alexander Neumann <62751754+rtpt-alexanderneumann@users.noreply.github.com>
2022-02-26 06:45:09 +01:00
Otto Richter (fnetX) 6039138323
Fix redirect when using lowercase reponame (#18775) (#18902)
* Previously,  `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL
* In other words both
   * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png`
   * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png`
were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png`
This isn't the case after  #17551. Specifically because of this [line](cbd5eecd14/modules/context/repo.go (L860)).

Co-authored-by: Ghanem <37152329+AbdulrhmnGhanem@users.noreply.github.com>
2022-02-26 08:16:22 +08:00
zeripath 382101ecc7
In disk_channel queues synchronously push to disk on shutdown (#18415) (#18788)
Partial Backport of #18415

Instead of using an asynchronous goroutine to push to disk on shutdown
just close the datachan and immediately push to the disk.

Prevents messages of incompletely flushed queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-22 20:08:35 +08:00
zeripath bec566282e
Immediately Hammer if second kill is sent (#18823) (#18826)
Backport #18823

Currently Gitea will wait for HammerTime or nice shutdown if kill -1 or kill -2
is sent. We should just immediately hammer if there is a second kill.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-20 01:37:52 +08:00
zeripath fa9be55018
Fix panic in EscapeReader (#18820) (#18821)
Backport #18820

There is a potential panic due to a mistaken resetting of the length parameter when
multibyte characters go over a read boundary.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-19 23:34:32 +08:00
zeripath c876124efe
Ensure git tag tests and others create test repos in tmpdir (#18447) (#18767)
Backport #18447

* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix #18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix #18077

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2022-02-15 19:16:38 +08:00
zeripath 7ebc3da7cb
Prevent dangling GetAttribute calls (#18754) (#18755)
* Prevent dangling GetAttribute calls

It appears possible that there could be a hang due to unread data from the
repo-attribute command pipes. This PR simply closes these during the defer.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move close into the defer

Signed-off-by: Andrew Thornton <art27@cantab.net>

* lets try again

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-14 15:27:55 -05:00
zeripath 2e36ba0a00
Send mail to issue/pr assignee/reviewer also when OnMention is set (#18707) (#18765)
Backport #18707

Addresses #17892, where emails notifications are not sent to assignees (issue and PR) and reviewers (PR) when they have the email setting Only email on mention enabled.

From the user experience perspective, when a user gets a issue/PR assigned or a PR review request, he/she would expect to be implicitly mentioned since the assignment or request is personal and targeting a single person only. Thus I see #17892 as a bug. Could we therefore mark this ticket as such?

The changed code just explicitly checks for the EmailNotificationsOnMention setting beside the existing EmailNotificationsEnabled check. Too rude?

@lunny mentioned a mock mail server for tests, is there something ready. How could I make use of it?

See #12774 (comment)

Fix #17892

Co-authored-by: flozzone <flozzone@gmail.com>
2022-02-14 21:13:41 +08:00
Lunny Xiao 913d6f3ff3
Fix isempty detection of git repository (#18746) (#18750)
* Fix isempty detection of git repository

* Fix IsEmpty check
2022-02-14 00:33:35 +08:00
zeripath 2e317d3f6e
Prevent security failure due to bad APP_ID (#18678) (#18682)
Backport #18678

WebAuthn may cause a security exception if the provided APP_ID is not allowed for the
current origin. Therefore we should reattempt authentication without the appid
extension.

Also we should allow [u2f] as-well as [U2F] sections.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-10 16:17:44 +01:00
zeripath 38fc6c75f3
Restart zero worker if there is still work to do (#18658) (#18672)
* Restart zero worker if there is still work to do (#18658)

Backport #18658

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix #18607

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/queue/workerpool.go
2022-02-08 23:28:21 +02:00
zeripath d4a075d738
If rendering has failed due to a net.OpError stop rendering (#18642) (#18645)
Backport #18642

When a net.OpError occurs during rendering the underlying connection is essentially
dead and therefore attempting to render further data will only cause further errors.

Therefore in serverErrorInternal detect if the passed in error is an OpError and
if so do not attempt any further rendering.

Fix #18629

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-07 09:25:05 +08:00
zeripath a13fb154ae
Stop logging an error when notes are not found (#18626) (#18635)
Backport #18626

This is an unnecessary logging event.

Fix #18616

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-06 09:43:15 +00:00
zeripath 36c66303df
Only attempt to flush queue if the underlying worker pool is not finished (#18593) (#18620)
* Only attempt to flush queue if the underlying worker pool is not finished (#18593)

Backport #18593

There is a possible race whereby a worker pool could be cancelled but yet the
underlying queue is not empty. This will lead to flush-all cycling because it
cannot empty the pool.

* On shutdown of Persistant Channel Queues close datachan and empty

Partial Backport #18415

Although we attempt to empty the datachan in queues - due to
races we are better off just closing the channel and forcibly emptying
it in shutdown.

Fix #18618

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Move zero workers warning to debug

Fix #18617

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/queue/manager.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update modules/queue/manager.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

Co-authored-by: Gusted <williamzijl7@hotmail.com>
2022-02-06 14:55:44 +08:00
zeripath bc196a35e1
Collaborator trust model should trust collaborators (#18539) (#18557)
Backport #18539

There was an unintended regression in #17917 which leads to only
repository admin commits being trusted. This PR restores the old logic.

Fix #18501

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-03 11:20:37 -05:00
zeripath 9c318a17f5
Add `GetUserTeams` (#18499) (#18531)
Backport #18499

* Correct use `UserID` in `SearchTeams`

- Use `UserID` in the `SearchTeams` function, currently it was useless
to pass such information. Now it does a INNER statement to `team_user`
which obtains UserID -> TeamID data.
- Make OrgID optional.
- Resolves #18484

* Seperate searching specific user

* Add condition back

* Use correct struct type

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-02-02 08:40:04 +00:00
Gusted d5027b6c09
Prevent NPE on partial match of compare URL and allow short SHA1 compare URLs (#18472) (#18473)
* Don't panic & allow shorter sha1 (#18472)

- Backport of #18472

* Improve comment

Co-authored-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-01-31 01:49:17 +02:00
Lunny Xiao 2f22337125
Fix broken when no commits and default branch is not master (#18423)
* Fix broken when no commits and default branch is not master

* Fix IsEmpty check

* Improve codes
2022-01-28 14:48:36 +08:00
Gusted 42991dc89a
Fix partial cloning a repo (#18373) (#18377)
* Fix partial cloning a repo (#18373)

- Backport from: #18373
- Backport isn't 1-1, because the frontport had a refactor in that area,
which v1.16 doesn't have.

* Include diff & use copy

* Add partial clone test

* patch

* Apply suggestions from code review

* globalArgs first

* avoid copy but make GlobalCMDArgs append first

* please linter

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-01-23 21:46:09 +00:00
wxiaoguang 160de9fbda
Fix mime-type detection for HTTP server (#18371) 2022-01-23 21:17:20 +08:00
Gusted d644289fcb
Backport: Disable content sniffing on `PlainTextBytes` (#18365)
- Backport of #18359
2022-01-23 01:58:09 +02:00
Peter Gardfjäll 89a3214352
Add MirrorUpdated field to Repository API type (#18267)
Add the last update time to the repository api type.

Close #18266
2022-01-18 13:18:30 +00:00
Ben Niemann 2871e58a01
Remove unneeded debug messages to stdout. (#18298)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-16 23:42:06 +08:00
techknowlogick 84145e45c5
Remove golang vendored directory (#18277)
* rm go vendor

* fix drone yaml

* add to gitignore
2022-01-14 18:16:05 -05:00
Lunny Xiao 35c3553870
Support webauthn (#17957)
Migrate from U2F to Webauthn

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-14 16:03:31 +01:00
6543 ff00b8688b
Fix NPE on try to get tag reference via API (#18245)
* fix npe

* rm gitRepo from Tag
2022-01-12 20:37:46 +00:00
luzpaz af92473920
Fix source typos (#18227)
Follow-up to #18219
2022-01-10 23:46:26 +08:00
luzpaz 8c647bf0f6
Fix various typos (#18219)
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon`

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 17:32:37 +08:00
Gusted 242dddfcb7
Remove `ioutil` (#18222)
- Don't use `ioutil` package anymore as it doesn't anything special
anymore since Go 1.16:

```
// As of Go 1.16, the same functionality is now provided
// by package io or package os, and those implementations
// should be preferred in new code.
```

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 09:48:13 +08:00
delvh 4f7764561a
Sort locales according to their names (#18211)
* Sort locales according to their names

* Fix documentation and sort case insensitive
2022-01-08 12:18:39 +00:00
zeripath 832f987d80
Restore setting of ctx.Repo.Mirror (#18205)
In #17933 repoAssignment no longer sets the ctx.Repo.Mirror field meaning that
attempting change mirror settings results in an NPE. This PR simply restores this.

Either we should remove this field or, we should set it. At present it seems simplest
to set it instead of going looking in the Data for the value although converting the
context to a bag of things may be the correct approach in the future.

Fix #18204

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-08 10:03:51 +08:00
Lunny Xiao a1c12fb0b3
Don't store assets modified time into generated files (#18193) 2022-01-06 21:33:17 -05:00
zeripath 21ed4fd8da
Add warning for BIDI characters in page renders and in diffs (#17562)
Fix #17514

Given the comments I've adjusted this somewhat. The numbers of characters detected are increased and include things like the use of U+300 to make à instead of à and non-breaking spaces.

There is a button which can be used to escape the content to show it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-07 02:18:52 +01:00
Gusted 4b3bfd7e89
Enable partial clone by default (#18195)
- Enable partial clones(which are by default disabled from git) by
default, unless configured otherwise.
- Resolves #18190
2022-01-06 06:38:38 +01:00
zeripath 1514e13bb8
Add option to convert CRLF to LF line endings for sendmail (#18075)
It appears that several versions of sendmail require that the mail is sent to them with
LF line endings instead of CRLF endings - which of course they will then convert back
to CRLF line endings to comply with the SMTP standard.

This PR adds another setting SENDMAIL_CONVERT_CRLF which will pass the message writer
through a filter. This will filter out and convert CRLFs to LFs before writing them
out to sendmail.

Fix #18024

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-06 01:43:45 +01:00