Commit Graph

3276 Commits

Author SHA1 Message Date
zeripath efd34d0d7d
Prevent intermittent failures in RepoIndexerTest (#19225 #19229) (#19228)
Backport #19225
Backport #19229

The RepoIndexerTest is failing with considerable frequency due to a race inherrent in
its design. This PR adjust this test to avoid the reliance on waiting for the populate
repo indexer to run and forcibly adds the repo to the queue. It then flushes the queue.

It may be worth separating out the tests somewhat by testing the Index function
directly away from the queue however, this forceful method should solve the current
problem.

Fix #19162

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-28 01:01:53 +02:00
Lunny Xiao 540541caa2
Hide sensitive content on admin panel progress monitor (#19218 & #19226) (#19231)
* Hide sensitive content on admin panel progress monitor (#19218)

Sanitize urls within git process descriptions.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>

* Do not include global arguments in process manager (#19226)

Backport #19226

The git command by default adds a number of global arguments. These are not
helpful to be displayed in the process manager and so should be skipped for
default process descriptions.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-03-27 18:21:59 +01:00
Robert Kaussow a13d64bf98
Bump goldmark to v1.4.11 (#19201) (#19203)
* Bump goldmark to v1.4.11

* fix go.sum

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add testcase

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-03-24 11:47:40 -04:00
zeripath e3d8e92bdc
Prevent redirect to Host (2) (#19175) (#19186)
Backport #19175

Unhelpfully Locations starting with `/\` will be converted by the
browser to `//` because ... well I do not fully understand. Certainly
the RFCs and MDN do not indicate that this would be expected. Providing
"compatibility" with the (mis)behaviour of a certain proprietary OS is
my suspicion. However, we clearly have to protect against this.

Therefore we should reject redirection locations that match the regular
expression: `^/[\\\\/]+`

Reference #9678

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 20:01:23 +00:00
zeripath b1a0a78a51
Redirect .wiki/* ui link to /wiki (#18831) (#19184)
Backport #18831

Redirect .wiki/* ui link to /wiki

fix #18590

Signed-off-by: a1012112796 <1012112796@qq.com>
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: a1012112796 <1012112796@qq.com>
2022-03-23 16:46:08 +00:00
Andrew Thornton 9c7d8b3096
Prevent start panic due to missing DotEscape function
Unfortunately #19169 causing a panic at startup in prod mode. This was hidden by dev
mode because the templates are compiled dynamically there. The issue is that DotEscape
is not in the original FuncMap at the time of compilation which causes a panic.

Ref #19169

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 16:09:57 +00:00
zeripath 93feb1a666
Fix the bug: deploy key with write access can not push (#19010) (#19182)
Backport #19010

Use DeployKeyID to replace the IsDeployKey, then CanWriteCode uses the DeployKeyID to check the write permission.

Fix #19009

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-23 13:44:41 +00:00
zeripath bb0e2121a3
Try to prevent autolinking of displaynames by email readers (#19169) (#19183)
Backport #19169

Unfortunately many email readers will (helpfully) detect url or url-like names and
automatically create links to them, even in HTML emails. This is not ideal when
usernames can have dots in them.

This PR tries to prevent this behaviour by sticking ZWJ characters between dots and
also set the meta tag to prevent format detection.

Not every email template has been changed in this way - just the activation emails but
it may be that we should be setting the above meta tag in all of our emails too.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 15:18:11 +02:00
zeripath d21b7fd3af
Clean paths when looking in Storage (#19124) (#19179)
Backport #19124

* Clean paths when looking in Storage

Ensure paths are clean for minio aswell as local storage.

Use url.Path not RequestURI/EscapedPath in storageHandler.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-03-23 09:23:00 +00:00
Norwin 743553f3e9
Cleanup protected branches when deleting users & teams (#19158) (#19174)
* Clean up protected_branches when deleting user

fixes #19094

* Clean up protected_branches when deleting teams

* fix issue

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-23 13:56:53 +08:00
zeripath a3ccbb5b7f
Ensure that setting.LocalURL always has a trailing slash (#19171) (#19177)
Backport #19171

Fix #19166

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-23 12:56:52 +08:00
techknowlogick f9ea4ab69a
Bump to build with go1.18 (#19120 et al) (#19127)
Backport #19120 
Backport #19099 
Backport #18874 
Backport #18420
Backport #19128
Backport #18270 

Bump to build with go1.18

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Jelle Hulter <jellehulter@gmail.com>
2022-03-19 18:46:47 +01:00
zeripath e6d46eeb55
Make migrations SKIP_TLS_VERIFY apply to git too (#19132) (#19141)
Backport #19132

Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command.

Fix #18998

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-19 16:20:26 +00:00
Gusted c1e6be47d7
Update golang.org/x/crypto (#19097) (#19098)
Backport #19097

* Update golang.org/x/crypto (#19097)

- Backport #19097

* Fix deprecation notice

* Backport workaround removal
2022-03-19 12:16:15 +00:00
Lunny Xiao 79a5e68816
Handle email address not exist (#19089) (#19121)
Backport #19089

* Handle email address not exist. (#19089)

* Fix lint about strings.Title

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-03-19 11:35:23 +00:00
6543 99861e3e06
Fix lfs bug (#19072) (#19080)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-14 15:59:54 +01:00
zeripath 0fe99cc00c
If rendering has failed due to a net.OpError stop rendering (attempt 2) (#19049) (#19056)
Backport #19049

Unfortunately #18642 does not work because a `*net.OpError` does not implement
the `Is` interface to make `errors.Is` work correctly - thus leading to the
irritating conclusion that a `*net.OpError` is not a `*net.OpError`.

Here we keep the `errors.Is` because presumably this will be fixed at
some point in the golang main source code but also we add a simply type
cast to also check.

Fix #18629

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 22:13:55 +01:00
zeripath 7aa29720f0
Improve SyncMirrors logging (#19045) (#19050)
Backport #19045

Yet another issue has come up where the logging from SyncMirrors does not provide
enough context. This PR adds more context to these logging events.

Related #19038

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 16:06:35 +01:00
6543 3e5c844a77
fix pam authorization (#19040) (#19047)
Backport #19040 

The PAM module has previously only checked the results of the authentication module.

However, in normal PAM practice most users will expect account module authorization to also be checked. Without doing this check in almost every configuration expired accounts and accounts with expired passwords will still be able to login.

This is likely to represent a significant gotcha in most configurations and cause most users configurations to be potentially insecure. Therefore we should add in the account authorization check.

## ⚠️ **BREAKING** ⚠️ 

Users of the PAM module who rely on account modules not being checked will need to change their PAM configuration.

However, as it is likely that the vast majority of users of PAM will be expecting account authorization to be checked in addition to authentication we should make this breaking change to make the default behaviour correct for the majority.

---

I suggest we backport this despite the BREAKING nature because of the surprising nature of this.

Thanks to @ysf for bringing this to our attention.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: ysf <34326+ysf@users.noreply.github.com>
2022-03-10 08:15:35 +00:00
zeripath dd52c08b74
Don't show context cancelled errors in attribute reader (#19006) (#19027)
Backport #19006

Fix #18997

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-08 19:20:37 +08:00
6543 87f8d37be5
git backend ignore replace objects (#18979) (#18980)
Co-authored-by: zeripath <art27@cantab.net>
2022-03-02 21:31:50 +00:00
Otto Richter (fnetX) a3f72303d1
Fix problem when self-assign notification (#18797) (#18976)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-02 20:11:55 +00:00
Otto Richter (fnetX) 578f19a682
Refactor admin user filter query parameters (#18965) (#18975)
Only pass `status_filter` on admin page
Use a more general method to pass query parameters, remove hard-coded keys

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-02 19:57:18 +01:00
Otto Richter (fnetX) 36e96e3481
Fix admin user list pagination (#18957) (#18960) 2022-03-01 23:00:03 +00:00
zeripath 5f9c18b2b3
Adjust error for already locked db and prevent level db lock on malformed connstr (#18923) (#18938)
Backport #18923

This PR adjusts the error returned when there is failure to lock the level db, and
permits a connections to the same leveldb where there is a different connection string.

Reference #18921
Reference #18917

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-28 15:45:38 +00:00
Gusted 4fb718d405
Don't treat BOM escape sequence as hidden character. (#18909) (#18910)
* Don't treat BOM escape sequence as hidden character. (#18909)

Backport #18909
2022-02-26 23:15:04 +01:00
Gusted df35049196
Allow adminstrator teams members to see other teams (#18918) (#18919)
Allow adminstrator teams members to see other teams (#18918)
2022-02-26 22:45:34 +01:00
silverwind ce75461380
Correctly link URLs to users/repos with dashes, dots or underscores (#18890) (#18908)
* Add tests for references with dashes

This commit adds tests for full URLs referencing repos names and user
names containing a dash.

* Extend regex to match URLs to repos/users with dashes

Co-authored-by: Alexander Neumann <62751754+rtpt-alexanderneumann@users.noreply.github.com>
2022-02-26 06:45:09 +01:00
Otto Richter (fnetX) 6039138323
Fix redirect when using lowercase reponame (#18775) (#18902)
* Previously,  `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL
* In other words both
   * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png`
   * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png`
were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png`
This isn't the case after  #17551. Specifically because of this [line](cbd5eecd14/modules/context/repo.go (L860)).

Co-authored-by: Ghanem <37152329+AbdulrhmnGhanem@users.noreply.github.com>
2022-02-26 08:16:22 +08:00
zeripath 382101ecc7
In disk_channel queues synchronously push to disk on shutdown (#18415) (#18788)
Partial Backport of #18415

Instead of using an asynchronous goroutine to push to disk on shutdown
just close the datachan and immediately push to the disk.

Prevents messages of incompletely flushed queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-22 20:08:35 +08:00
zeripath bec566282e
Immediately Hammer if second kill is sent (#18823) (#18826)
Backport #18823

Currently Gitea will wait for HammerTime or nice shutdown if kill -1 or kill -2
is sent. We should just immediately hammer if there is a second kill.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-20 01:37:52 +08:00
zeripath fa9be55018
Fix panic in EscapeReader (#18820) (#18821)
Backport #18820

There is a potential panic due to a mistaken resetting of the length parameter when
multibyte characters go over a read boundary.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-19 23:34:32 +08:00
zeripath c876124efe
Ensure git tag tests and others create test repos in tmpdir (#18447) (#18767)
Backport #18447

* Ensure git tag tests and other create test repos in tmpdir

There are a few places where tests appear to reuse testing repos which
causes random CI failures.

This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.

Fix #18444

* Change log root for integration tests to use the REPO_TEST_DIR

There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.

Fix #18077

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2022-02-15 19:16:38 +08:00
zeripath 7ebc3da7cb
Prevent dangling GetAttribute calls (#18754) (#18755)
* Prevent dangling GetAttribute calls

It appears possible that there could be a hang due to unread data from the
repo-attribute command pipes. This PR simply closes these during the defer.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* move close into the defer

Signed-off-by: Andrew Thornton <art27@cantab.net>

* lets try again

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-14 15:27:55 -05:00
zeripath 2e36ba0a00
Send mail to issue/pr assignee/reviewer also when OnMention is set (#18707) (#18765)
Backport #18707

Addresses #17892, where emails notifications are not sent to assignees (issue and PR) and reviewers (PR) when they have the email setting Only email on mention enabled.

From the user experience perspective, when a user gets a issue/PR assigned or a PR review request, he/she would expect to be implicitly mentioned since the assignment or request is personal and targeting a single person only. Thus I see #17892 as a bug. Could we therefore mark this ticket as such?

The changed code just explicitly checks for the EmailNotificationsOnMention setting beside the existing EmailNotificationsEnabled check. Too rude?

@lunny mentioned a mock mail server for tests, is there something ready. How could I make use of it?

See #12774 (comment)

Fix #17892

Co-authored-by: flozzone <flozzone@gmail.com>
2022-02-14 21:13:41 +08:00
Lunny Xiao 913d6f3ff3
Fix isempty detection of git repository (#18746) (#18750)
* Fix isempty detection of git repository

* Fix IsEmpty check
2022-02-14 00:33:35 +08:00
zeripath 2e317d3f6e
Prevent security failure due to bad APP_ID (#18678) (#18682)
Backport #18678

WebAuthn may cause a security exception if the provided APP_ID is not allowed for the
current origin. Therefore we should reattempt authentication without the appid
extension.

Also we should allow [u2f] as-well as [U2F] sections.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-10 16:17:44 +01:00
zeripath 38fc6c75f3
Restart zero worker if there is still work to do (#18658) (#18672)
* Restart zero worker if there is still work to do (#18658)

Backport #18658

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix #18607

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/queue/workerpool.go
2022-02-08 23:28:21 +02:00
zeripath d4a075d738
If rendering has failed due to a net.OpError stop rendering (#18642) (#18645)
Backport #18642

When a net.OpError occurs during rendering the underlying connection is essentially
dead and therefore attempting to render further data will only cause further errors.

Therefore in serverErrorInternal detect if the passed in error is an OpError and
if so do not attempt any further rendering.

Fix #18629

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-07 09:25:05 +08:00
zeripath a13fb154ae
Stop logging an error when notes are not found (#18626) (#18635)
Backport #18626

This is an unnecessary logging event.

Fix #18616

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-06 09:43:15 +00:00
zeripath 36c66303df
Only attempt to flush queue if the underlying worker pool is not finished (#18593) (#18620)
* Only attempt to flush queue if the underlying worker pool is not finished (#18593)

Backport #18593

There is a possible race whereby a worker pool could be cancelled but yet the
underlying queue is not empty. This will lead to flush-all cycling because it
cannot empty the pool.

* On shutdown of Persistant Channel Queues close datachan and empty

Partial Backport #18415

Although we attempt to empty the datachan in queues - due to
races we are better off just closing the channel and forcibly emptying
it in shutdown.

Fix #18618

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Move zero workers warning to debug

Fix #18617

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/queue/manager.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update modules/queue/manager.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

Co-authored-by: Gusted <williamzijl7@hotmail.com>
2022-02-06 14:55:44 +08:00
zeripath bc196a35e1
Collaborator trust model should trust collaborators (#18539) (#18557)
Backport #18539

There was an unintended regression in #17917 which leads to only
repository admin commits being trusted. This PR restores the old logic.

Fix #18501

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-02-03 11:20:37 -05:00
zeripath 9c318a17f5
Add `GetUserTeams` (#18499) (#18531)
Backport #18499

* Correct use `UserID` in `SearchTeams`

- Use `UserID` in the `SearchTeams` function, currently it was useless
to pass such information. Now it does a INNER statement to `team_user`
which obtains UserID -> TeamID data.
- Make OrgID optional.
- Resolves #18484

* Seperate searching specific user

* Add condition back

* Use correct struct type

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-02-02 08:40:04 +00:00
Gusted d5027b6c09
Prevent NPE on partial match of compare URL and allow short SHA1 compare URLs (#18472) (#18473)
* Don't panic & allow shorter sha1 (#18472)

- Backport of #18472

* Improve comment

Co-authored-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-01-31 01:49:17 +02:00
Lunny Xiao 2f22337125
Fix broken when no commits and default branch is not master (#18423)
* Fix broken when no commits and default branch is not master

* Fix IsEmpty check

* Improve codes
2022-01-28 14:48:36 +08:00
Gusted 42991dc89a
Fix partial cloning a repo (#18373) (#18377)
* Fix partial cloning a repo (#18373)

- Backport from: #18373
- Backport isn't 1-1, because the frontport had a refactor in that area,
which v1.16 doesn't have.

* Include diff & use copy

* Add partial clone test

* patch

* Apply suggestions from code review

* globalArgs first

* avoid copy but make GlobalCMDArgs append first

* please linter

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-01-23 21:46:09 +00:00
wxiaoguang 160de9fbda
Fix mime-type detection for HTTP server (#18371) 2022-01-23 21:17:20 +08:00
Gusted d644289fcb
Backport: Disable content sniffing on `PlainTextBytes` (#18365)
- Backport of #18359
2022-01-23 01:58:09 +02:00
Peter Gardfjäll 89a3214352
Add MirrorUpdated field to Repository API type (#18267)
Add the last update time to the repository api type.

Close #18266
2022-01-18 13:18:30 +00:00
Ben Niemann 2871e58a01
Remove unneeded debug messages to stdout. (#18298)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-16 23:42:06 +08:00