Commit Graph

3305 Commits

Author SHA1 Message Date
zeripath 736b7b25a4
Set Setpgid on child git processes (#19865) (#19881) 2022-06-03 23:39:15 -04:00
zeripath daf14b275a
Ensure responses are context.ResponseWriters (#19843) (#19859)
* Ensure responses are context.ResponseWriters (#19843)

Backport #19843

In order for web.Wrap to be able to detect if a response has been written
we need to wrap any non-context.ResponseWriters as a such. Otherwise
responses will be incorrectly detected as non-written to and handlers can
double run.

In the case of GZip this handler will change the response to a non-context.RW
and this failure to correctly detect response writing causes fallthrough and
a NPE.

Fix #19839

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix test

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-06-03 17:38:29 -04:00
Lauris BH 0e9499ada7
Fix raw endpoint PDF file headers (#19825) (#19826) 2022-05-28 18:40:03 +03:00
zeripath be99eb26a2
Detect truncated utf-8 characters at the end of content as still representing utf-8 (#19773) (#19774)
Backport #19773

Our character detection algorithm can potentially incorrectly detect utf-8 as iso-8859-x
if there is a truncated character at the end of the partially read file.

This PR changes the detection algorithm to truncated utf8 characters at the end of the
buffer.

Fix #19743

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-05-21 22:26:08 +08:00
silentcodeg fe9458591a
[doctor] pq: syntax error at or near "." quote user table name (#19765) (#19770)
Backport #19765
2022-05-21 02:00:52 +02:00
6543 8f44d00f22
Delete user related oauth stuff on user deletion too (#19677) (#19680)
Backport (#19677)

* delete user related oauth stuff on user deletion too

* extend doctor check-db-consistency

* make it build for v1.16.x
2022-05-12 13:32:48 +02:00
singuliere f4fb8dbc87
[doctor] Add check/fix for bogus action rows (#19656) (#19669)
Co-authored-by: Loïc Dachary <loic@dachary.org>

Conflicts:
	models/consistency_test.go
	 trivial context conflict.
2022-05-10 16:05:34 +03:00
Lunny Xiao c7c18e0eb2
Revert "Add finalizers to ensure that repos are closed and blobreaders are closed (#19495) (#19496)" (#19659)
This reverts commit 88da50674f.

because it caused a memleak
2022-05-09 13:03:44 +02:00
Jimmy Praet c8a83ace59
Only show accessible teams in dashboard dropdown list (#19642) (#19645)
Fixes #19637
2022-05-07 17:50:34 +02:00
wxiaoguang 46637b1164
Call MultipartForm.RemoveAll when request finishes (#19606) (#19607) 2022-05-05 16:13:59 +02:00
zeripath 88da50674f
Add finalizers to ensure that repos are closed and blobreaders are closed (#19495) (#19496)
It may be prudent to add runtime finalizers to the git.Repository and
git.blobReader objects to absolutely ensure that these are both properly
cancelled, cleaned and closed out.

This commit is a backport of an extract from #19448

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-05-02 20:44:45 +08:00
6543 755d8e21ad
Migration: only write commit-graph if wiki clone was successfull (#19563) (#19568) 2022-05-01 00:22:42 +02:00
Jimmy Praet 7c0bf06d96
Respect DefaultUserIsRestricted system default when creating new user (#19310 ) (#19560) 2022-04-30 15:00:14 +02:00
wxiaoguang b86606fa38
Support `hostname:port` to pass host matcher's check (#19543) (#19544)
Backport #19543 
hostmatcher: split the hostname from the hostname:port string, use the correct hostname to do the match.
2022-04-29 01:41:58 +08:00
zeripath 74602bb487
Prevent intermittent race in attribute reader close (#19537) (#19539)
Backport #19537

There is a potential rare race possible whereby the c.running channel could
be closed twice. Looking at the code I do not see a need for this c.running
channel and therefore I think we can remove this. (I think the c.running
might have been some attempt to prevent a hang but the use of os.Pipes should
prevent that.)

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-04-28 17:00:01 +02:00
Gusted 1465e0cbb2
Fix 64-bit atomic operations on 32-bit machines (#19531) (#19532)
- Backport #19531
  - Doing 64-bit atomic operations on 32-bit machines is a bit tricky by golang, as they can only be done under certain set of conditions(https://pkg.go.dev/sync/atomic#pkg-note-BUG).
  - This PR fixes such case whereby the conditions weren't met, it moves the int64 to the first field of the struct, which will 64-bit operations happening on this property on 32-bit machines.
  - Resolves #19518
2022-04-27 10:32:28 -05:00
zeripath 39a0db6ecf
Prevent dangling archiver goroutine (#19516) (#19526)
Backport #19516

Within doArchive there is a service goroutine that performs the
archiving function.  This goroutine reports its error using a `chan
error` called `done`. Prior to this PR this channel had 0 capacity
meaning that the goroutine would block until the `done` channel was
cleared - however there are a couple of ways in which this channel might
not be read.

The simplest solution is to add a single space of capacity to the
goroutine which will mean that the goroutine will always complete and
even if the `done` channel is not read it will be simply garbage
collected away.

(The PR also contains two other places when setting up the indexers
which do not leak but where the blocking of the sending goroutine is
also unnecessary and so we should just add a small amount of capacity
and let the sending goroutine complete as soon as it can.)

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>

Co-authored-by: 6543 <6543@obermui.de>
2022-04-27 16:05:52 +08:00
6543 8d7f1e430a
User specific repoID or xorm builder conditions for issue search (#19475) (#19476) 2022-04-25 15:28:47 +02:00
Pilou a6b32adc45
[doctor] authorized-keys: fix displayed check name (backport #19464) (#19484)
The registered check name is authorized-keys, not authorized_keys.
2022-04-25 13:45:18 +02:00
6543 1d665da32f
Prevent dangling cat-file calls (goroutine alternative) (#19454) (#19466)
If an `os/exec.Command` is passed non `*os.File` as an input/output, go
will create `os.Pipe`s and wait for their closure in `cmd.Wait()`.  If
the code following this is responsible for closing `io.Pipe`s or other
handlers then on process death from context cancellation the `Wait` can
hang.

There are two possible solutions:

1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these.
2. create a goroutine waiting on the context cancellation that will close the inputs.

This PR provides the second option - which is a simpler change that can
be more easily backported.

Closes #19448

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2022-04-22 16:58:50 +01:00
Gusted 09adc26eb6
Set correct PR status on 3way on conflict checking (#19457) (#19458)
- Backport #19457
  - When 3-way merge is enabled for conflict checking, it has a new interesting behavior that it doesn't return any error when it found a conflict, so we change the condition to not check for the error, but instead check if conflictedfiles is populated, this fixes a issue whereby PR status wasn't correctly on conflicted PR's.
  - Refactor the mergeable property(which was incorrectly set and lead me this bug) to be more maintainable.
  - Add a dedicated test for conflicting checking, so it should prevent future issues with this.
  - Ref: Fix the latest error for https://gitea.com/gitea/go-sdk/pulls/579

Co-authored-by: zeripath <art27@cantab.net>
2022-04-22 09:11:42 +08:00
6543 297346a762
RepoAssignment ensure to close before overwrite (#19449) (#19460)
* check if GitRepo already open and close if

* Only run RepoAssignment once
2022-04-21 18:55:44 +02:00
6543 a785c46ca8
Add uploadpack.allowAnySHA1InWant to allow --filter=blob:none with older git clients (#19430) (#19438)
Older git clients need uploadpack.allowAnySHA1InWant if partial cloning is allowed.

Fix #19118

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2022-04-20 20:54:36 +02:00
zeripath 08eecba32b
When updating mirror repo intervals by API reschedule next update too (#19429) (#19433)
Backport #19429

When a mirror repo interval is updated by the UI it is rescheduled with that interval
however the API does not do this. The API also lacks the enable_prune option.

This PR adds this functionality in to the API Edit Repo endpoint.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-04-20 16:04:26 +02:00
Lunny Xiao 9b4746967c
Only request write when necessary (#18657) (#19422)
* Only request write when necessary

- Only request write for `INTERNAL_TOKEN_URI` when no token was found.
- Resolves #18655

* Fix perm

* Update setting.go

* Update setting.go

* Update setting.go

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2022-04-19 12:10:24 -04:00
silverwind 61c7732e12
Disable service worker by default (#18914) (#19342)
The service worker causes a lot of issues with JS errors after instance
upgrades while not bringing any real performance gain over regular HTTP
caching.

Disable it by default for this reason. Maybe later we can remove it
completely, as I simply see no benefit in having it.
2022-04-07 20:08:24 +02:00
wxiaoguang 57c2ca7f26
Fix invalid CSRF token bug, make sure CSRF tokens can be up-to-date (#19338)
There was a bug that the CSRF token wouldn't in 24h. This fix just does what the CSRF function comment says: If this request is a GET request, it will generate a new token. Then the CSRF token can be kept up-to-date.
2022-04-06 23:47:58 +08:00
Lunny Xiao 0704009dd7
Revert the minimal golang version requirement from 1.17 to 1.16 and add a warning in Makefile (#19319)
* Revert the minimal golang version requirement from 1.17 to 1.16 and add a warning in Makefile

* Apply suggestions from code review

Co-authored-by: John Olheiser <john.olheiser@gmail.com>

* 1.16

* Update modules/util/net.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* correct bool conditional

yay tests for catching this :)

* Update hostmatcher.go

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
2022-04-05 13:32:24 -04:00
zeripath db43f63c53
Use full output of git show-ref --tags to get tags for PushUpdateAddTag (#19235) (#19236)
* Use full output of git show-ref --tags to get tags for PushUpdateAddTag (#19235)

Strangely #19038 appears to relate to an issue whereby a tag appears to
be listed in `git show-ref --tags` but then does not appear when `git
show-ref --tags -- short_name` is called.

As a solution though I propose to stop the second call as it is
unnecessary and only likely to cause problems.

I've also noticed that the tags calls are wildly inefficient and aren't using the common cat-files - so these have been added.

I've also noticed that the git commit-graph is not being written on mirroring - so I've also added writing this to the migration which should improve mirror rendering somewhat.

Fix #19038

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>

* fix rebase relict

Co-authored-by: 6543 <6543@obermui.de>
2022-03-29 23:19:57 +03:00
zeripath efd34d0d7d
Prevent intermittent failures in RepoIndexerTest (#19225 #19229) (#19228)
Backport #19225
Backport #19229

The RepoIndexerTest is failing with considerable frequency due to a race inherrent in
its design. This PR adjust this test to avoid the reliance on waiting for the populate
repo indexer to run and forcibly adds the repo to the queue. It then flushes the queue.

It may be worth separating out the tests somewhat by testing the Index function
directly away from the queue however, this forceful method should solve the current
problem.

Fix #19162

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-28 01:01:53 +02:00
Lunny Xiao 540541caa2
Hide sensitive content on admin panel progress monitor (#19218 & #19226) (#19231)
* Hide sensitive content on admin panel progress monitor (#19218)

Sanitize urls within git process descriptions.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>

* Do not include global arguments in process manager (#19226)

Backport #19226

The git command by default adds a number of global arguments. These are not
helpful to be displayed in the process manager and so should be skipped for
default process descriptions.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-03-27 18:21:59 +01:00
Robert Kaussow a13d64bf98
Bump goldmark to v1.4.11 (#19201) (#19203)
* Bump goldmark to v1.4.11

* fix go.sum

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add testcase

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-03-24 11:47:40 -04:00
zeripath e3d8e92bdc
Prevent redirect to Host (2) (#19175) (#19186)
Backport #19175

Unhelpfully Locations starting with `/\` will be converted by the
browser to `//` because ... well I do not fully understand. Certainly
the RFCs and MDN do not indicate that this would be expected. Providing
"compatibility" with the (mis)behaviour of a certain proprietary OS is
my suspicion. However, we clearly have to protect against this.

Therefore we should reject redirection locations that match the regular
expression: `^/[\\\\/]+`

Reference #9678

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 20:01:23 +00:00
zeripath b1a0a78a51
Redirect .wiki/* ui link to /wiki (#18831) (#19184)
Backport #18831

Redirect .wiki/* ui link to /wiki

fix #18590

Signed-off-by: a1012112796 <1012112796@qq.com>
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: a1012112796 <1012112796@qq.com>
2022-03-23 16:46:08 +00:00
Andrew Thornton 9c7d8b3096
Prevent start panic due to missing DotEscape function
Unfortunately #19169 causing a panic at startup in prod mode. This was hidden by dev
mode because the templates are compiled dynamically there. The issue is that DotEscape
is not in the original FuncMap at the time of compilation which causes a panic.

Ref #19169

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 16:09:57 +00:00
zeripath 93feb1a666
Fix the bug: deploy key with write access can not push (#19010) (#19182)
Backport #19010

Use DeployKeyID to replace the IsDeployKey, then CanWriteCode uses the DeployKeyID to check the write permission.

Fix #19009

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-23 13:44:41 +00:00
zeripath bb0e2121a3
Try to prevent autolinking of displaynames by email readers (#19169) (#19183)
Backport #19169

Unfortunately many email readers will (helpfully) detect url or url-like names and
automatically create links to them, even in HTML emails. This is not ideal when
usernames can have dots in them.

This PR tries to prevent this behaviour by sticking ZWJ characters between dots and
also set the meta tag to prevent format detection.

Not every email template has been changed in this way - just the activation emails but
it may be that we should be setting the above meta tag in all of our emails too.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-23 15:18:11 +02:00
zeripath d21b7fd3af
Clean paths when looking in Storage (#19124) (#19179)
Backport #19124

* Clean paths when looking in Storage

Ensure paths are clean for minio aswell as local storage.

Use url.Path not RequestURI/EscapedPath in storageHandler.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-03-23 09:23:00 +00:00
Norwin 743553f3e9
Cleanup protected branches when deleting users & teams (#19158) (#19174)
* Clean up protected_branches when deleting user

fixes #19094

* Clean up protected_branches when deleting teams

* fix issue

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-23 13:56:53 +08:00
zeripath a3ccbb5b7f
Ensure that setting.LocalURL always has a trailing slash (#19171) (#19177)
Backport #19171

Fix #19166

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-23 12:56:52 +08:00
techknowlogick f9ea4ab69a
Bump to build with go1.18 (#19120 et al) (#19127)
Backport #19120 
Backport #19099 
Backport #18874 
Backport #18420
Backport #19128
Backport #18270 

Bump to build with go1.18

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Jelle Hulter <jellehulter@gmail.com>
2022-03-19 18:46:47 +01:00
zeripath e6d46eeb55
Make migrations SKIP_TLS_VERIFY apply to git too (#19132) (#19141)
Backport #19132

Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command.

Fix #18998

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-19 16:20:26 +00:00
Gusted c1e6be47d7
Update golang.org/x/crypto (#19097) (#19098)
Backport #19097

* Update golang.org/x/crypto (#19097)

- Backport #19097

* Fix deprecation notice

* Backport workaround removal
2022-03-19 12:16:15 +00:00
Lunny Xiao 79a5e68816
Handle email address not exist (#19089) (#19121)
Backport #19089

* Handle email address not exist. (#19089)

* Fix lint about strings.Title

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-03-19 11:35:23 +00:00
6543 99861e3e06
Fix lfs bug (#19072) (#19080)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-03-14 15:59:54 +01:00
zeripath 0fe99cc00c
If rendering has failed due to a net.OpError stop rendering (attempt 2) (#19049) (#19056)
Backport #19049

Unfortunately #18642 does not work because a `*net.OpError` does not implement
the `Is` interface to make `errors.Is` work correctly - thus leading to the
irritating conclusion that a `*net.OpError` is not a `*net.OpError`.

Here we keep the `errors.Is` because presumably this will be fixed at
some point in the golang main source code but also we add a simply type
cast to also check.

Fix #18629

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 22:13:55 +01:00
zeripath 7aa29720f0
Improve SyncMirrors logging (#19045) (#19050)
Backport #19045

Yet another issue has come up where the logging from SyncMirrors does not provide
enough context. This PR adds more context to these logging events.

Related #19038

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-03-10 16:06:35 +01:00
6543 3e5c844a77
fix pam authorization (#19040) (#19047)
Backport #19040 

The PAM module has previously only checked the results of the authentication module.

However, in normal PAM practice most users will expect account module authorization to also be checked. Without doing this check in almost every configuration expired accounts and accounts with expired passwords will still be able to login.

This is likely to represent a significant gotcha in most configurations and cause most users configurations to be potentially insecure. Therefore we should add in the account authorization check.

## ⚠️ **BREAKING** ⚠️ 

Users of the PAM module who rely on account modules not being checked will need to change their PAM configuration.

However, as it is likely that the vast majority of users of PAM will be expecting account authorization to be checked in addition to authentication we should make this breaking change to make the default behaviour correct for the majority.

---

I suggest we backport this despite the BREAKING nature because of the surprising nature of this.

Thanks to @ysf for bringing this to our attention.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: ysf <34326+ysf@users.noreply.github.com>
2022-03-10 08:15:35 +00:00
zeripath dd52c08b74
Don't show context cancelled errors in attribute reader (#19006) (#19027)
Backport #19006

Fix #18997

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-08 19:20:37 +08:00
6543 87f8d37be5
git backend ignore replace objects (#18979) (#18980)
Co-authored-by: zeripath <art27@cantab.net>
2022-03-02 21:31:50 +00:00