Commit Graph

9305 Commits

Author SHA1 Message Date
Han-Wen NIenhuys d174684273 Merge "PatchApplier: Check for existence of src/dest files before any operation" 2023-03-31 06:24:32 -04:00
Nitzan Gur-Furman 903645835b PatchApplier: Check for existence of src/dest files before any operation
Change-Id: Ia3ec0ce1af65114b48669157a934f70f1e22fd37
Bug: Google b/271474227
2023-03-31 06:22:29 -04:00
Martin Fick c5617711a1 Revert "RefDirectory: Throw exception if CAS of packed ref list fails"
This reverts commit 9c33f7364d.

Reason for revert: This change was based on the false claim that the
packedrefs file lock is held while the CAS is being done, but it is
actually released before the CAS (the in memory lock is still held,
however that does not prevent external actors from updating the
packedrefs files and then another thread from subsequently re-reading it
and updating the in memory packedRefList). Although reverting this
change can cause the CAS to fail, it should not actually matter since
the failure would indicate that another thread has already updated the
in memory packedRefList to either the same version this thread was
trying to update it too, or to a more recent version. Either way,
failing the CAS is then appropriate and should not be problematic.

Although this change reverts the code in the RefDirectory class, it
keeps the "improvements" to the test so that it continues to pass
reliably. The reason for the quotes around the word "improvements" is
because I believe the test alteration actually dramatically changes the
intent of the test, and that the original intent of the test is
untestable with the GC and RefDirectory classes as is.

Change-Id: I3acee7527bb542996dcdfaddfb2bdb45ec444db5
Signed-off-by: Martin Fick <quic_mfick@quicinc.com>
2023-03-30 18:04:16 -04:00
Kaushik Lingarkar 5ae8d28faa RefDirectory.delete: Prevent failures when packed-refs is outdated
The in-memory copy of packed refs might be outdated by the time the
packed-refs lock is acquired, so ensure the one read from disk is
used after acquiring the lock to prevent commit packed-refs from
throwing an exception. As a side-effect, since this updates the
in-memory copy of packed-refs when it is re-read from disk, it can
prevent other callers needing to re-read if it had changed.

Change-Id: I724c866b330b397e955b5eb04b259eedd9911e93
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:32:50 +02:00
Kaushik Lingarkar 4f7627be24 RefDirectory.pack: Only rely on packed refs from disk
Since packed-refs is read from disk anyway, don't rely on the
in-memory copy as that is racy and if outdated, could result in
commit of pack-refs throwing an exception. This change also avoids
a possible unnecessary double read of packed-refs from disk.

Change-Id: I684a64991f53f8bdad58bbd248aae6522d11267d
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:32:50 +02:00
Kaushik Lingarkar 4652dd956e RefDirectory: Make pack() and commitPackRefs() void
There are no more callers (since Iae71cb3) of these methods that need
the returned value. These methods should not have been returning
anything in the first place as that can introduce bugs such as the
one described in Iae71cb3.

Change-Id: I1d083a91603da803a106cfb1506925a82c2ef809
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:32:50 +02:00
Kaushik Lingarkar 33c00f3347 Implement a snapshotting RefDirectory for use in request scope
Introduce a SnapshottingRefDirectory class which allows users to get
a snapshot of the ref database and use it in a request scope (for
example a Gerrit query) instead of having to re-read packed-refs
several times in a request.

This can potentially be further improved to avoid scanning/reading a
loose ref several times in a request. This would especially help
repeated lookups of a packed ref, where we check for the existence of
a loose ref each time.

Change-Id: I634b92877f819f8bf36a3b9586bbc1815108189a
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:25:53 +02:00
Kaushik Lingarkar d7400517bf Fix RefDirectoryTest.testGetRefs_LooseSorting_Bug_348834
Since the first attempt to read a ref is not expected to trigger
a RefsChangedEvent, update the test to ensure 'lastNotifiedModCnt'
is not 0 before we start the actual work. The test has been passing
luckily because createBareRepository in setUp() happens to bump
'lastNotifiedModCnt'.

Change-Id: Ibd981f677920e8c3b965aa742fe669c42b8c1c93
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:25:53 +02:00
Kaushik Lingarkar 47f2f3613c PackedBatchRefUpdate: Ensure updates are applied on latest packed refs
In the window between refs being packed (via refDb.pack) and obtaining
updates (via applyUpdates), packed-refs may have been updated by another
actor and relying on the previously read contents may lead to losing the
updates done by the other actor. To help avoid this, read packed-refs
from disk to ensure we have the latest copy after it is locked and
before committing updates to it.

Bug: 581641
Change-Id: Iae71cb30830b307d0df929c9131911ee476c711c
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:25:52 +02:00
Thomas Wolf bf9b704d19 Record my e-mail change in .mailmap
Change-Id: I3bf867f2d91c92315d77015bdc87cd60e4ece78a
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-03-29 21:06:11 +02:00
Thomas Wolf 62a3bf8b96 [releng] Bump japicmp base version
To 6.5.0.202303070854-r.

Change-Id: Ifdf7c85871263ef3cc810aedc207b3598ba01d86
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-03-29 21:06:11 +02:00
Thomas Wolf d33b9ab5d3 PatchApplier: missing @since, and minor formatting
Change-Id: I561ca2f522579571b29d3e6f35f24e201d1c1663
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-03-29 21:06:11 +02:00
Ivan Frade a7f8f66523 Merge "BasePackFetchConnection: support negotiationTip feature" 2023-03-28 19:29:17 -04:00
Ronald Bhuleskar 738dacb7fb BasePackFetchConnection: support negotiationTip feature
By default, Git will report, to the server, commits reachable from all local refs to find common commits in an attempt to reduce the size of the to-be-received packfile. If specified with negotiation tip, Git will only report commits reachable from the given tips. This is useful to speed up fetches when the user knows which local ref is likely to have commits in common with the upstream ref being fetched.

When negotation-tip is on, use the wanted refs instead of all refs as source of the "have" list to send.

This is controlled by the `fetch.usenegotationtip` flag, false by default. This works only for programmatic fetches and there is no support for it yet in the CLI.

Change-Id: I19f8fe48889bfe0ece7cdf78019b678ede5c6a32
2023-03-28 14:29:54 -07:00
Ivan Frade ff74708466 PatchApplierTest: Remove test data with Apache license
Some test data introduced in [1] includes code with Apache
license. This triggers warnings in license analyzers.

Remove the licensed files and the test case as a quick relief. We
should restore the test with appropiate data in a follow-up change.

[1] https://git.eclipse.org/r/c/jgit/jgit/+/197724

Change-Id: I42670dc7d994f77d2c7f2c2156bcf1e112374022
2023-03-28 05:58:28 -04:00
Han-Wen NIenhuys 5166ded098 Merge "Fix PatchApplier error handling." 2023-03-28 05:51:18 -04:00
Nitzan Gur-Furman 3a913a8c34 Fix PatchApplier error handling.
1. For general errors, throw IOException instead of wrapping them with
PatchApplyException. The wrapping was moved (back) to ApplyCommand.
2. For file specific errors, log the errors as part of
PatchApplier::Result.
3. Change applyPatch() to receive the parsed Patch object, so the caller
can decide how to handle parsing errors.

Background: this utility class was extracted from ApplyCommand on V6.4.0.
During the extraction, we left the exception wrapping by
PatchApplyException intact. This attitude made it harder for the callers to
distinguish between the actual error causes.

Change-Id: Ib0f2b5e97a13df2339d8b65f2fea1c819c161ac3
2023-03-28 11:18:08 +02:00
Matthias Sohn a30c1da323 Merge branch 'stable-6.5'
* stable-6.5:
  Ensure FileCommitGraph scans commit-graph file if it already exists

Change-Id: I5218ff5214222c7d6d96e452cf427eea1f20c316
2023-03-27 11:02:52 +02:00
kylezhao 827849017d Ensure FileCommitGraph scans commit-graph file if it already exists
When commit-graph file already exists in the repository, a newly
created FileCommitGraph didn't scan CommitGraph until the file was
modified, resulting in wrong result.

Change-Id: Ic85676f2d3b6a88f3ae28d4065729926b6fb2f23
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-03-27 10:51:07 +02:00
Matthias Sohn f9e76d040e Merge branch 'stable-6.5'
* stable-6.5:
  GC: Close File.lines stream

Change-Id: I498359a229b9bd3b62415870d3fdeae281977aa6
2023-03-23 09:08:05 +01:00
Matthias Sohn 67fcf76b4b Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  GC: Close File.lines stream

Change-Id: I7e3a4b3671e779fd62062c4e10d224f432e39b54
2023-03-23 09:07:33 +01:00
Matthias Sohn cd2dc85f31 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  GC: Close File.lines stream

Change-Id: I99455916d447f5dffed85e9a5c1d51b323f07a16
2023-03-23 09:07:09 +01:00
Matthias Sohn 137efda0ba Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  GC: Close File.lines stream

Change-Id: Id93b1933a5ce1ede9eb388c9fd54a4b3749694bf
2023-03-23 09:06:43 +01:00
Matthias Sohn b118e7b4c4 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  GC: Close File.lines stream

Change-Id: Ia2be0b05ed860125a388b01d6c291832f08dd990
2023-03-23 09:06:16 +01:00
Matthias Sohn 5b16c8ae15 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  GC: Close File.lines stream

Change-Id: I2f9e6da5584a40bb4b4efed0b87ae456f119d757
2023-03-23 09:05:42 +01:00
Matthias Sohn 55164c43b9 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  GC: Close File.lines stream

Change-Id: Ib473750e5a3ad3d74b0cb41f25052890f50a975c
2023-03-23 09:04:50 +01:00
Xing Huang 3212c8fa38 GC: Close File.lines stream
From File#lines javadoc: The returned stream from File Lines
encapsulates a Reader. If timely disposal of file system resources is
required, the try-with-resources construct should be used to ensure
that the stream's close method is
invoked after the stream operations are completed.

Wrap File.lines with try-with-resources.

Change-Id: I82c6faa3ef1083f6c7e964f96e9540b4db18eee8
Signed-off-by: Xing Huang <xingkhuang@google.com>
(cherry picked from commit 172a207945)
2023-03-23 08:19:26 +01:00
Xing Huang 172a207945 GC: Close File.lines stream
From File#lines javadoc: The returned stream from File Lines
encapsulates a Reader. If timely disposal of file system resources is
required, the try-with-resources construct should be used to ensure
that the stream's close method is
invoked after the stream operations are completed.

Wrap File.lines with try-with-resources.

Signed-off-by: Xing Huang <xingkhuang@google.com>
Change-Id: I82c6faa3ef1083f6c7e964f96e9540b4db18eee8
Signed-off-by: Xing Huang <xingkhuang@google.com>
2023-03-21 17:57:12 -05:00
Matthias Sohn 228e4de484 Merge branch 'stable-6.5'
* stable-6.5:
  Rerun flaky tests 3 times
  Prepare 6.5.1-SNAPSHOT builds
  JGit v6.5.0.202303070854-r
  Ignore generated org.eclipse.jgit.benchmarks/dependency-reduced-pom.xml
  [sshd] Fix calculation of timeout in AbstractClientProxyConnector
  Silence API error raised for removed BranchRebaseMode#PRESERVE

Change-Id: Ie615980c81371ee26b2395e67e026bbd17422fbd
2023-03-07 16:41:19 +01:00
Matthias Sohn c6b979444b Rerun flaky tests 3 times
This can help to reduce time spent on rebuilds caused by flaky tests.

Change-Id: I8f7831e470fd8aa065f0b8728f62f39734d05016
2023-03-07 16:39:20 +01:00
Matthias Sohn 8dcb02140d Prepare 6.5.1-SNAPSHOT builds
Change-Id: Idd9977ac08a339906e33beb73f57f8f6885ad86f
2023-03-07 16:39:19 +01:00
Matthias Sohn c72dd241f4 JGit v6.5.0.202303070854-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8da37ead0bd527bc4990ed5f8d5d4fb4f4d5cf01
2023-03-07 14:54:32 +01:00
Matthias Sohn 7241ecd814 Ignore generated org.eclipse.jgit.benchmarks/dependency-reduced-pom.xml
Change-Id: I8afe0e5128a63e0df02c5350772c942912408171
2023-03-07 14:53:57 +01:00
Stephan Wahlbrink 47ecb0096e [sshd] Fix calculation of timeout in AbstractClientProxyConnector
The previous implementation mixed nano seconds (elapsed) and milli
seconds (remaining) without conversion.

Change-Id: I9e1654afa47fa32c94808af3b2dd0418a372fb00
2023-03-06 23:45:18 +01:00
Matthias Sohn 9be2b7f8a8 Silence API error raised for removed BranchRebaseMode#PRESERVE
It was replaced by MERGES to match C git which did that in 2.34.

Change-Id: Ib6a33b4a3650345bf0f9d3726dd9e14c5797e836
2023-03-06 21:39:42 +01:00
Matthias Sohn 0687c73a12 Merge branch 'stable-6.5'
* stable-6.5:
  [errorprone] Suppress [Finally] warnings
  Update Orbit to R20230302014618 for 2023-03
  Improve test coverage when core.trustPackedRefsStat set to after_open
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302281825-rc1
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302221508-m3

Change-Id: Ice109c060d14c455262f61aed088111b238d735b
2023-03-03 16:04:00 +01:00
Matthias Sohn ec1dcfe500 Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: Id0a3b571aa6bbf159b225794ddc884f79431c670
2023-03-03 16:03:26 +01:00
Matthias Sohn cd72ecbc07 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I236c5cbb638493cc4b23475be50d0f724d62a688
2023-03-03 16:02:49 +01:00
Matthias Sohn 9dd912a1c2 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f
2023-03-03 16:02:16 +01:00
Matthias Sohn ebad3881b7 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3
2023-03-03 16:01:26 +01:00
Matthias Sohn 1d073e30d7 [errorprone] Suppress [Finally] warnings
In these cases we use Throwable#addSuppressed to ensure the exception
thrown in the catch block preceding the finally block throwing another
exception isn't lost.

Change-Id: I96e78a5c15238ab77ac90ca1901850ce19acfcd8
2023-03-02 23:33:27 +01:00
Matthias Sohn 1cff378a85 Update Orbit to R20230302014618 for 2023-03
and update
- com.google.gson to 2.10.1.v20230109-0753

Change-Id: I9745797a278c43ec62a82b5620bb72f075a23c1f
2023-03-02 23:33:14 +01:00
Prudhvi Akhil Alahari a4ca500d26 Improve test coverage when core.trustPackedRefsStat set to after_open
As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.

Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
2023-03-02 21:20:02 +05:30
Matthias Sohn f34ae6fe31 Prepare 6.6.0-SNAPSHOT builds
Change-Id: I17893f9db12bcb208866f40a06cd4f1ccbb4fe30
2023-03-01 15:40:45 +01:00
Matthias Sohn 69671a7026 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I313e3deed8fa00df0406b3d7b73e5b643dc25a05
2023-03-01 15:30:29 +01:00
Matthias Sohn f43560a760 JGit v6.5.0.202302281825-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I1eb2e87b70c2da1dc81468cdc7ecf7dbd21d4190
2023-03-01 00:23:58 +01:00
Matthias Sohn e060ac836d Merge branch 'master' into stable-6.5
* master:
  Change config pull.rebase=preserve to pull.rebase=merges
  BatchingProgressMonitor: expose time spent per task
  PackWriter: offer to write an object-size index for the pack
  Fix formatting in GC#doGc
  PackExt: Define new extension for the object size index

Change-Id: I174db870e6c2fcb72ff535b39bb6e1c3358f11b8
2023-03-01 00:11:41 +01:00
Matthias Sohn b4fc53e2d1 Prepare 6.5.0-SNAPSHOT builds
Change-Id: Ie95ce8fcc2a570505d23d5a1642e33c041212260
2023-03-01 00:10:18 +01:00
Pavel Salamon 0518a6b0c1 Change config pull.rebase=preserve to pull.rebase=merges
The native git option to preserve merge commits during rebase
has been changed from pull.rebase=preserve to pull.rebase=merges.

This changeset in jgit makes the same config change. The old "preserve"
option is no longer recognized and is replaced by new option called
"merges".

This makes jgit's rebase configuration compatible with native git
versions 2.34 and newer where the old "preserve" option has been
removed.

Change-Id: Ic07ff954e258115e76465a1593ef3259f4c418a3
2023-02-28 23:44:41 +01:00
Matthias Sohn 2d0b908048 BatchingProgressMonitor: expose time spent per task
Display elapsed time per task if enabled via
ProgressMonitor#showDuration or if system property or environment
variable GIT_TRACE_PERFORMANCE is set to "true". If both the system
property and the environment variable are set the system property takes
precedence.

E.g. using jgit CLI:

$ GIT_TRACE_PERFORMANCE=true jgit clone https://foo.bar/foobar
Cloning into 'foobar'...
remote: Counting objects: 1 [0.002s]
remote: Finding sources: 100% (15531/15531) [0.006s]
Receiving objects:      100% (169737/169737) [13.045s]
Resolving deltas:       100% (67579/67579) [1.842s]

Change-Id: I4d624e7858b286aeddbe7d4e557589986d73659e
2023-02-27 16:41:33 -05:00