Commit Graph

9472 Commits

Author SHA1 Message Date
Stephan Wahlbrink 47ecb0096e [sshd] Fix calculation of timeout in AbstractClientProxyConnector
The previous implementation mixed nano seconds (elapsed) and milli
seconds (remaining) without conversion.

Change-Id: I9e1654afa47fa32c94808af3b2dd0418a372fb00
2023-03-06 23:45:18 +01:00
Matthias Sohn 9be2b7f8a8 Silence API error raised for removed BranchRebaseMode#PRESERVE
It was replaced by MERGES to match C git which did that in 2.34.

Change-Id: Ib6a33b4a3650345bf0f9d3726dd9e14c5797e836
2023-03-06 21:39:42 +01:00
Matthias Sohn 0687c73a12 Merge branch 'stable-6.5'
* stable-6.5:
  [errorprone] Suppress [Finally] warnings
  Update Orbit to R20230302014618 for 2023-03
  Improve test coverage when core.trustPackedRefsStat set to after_open
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302281825-rc1
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302221508-m3

Change-Id: Ice109c060d14c455262f61aed088111b238d735b
2023-03-03 16:04:00 +01:00
Matthias Sohn ec1dcfe500 Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: Id0a3b571aa6bbf159b225794ddc884f79431c670
2023-03-03 16:03:26 +01:00
Matthias Sohn cd72ecbc07 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I236c5cbb638493cc4b23475be50d0f724d62a688
2023-03-03 16:02:49 +01:00
Matthias Sohn 9dd912a1c2 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f
2023-03-03 16:02:16 +01:00
Matthias Sohn ebad3881b7 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3
2023-03-03 16:01:26 +01:00
Matthias Sohn 1d073e30d7 [errorprone] Suppress [Finally] warnings
In these cases we use Throwable#addSuppressed to ensure the exception
thrown in the catch block preceding the finally block throwing another
exception isn't lost.

Change-Id: I96e78a5c15238ab77ac90ca1901850ce19acfcd8
2023-03-02 23:33:27 +01:00
Matthias Sohn 1cff378a85 Update Orbit to R20230302014618 for 2023-03
and update
- com.google.gson to 2.10.1.v20230109-0753

Change-Id: I9745797a278c43ec62a82b5620bb72f075a23c1f
2023-03-02 23:33:14 +01:00
Prudhvi Akhil Alahari a4ca500d26 Improve test coverage when core.trustPackedRefsStat set to after_open
As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.

Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
2023-03-02 21:20:02 +05:30
Matthias Sohn f34ae6fe31 Prepare 6.6.0-SNAPSHOT builds
Change-Id: I17893f9db12bcb208866f40a06cd4f1ccbb4fe30
2023-03-01 15:40:45 +01:00
Matthias Sohn 69671a7026 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I313e3deed8fa00df0406b3d7b73e5b643dc25a05
2023-03-01 15:30:29 +01:00
Matthias Sohn f43560a760 JGit v6.5.0.202302281825-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I1eb2e87b70c2da1dc81468cdc7ecf7dbd21d4190
2023-03-01 00:23:58 +01:00
Matthias Sohn e060ac836d Merge branch 'master' into stable-6.5
* master:
  Change config pull.rebase=preserve to pull.rebase=merges
  BatchingProgressMonitor: expose time spent per task
  PackWriter: offer to write an object-size index for the pack
  Fix formatting in GC#doGc
  PackExt: Define new extension for the object size index

Change-Id: I174db870e6c2fcb72ff535b39bb6e1c3358f11b8
2023-03-01 00:11:41 +01:00
Matthias Sohn b4fc53e2d1 Prepare 6.5.0-SNAPSHOT builds
Change-Id: Ie95ce8fcc2a570505d23d5a1642e33c041212260
2023-03-01 00:10:18 +01:00
Pavel Salamon 0518a6b0c1 Change config pull.rebase=preserve to pull.rebase=merges
The native git option to preserve merge commits during rebase
has been changed from pull.rebase=preserve to pull.rebase=merges.

This changeset in jgit makes the same config change. The old "preserve"
option is no longer recognized and is replaced by new option called
"merges".

This makes jgit's rebase configuration compatible with native git
versions 2.34 and newer where the old "preserve" option has been
removed.

Change-Id: Ic07ff954e258115e76465a1593ef3259f4c418a3
2023-02-28 23:44:41 +01:00
Matthias Sohn 2d0b908048 BatchingProgressMonitor: expose time spent per task
Display elapsed time per task if enabled via
ProgressMonitor#showDuration or if system property or environment
variable GIT_TRACE_PERFORMANCE is set to "true". If both the system
property and the environment variable are set the system property takes
precedence.

E.g. using jgit CLI:

$ GIT_TRACE_PERFORMANCE=true jgit clone https://foo.bar/foobar
Cloning into 'foobar'...
remote: Counting objects: 1 [0.002s]
remote: Finding sources: 100% (15531/15531) [0.006s]
Receiving objects:      100% (169737/169737) [13.045s]
Resolving deltas:       100% (67579/67579) [1.842s]

Change-Id: I4d624e7858b286aeddbe7d4e557589986d73659e
2023-02-27 16:41:33 -05:00
Ivan Frade ca2c57b2ec PackWriter: offer to write an object-size index for the pack
PackWriter callers tell the writer what do the want to include in the
pack and invoke #writePack(). Afterwards, they can invoke #writeIndex()
to write the corresponding pack index.

Mirror this for the object-size index, adding a #writeObjectSizeIndex()
method.

Change-Id: Ic319975c72c239cd6488303f7d4cced797e6fe00
2023-02-24 12:56:33 -08:00
Matthias Sohn cfacc43b52 Fix formatting in GC#doGc
Change-Id: Ifa3adb66d4e0404bab4036d6b165d6c4dafe921a
2023-02-24 15:18:39 +01:00
Ivan Frade ad07196d60 PackExt: Define new extension for the object size index
Change-Id: I6bbaf43b4e6fb456ca0e9e0c6efcfeded0f94d6d
2023-02-23 09:32:20 -08:00
Matthias Sohn 2ca587b36e JGit v6.5.0.202302221508-m3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I00bf91823e2fe6503d52ae23bc1854f0b1d66dbd
2023-02-22 21:09:10 +01:00
Matthias Sohn 8c2afe6418 Merge branch 'master' into stable-6.5
* master:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I0a599ec9241dd3b9b9a61122b2eda6bd647bcc22
2023-02-22 21:07:34 +01:00
Matthias Sohn 176f17d05e Merge branch 'stable-6.4'
* stable-6.4:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: Ia2797b44a60342eb9df53f0b3d674cba92a512fc
2023-02-22 21:06:41 +01:00
Matthias Sohn f4eda3360a Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I5af34c92e423a651db53b4dc45ed844d5f39910d
2023-02-22 21:05:55 +01:00
Matthias Sohn 636f377e4e Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I5b6b10622b61fde3f0f10455a74ae159a0b69082
2023-02-22 21:03:52 +01:00
Matthias Sohn 6cc741aa23 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I3ffe92566cc145053bb762f612dd96bc6d542c62
2023-02-22 21:03:22 +01:00
Matthias Sohn b526829fba Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: Idea23e555c024557d7e39a86efe25f609400b962
2023-02-22 21:02:47 +01:00
Matthias Sohn 238f1693f7 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  If tryLock fails to get the lock another gc has it
  Fix GcConcurrentTest#testInterruptGc
  Don't swallow IOException in GC.PidLock#lock
  Check if FileLock is valid before using or releasing it

Change-Id: I708d0936fa86b028e4da4e7e21f332f8b48ad293
2023-02-22 21:02:09 +01:00
Matthias Sohn d9f75e8bb2 If tryLock fails to get the lock another gc has it
Change-Id: Ifd3bbcc5e0591883b774d23256949a83010ea134
2023-02-22 20:38:43 +01:00
Matthias Sohn 1691e38779 Fix GcConcurrentTest#testInterruptGc
With the new GC.PidLock interrupting a running GC throws a
ClosedByInterruptException.

Change-Id: I7ccea1ae9a43d4edfdab2fcfd1324c64cc22b38f
2023-02-22 20:38:29 +01:00
Matthias Sohn 49f5273867 Don't swallow IOException in GC.PidLock#lock
This broke the test GcConcurrentTest#testInterruptGc which expects
ClosedByInterruptException when the thread doing gc is interrupted.

Change-Id: I89e02fc37aceeccb04c20cfc5b71cb8fa21793d6
2023-02-22 19:27:30 +01:00
Matthias Sohn a6da439b47 Check if FileLock is valid before using or releasing it
Change-Id: I23ba67b61b9b03772f33a929c080c0d02b8c8652
2023-02-22 02:56:06 +01:00
Matthias Sohn 95240cefac Merge branch 'master' into stable-6.5
* master:
  Use Java 11 ProcessHandle to get pid of the current process
  UploadPack: use allow-any-sha1-in-want configuration
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I71c8ee39e0f3438cbc1992c24f9f6c33ece88c07
2023-02-22 02:18:10 +01:00
Matthias Sohn e92212a5a0 Merge branch 'stable-6.4'
* stable-6.4:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ifa4e56b6ecca9305f3f1685e45450019bfc82e22
2023-02-22 01:29:32 +01:00
Matthias Sohn dcd6367391 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ic40dbab18616d8d9fe3820b9890c86652b80eb47
2023-02-22 01:28:27 +01:00
Matthias Sohn c70374e641 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I53cf9675deac0b588048d8224216d2a7e8bd16ec
2023-02-22 01:27:50 +01:00
Matthias Sohn 628ca9bd6f Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: I0562a4a224779ccf1e4cc1ff8f5a352e55ab220a
2023-02-22 01:27:16 +01:00
Matthias Sohn 4c111e59d0 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Use Java 11 ProcessHandle to get pid of the current process
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ib9a2419253ffcbc90874adbfdb8129fee3178210
2023-02-22 01:26:36 +01:00
Matthias Sohn 2a2a208fa1 Use Java 11 ProcessHandle to get pid of the current process
Change-Id: I790f218601c1d5e1b39c4101e3b2708e76b9d782
2023-02-22 01:06:06 +01:00
Matthias Sohn aa13d1daf5 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Acquire file lock "gc.pid" before running gc
  Silence API errors introduced by 9424052f

Change-Id: Ibb5c46cb79377d2d2cd7d4586f31c86665d2851c
2023-02-22 01:00:26 +01:00
kylezhao d789fe2f4d UploadPack: use allow-any-sha1-in-want configuration
C git 2.11 supports setting the equivalent of RequestPolicy.ANY with
uploadpack.allowAnySHA1InWant[1]. Parse this into TransportConfig and
use it from UploadPack.

Add additional tests for [2] and this change.

We can execute "git clone --filter=blob:none --no-checkout" successfully
with config uploadPack.allowFilter is true. But when we checkout, the
git will fetch other missing objects required by the checkout(this is
why we need this config).

When both uploadPack.allowFilter and uploadPack.allowAnySHA1InWant are
true, jgit will support partial clone. If you are using an extremely
large monorepo, this feature can help. It allows users to work on an
incomplete repo which reduces disk usage.

[1] f8edeaa05d
[2] change Id39771a6e42d8082099acde11249306828a053c0

Bug: 573390
Change-Id: I8fe75f03bf1fea7c11e0d67c8637bd05dd1f9b89
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-02-21 09:11:21 +01:00
Matthias Sohn 8eee800fb1 Acquire file lock "gc.pid" before running gc
Git guards gc by locking a lock file "gc.pid" before starting execution.
The lock file contains the pid and hostname of the process holding the
lock. Git tries to kill the process holding that lock if the lock file
wasn't modified in the last 12 hours and was started from the same host.

Teach JGit to acquire this lock before running gc but skip execution if
another process already holds the lock. Killing the other process could
be undesired if it's a long running application.

If the lock file wasn't modified in the last 12 hours try to lock it and
run gc if locking succeeds.

Register a shutdown hook for the lock file to ensure it is cleaned up if
the process is gracefully killed.

Change-Id: I00b838dcbf4fb0d03863bf7a2cd86b743c6c6971
2023-02-21 00:18:33 +01:00
Matthias Sohn 380f091fa5 Silence API errors introduced by 9424052f
Change-Id: Ia9e619a8fa06648086b583c994e4b107ae06c44d
2023-02-21 00:18:33 +01:00
Matthias Sohn c8683db55d Merge branch 'master' into stable-6.5
* master:
  Externalize strings introduced in c9552aba
  Silence API error introduced by 596c445a
  PackConfig: add entry for minimum size to index
  Fix getPackedRefs to not throw NoSuchFileException
  PackObjectSizeIndex: interface and impl for the object-size index
  UInt24Array: Array of unsigned ints encoded in 3 bytes.
  PackIndex: expose the position of an object-id in the index
  Add pack options to preserve and prune old pack files
  DfsPackFile/DfsGC: Write commit graphs and expose in pack
  ObjectReader: Allow getCommitGraph to throw IOException
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec
  UploadPack: consume delimiter in object-info command
  PatchApplier fix - init cache with provided tree
  Avoid error-prone warning
  Fix unused exception error-prone warning
  UploadPack: advertise object-info command if enabled
  Move MemRefDatabase creation in a separate method.
  DfsReaderIoStats: Add Commit Graph fields into DfsReaderIoStats

Change-Id: Ic9f91f2139432999b99c444302457b3c08911009
2023-02-20 22:18:22 +01:00
Matthias Sohn 5d5a0d5376 Externalize strings introduced in c9552aba
Change-Id: I81bb78344df61e6eb42622fcef6235d4da0ae052
2023-02-20 21:40:40 +01:00
Matthias Sohn 37dd45e8a9 Silence API error introduced by 596c445a
Change-Id: I961ba2d89c11373ccb81e6450d7d951204ffca36
2023-02-20 21:31:09 +01:00
Matthias Sohn fe64445c11 Merge branch 'stable-6.4'
* stable-6.4:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I36051c623fcd480aa80ed32b4e89f9bdd1b798e0
2023-02-20 21:29:30 +01:00
Matthias Sohn f8e6bcba48 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I1073098fb06eabafdb3c5e7fcf44d55b86a1b152
2023-02-20 21:01:38 +01:00
Matthias Sohn 6ea0e11869 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Fix getPackedRefs to not throw NoSuchFileException
  Add pack options to preserve and prune old pack files
  Allow to perform PackedBatchRefUpdate without locking loose refs
  Document option "core.sha1Implementation" introduced in 59029aec

Change-Id: I765c7302ce84a6a9c28fdef29da2bfaa49477c6e
2023-02-20 20:59:14 +01:00
Ivan Frade 596c445af2 PackConfig: add entry for minimum size to index
The object size index can have up to #(blobs-in-repo) entries, taking
a relevant amount of memory. Let operators configure the threshold size
to include objects in the size index.

The index will include objects with size *at or above* this
value (with -1 for none). This is more effective for the
filter-by-size case.

Lowering the threshold adds more objects to the index. This improves
performance at the cost of memory/storage space. For the object-size
case, more calls will use the index instead of reading IO. For the
filter-by-size case, lower threshold means better granularity (if
ObjectReader#isSmallerThan is implemented based only on the index).

Change-Id: I6ccd9334adbbc2abf95fde51dbbfc85b8230ade0
2023-02-16 10:25:44 -08:00