Support the new option index.skipHash which was introduced in git 2.40
[1]. If it is set to true skip computing the git index checksum. This
accelerates Git commands that manipulate the index, such as git add, git
commit, or git status. Instead of storing the checksum, write a trailing
set of bytes with value zero, indicating that the computation was
skipped.
Accept a skipped checksum consisting of 20 null bytes when reading the
index since the option could have been set to true at the time when the
index was written.
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-indexskipHash
Bug: 581723
Change-Id: I28ebe44c5ca1cbcb882438665d686452a0c111b2
As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.
Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
* stable-6.2:
If tryLock fails to get the lock another gc has it
Fix GcConcurrentTest#testInterruptGc
Don't swallow IOException in GC.PidLock#lock
Check if FileLock is valid before using or releasing it
Change-Id: I5b6b10622b61fde3f0f10455a74ae159a0b69082
* stable-6.1:
If tryLock fails to get the lock another gc has it
Fix GcConcurrentTest#testInterruptGc
Don't swallow IOException in GC.PidLock#lock
Check if FileLock is valid before using or releasing it
Change-Id: I3ffe92566cc145053bb762f612dd96bc6d542c62
* stable-6.0:
If tryLock fails to get the lock another gc has it
Fix GcConcurrentTest#testInterruptGc
Don't swallow IOException in GC.PidLock#lock
Check if FileLock is valid before using or releasing it
Change-Id: Idea23e555c024557d7e39a86efe25f609400b962
* stable-5.13:
If tryLock fails to get the lock another gc has it
Fix GcConcurrentTest#testInterruptGc
Don't swallow IOException in GC.PidLock#lock
Check if FileLock is valid before using or releasing it
Change-Id: I708d0936fa86b028e4da4e7e21f332f8b48ad293
* stable-6.2:
Shortcut during git fetch for avoiding looping through all local refs
FetchCommand: fix fetchSubmodules to work on a Ref to a blob
Silence API warnings introduced by I466dcde6
Allow the exclusions of refs prefixes from bitmap
PackWriterBitmapPreparer: do not include annotated tags in bitmap
BatchingProgressMonitor: avoid int overflow when computing percentage
Speedup GC listing objects referenced from reflogs
FileSnapshotTest: Add more MISSING_FILE coverage
Change-Id: I2ff386d9a096277360e6c7bd5535b49984620fb3
* stable-6.1:
Shortcut during git fetch for avoiding looping through all local refs
FetchCommand: fix fetchSubmodules to work on a Ref to a blob
Silence API warnings introduced by I466dcde6
Allow the exclusions of refs prefixes from bitmap
PackWriterBitmapPreparer: do not include annotated tags in bitmap
BatchingProgressMonitor: avoid int overflow when computing percentage
Speedup GC listing objects referenced from reflogs
FileSnapshotTest: Add more MISSING_FILE coverage
Change-Id: Iff2fba026b49463016015b2fae1a42cf76ee2dbb
* stable-6.0:
Shortcut during git fetch for avoiding looping through all local refs
FetchCommand: fix fetchSubmodules to work on a Ref to a blob
Silence API warnings introduced by I466dcde6
Allow the exclusions of refs prefixes from bitmap
PackWriterBitmapPreparer: do not include annotated tags in bitmap
BatchingProgressMonitor: avoid int overflow when computing percentage
Speedup GC listing objects referenced from reflogs
FileSnapshotTest: Add more MISSING_FILE coverage
Change-Id: Ib5055f2f3b8a313c178d6f6c7c5630285ad5a726
* stable-5.13:
Shortcut during git fetch for avoiding looping through all local refs
FetchCommand: fix fetchSubmodules to work on a Ref to a blob
Silence API warnings introduced by I466dcde6
Allow the exclusions of refs prefixes from bitmap
PackWriterBitmapPreparer: do not include annotated tags in bitmap
BatchingProgressMonitor: avoid int overflow when computing percentage
Speedup GC listing objects referenced from reflogs
FileSnapshotTest: Add more MISSING_FILE coverage
Change-Id: I58ad4c210a5e7e5a1ba6b22315b04211c8909950
When running a GC.repack() against a repository with over one
thousands of refs/heads and tens of millions of ObjectIds,
the calculation of all bitmaps associated with all the refs
would result in an unreasonable big file that would take up to
several hours to compute.
Test scenario: repo with 2500 heads / 10M obj Intel Xeon E5-2680 2.5GHz
Before this change: 20 mins
After this change and 2300 heads excluded: 10 mins (90s for bitmap)
Having such a large bitmap file is also slow in the runtime
processing and have negligible or even negative benefits, because
the time lost in reading and decompressing the bitmap in memory
would not be compensated by the time saved by using it.
It is key to preserve the bitmaps for those refs that are mostly
used in clone/fetch and give the ability to exlude some refs
prefixes that are known to be less frequently accessed, even
though they may actually be actively written.
Example: Gerrit sandbox branches may even be actively
used and selected automatically because its commits are very
recent, however, they may bloat the bitmap, making it ineffective.
A mono-repo with tens of thousands of developers may have
a relatively small number of active branches where the
CI/CD jobs are continuously fetching/cloning the code. However,
because Gerrit allows the use of sandbox branches, the
total number of refs/heads may be even tens to hundred
thousands.
Change-Id: I466dcde69fa008e7f7785735c977f6e150e3b644
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
The annotated tags should be excluded from the bitmap associated
with the heads-only packfile. However, this was not happening
because of the check of exclusion of the peeled object instead
of the objectId to be excluded from the bitmap.
Sample use-case:
refs/heads/main
^
|
commit1 <-- commit2 <- annotated-tag1 <- tag1
^
|
commit0
When creating a bitmap for the above commit graph, before this
change all the commits are included (3 bitmaps), which is
incorrect, because all commits reachable from annotated tags
should not be included.
The heads-only bitmap should include only commit0 and commit1
but because PackWriterBitPreparer was checking for the peeled
pointer of tag1 to be excluded (commit2) which was not found in
the list of tags to exclude (annotated-tag1), the commit2 was
included, even if it wasn't reachable only from the head.
Add an additional check for exclusion of the original objectId
for allowing the exclusion of annotated tags and their pointed
commits. Add one specific test associated with an annotated tag
for making sure that this use-case is covered also.
Example repository benchmark for measuring the improvement:
# refs: 400k (2k heads, 88k tags, 310k changes)
# objects: 11M (88k of them are annotate tags)
# packfiles: 2.7G
Before this change:
GC time: 5h
clone --bare time: 7 mins
After this change:
GC time: 20 mins
clone --bare time: 3 mins
Bug: 581267
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Change-Id: Iff2bfc6587153001837220189a120ead9ac649dc
When cloning huge repositories I observed percentage of object counts
turning negative. This happened if lastWork * 100 exceeded
Integer.MAX_VALUE.
Change-Id: Ic5f5cf5a911a91338267aace4daba4b873ab3900
Add a couple tests that confirm what the docs say about isModified() and
equals(MISSING_FILE) behavior.
Change-Id: I6093040ba3594934c3270331405a44b2634b97c5
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
* stable-6.2:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa
* stable-6.1:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286
* stable-6.0:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
* stable-5.13:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.
When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.
Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e
Previous code would do a content merge on symlinks, and write the merge
result to the working tree as a file. C git doesn't do this; it leaves
a symlink in the working tree unchanged, or in a delete-modify conflict
it would check out "theirs".
Moreover, previous code would write the merge result to the link target,
not to the link. This would overwrite an existing link target, or fail
if the link pointed to a directory.
In link/file conflicts or file/link conflicts, C git always puts the
file into the working tree.
Change conflict handling accordingly. Add tests for all the conflict
cases.
Bug: 580347
Change-Id: I3cffcb4bcf8e336a85186031fff23f0c4b6ee19d
Signed-off-by: Thomas Wolf <twolf@apache.org>
DAG."
This reverts commit 6297491e8a.
This is done as a quick fix for the failure of egit tests caused by the
introduction of FilteredRevCommit.
Bug: 580690
Change-Id: Ia6b651dd11b0a4b02d5e52247eb4bf13adf94e27
BlameGenerator."
This reverts commit 5747bba48b.
This is done as a quick fix for the failure of egit tests caused by the
introduction of FilteredRevCommit.
Bug: 580690
Change-Id: Ia0178bc2de4fc825a81207bbd7979bf3a386c955
An invalid path in the manifest (e.g. '.') is reported by DirCache in a
runtime exception. In server context this becomes a 500 instead of a user error.
Wrap the runtime invalid path exception into a checked ManifestErrorException that
caller can handle.
Change-Id: I61a2104922765506ae232334891057bb06141d97
On executing a copy, mark the destination as updated.
On executing a rename, mark both source and destination as updated.
Change-Id: Ied5b9b0e5a14eac59a06cdd0961e25e143f50ff0
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Change-Id: Ie127cb710d004079e9f53a5802130afdb49a7de1
Previous code loaded the WorkingTreeOptions afresh for every single
file being checked out. This checked the git config (all three files,
repo, user and system config) for having been modified every time.
These checks can be costly, for instance on Windows, or if one of the
three config files is not on a local disk, or on an otherwise slow
storage.
Improve this by loading the options and thus checking the git config
only once before the checkout.
Bug: 579715
Change-Id: I21cd5a808f9d90b5ca2d022f91f0eeb8ca26091c
Signed-off-by: Thomas Wolf <twolf@apache.org>
With core.symlinks=false, symlinks are checked out as plain files.
When such a file is re-added to the index, and the index already
contains a symlink there, add the file as a symlink. Previous code
changed the index entry to a regular file.
Bug: 580412
Change-Id: I5497bedc3da89c8b10120b8077c56bc5b67cb791
Signed-off-by: Thomas Wolf <twolf@apache.org>
String.startsWith() is not a valid test for file path prefixes:
directory "a" is _not_ a prefix of a file "ab", only of "a/b".
Add a proper Paths.isEqualOrPrefix() method and use it in CleanCommand.
Bug: 580478
Change-Id: I6863e6ba94a8ffba6561835cc57044a0945d2770
Signed-off-by: Thomas Wolf <twolf@apache.org>
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Change-Id: I5dab25301d6aef7df6a0bc25a4c553c730199272
This makes RevCommit extensible to allow having different structure of
child-parent relationship. This change is a pre-requsite for having a
FilteredRevCommit that overrides parents from the RevCommit. That then
provides a cheaper way to walk over a subset of RevCommits instead of
an expensive way that applies filters while walking over selected
commits. Useful with Blame which works on a single file and that can be
made performant, if we know all the commits needed by the Blame
algorithm. So Blame algorithm can avoid walking over finding what
commits to blame on.
This change makes parents field on RevCommit private and exposes it
thrrough overrideable methods such as getParents, getParent at index,
getParentCount and setParents. All other files other than RevCommit are
updating the usages of accessing RevCommits parents.
Change-Id: I2d13b001c599cc4ebc92d1ab6e07b07acb3b7fe5
The FilteredRenameDetectorTest was added in [1], but bazel build rules
were not updated.
[1] https://git.eclipse.org/r/c/jgit/jgit/+/194200
Change-Id: I7fc713e19b4768176eb84e9768137431b33a805e
This adds support for shallow cloning. The CloneCommand and the
FetchCommand now have the new methods setDepth, setShallowSince and
addShallowExclude to tell the server that the client doesn't want to
download the complete history.
Bug: 475615
Change-Id: Ic80fb6efb5474543ae59be590ebe385bec21cc0d
Matthias Sohn