Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.
Pull up that functionality to a superclass, so it can be reused.
Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.
Add a custom consumer to report loading events and eviction events when
enabled.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).
To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.
Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.
The check was already performed when a new FileBasedConfig
is created at L158:
// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());
The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.
This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.
Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* stable-6.0:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Id7540d03991cdcf6f405e946b8cbbcc6a9696a31
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Ibd69e9d941ad9262b61dd0c4368e48cb82597a12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.12:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I6576872cc0f5dd452252fa6e4526086cdee65c28
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ib80336a42e22da729b9db1e573772504cc0a3e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I9e79ea2a0c554a184e4ce3b13e375eac8b7a4ac5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2a84c838a886d1d6383c34f50b418baa743c57b0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.8:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I88a629e571fec5a9820114ebf5765b5d94a276bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.7:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ied786ab5e3c0dd05f701705fce2d4ad85502c4d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.6:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I454622dae6eb95aedbd858e3b12da72282d36673
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.5:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2622f1d384a88a556ba9d88f0d08a37af69e530c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.4:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ia1665dd92ccc3811a6116f41421a05aca10fc6eb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.3:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I52a57a17abe60e30e3d7615f8cb4d0c5e6aebd9b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.2:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Id37f47a5ef2e3c8329eca30c171941f7e5606a85
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.1:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I625667c2718ab31ae7df907c3dd6024a933913b8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This reverts commit f829f5f838.
Using MISSING_FILEKEY as indicator for a non-existing file doesn't work
on Windows.
Bug: 577954
Change-Id: I92102a3d259f6cc0f367096a3213cfa794466817
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".
Follow the example below:
. .
└── src └── src
└── d1 ==> └── d1
└─ file1 ├─ file1
└── file2
c.parents[i] c
The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".
We should add an additional check for no paths matching the filter.
Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
Speed up bitmap creation by loading reverse index in parallel
to reading bitmap from storage. Latency changes from
(time_to_read_bitmap + time_to_load_reverse_index) to
max(time_to_read_bitmap, time_to_load_reverse_index).
Add new option to DfsReaderOptions to control parallel reverse index
loading. Static cached thread pool is added to PackBitmapIndexV1 for
reverse index loading, and when not in use consumes minimal resources.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ia37a1d739631d053e8bddb925ac8b0b81d22379e
Return immediately in scanRef if the loose ref was identified as
missing when a snapshot was attempted for the ref. This will help
performance of scanRef when the ref is packed but has a corresponding
empty dir in 'refs/'.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~2.5s, where as with this change it takes ~1.5s.
Change-Id: I926bc41b9ae89a1a792b1b5ec9a17b05271c906b
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Doing a getFileStoreAttributes call even when the file doesn't
exist is unnecessary. This call is particularly slow on some
filesystems. Instead, do it only when the file exists and load
the appropriate cache.
This update can help speed up RefDirectory.exactRef when the ref
is packed, but has a corresponding empty dir for it under 'refs/'.
This scenario can happen when an atomic 'BatchRefUpdate' creates
new sharded refs.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~30s, where as with this change it takes ~2.5s.
Change-Id: I4a5d4c6a652dbeed1f4bc3b4f2b2f1416f7ca0e7
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Let GC#gc return collection of newly created packs as CompletableFuture
to enable using gc() asynchronously.
Change-Id: I3627014fd458c738cfe54225e631d6f7d9cfb1a7
* stable-6.0:
FS: debug logging only if system config file cannot be found
FS: debug logging only if system config file cannot be found
Update .factorypath used by annotation processor for benchmarks
Use maven-compiler-plugin's release tag instead of source and target
Don't use deprecated Repository#getAllRefs in Repository
Don't use deprecated Repository#getAllRefs in FileRepository
RevListTest: fix warning that method parameter hides field 'git'
Implement RecordingLogger based on org.slf4j.Logger
Let ObjectDatabase implement AutoClosable
Change-Id: Ie6b3cfa66b319033d4448dcf20362b753c0e9d7c
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
org.eclipse.jgit.internal.diffmergetool
org.eclipse.jgit.internal.fsck
This was broken with changes for bug 356832.
Bug: 356832
Change-Id: Id008d058769b4923d545a9373b45ceb3a3d27a08
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
see: http://git-scm.com/docs/git-difftool
* add command line support for "jgit difftool"
* show supported commands with "jgit difftool --help"
* added "git difftool --tool-help" to show the tools (empty now)
* prepare for all other commands
Bug: 356832
Change-Id: Ice0c13ef7953a20feaf25e7746d62b94ff4e89e5
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- Repository#getAllRefsByPeeledObjectId
Bug: 534731
Change-Id: Id6956ff112560e6314d4335238494708346f2338
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- AdvertiseRefsHook#advertiseRefs
- ReceivePack#setAdvertisedRefs
- Repository#getAdditionalHaves
Bug: 534731
Change-Id: I85bc6ce5815d40be5f80042c53f4663072d96be5
Provide a public interface PackLock exposing only the unlock() method.
Rename the internal PackLock class to PackLockImpl and have it implement
the new interface.
This way PackParser doesn't expose an internal class via its API
anymore, and client can still unlock pack locks that were created.
Bug: 576340
Change-Id: I976739f4ab28fe1f9ba7f35653a69a913aa68841
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Remove the unused parameter, which had a non-API type anyway.
Bug: 576340
Change-Id: Id71c01a643e1f31a8ff61ff69f7915c373db3263
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
While building the commit for the destination project, RepoCommand
catches GitApiExceptions and wraps them into ManifestErrorException
(a subclass of GitApiException). This hides the real exception from the
caller and prevent them to do a fine-grained catch.
Specifically this is problematic for gerrit's supermanifest plugin, that
won't see ConcurrentRefUpdate exceptions to detect lock-failures.
Use ManifestErrorException to wrap non-GitApiExceptions, let
GitApiExceptions pass through as they are.
Change-Id: Ia2cda244e65993bb07c89cd6435507d5d0754dd4
* stable-5.12:
Better git system config finding
Change-Id: Iebbae08b6bb6ef510ca07329df77223bc2128ec1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Better git system config finding
Change-Id: I47946b91658cb2e38709179b7a8513b1211431e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Better git system config finding
Change-Id: I460d855ea7878b279dbaffa6eb7ce5ca93f4c12c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Better git system config finding
Change-Id: I1022d003e0d3b9d452abfed9ac49663d0a93c6e6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 9446e62733)
Use the GitAPIException CanceledException instead of IOException
CancelledException in the rename detector.
Change-Id: I5121719eb714a123ec57769fc113c84857bd3c6c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Skip javadoc generation for test bundles.
Use character entities < and > for < and > outside of
code-formatted spans.
Change-Id: I66e1a1dc98881c61f93c9e5561c5513896b2ba01
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
CommitConfig.getCommitTemplateContent() has changed API.
Change-Id: If72ed79c43c167b99f356b69cc095d6bbf4d66e8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fixes an issue that commit template file could not be found if it has a
relative path instead of absolute path.
Relative path is probably common if git config --local is used.
Bug: 446355
Change-Id: I8ddf2be672647be825fd9c01af82809d31bb8356
With change https://git.eclipse.org/r/c/jgit/jgit/+/186455 a thread
loading a bitmap index could hold two ref locks at the same time (one
for bitmap and one for either index or reverse index). So it is possible
that two threads loading bitmaps end up in a deadlock for ref locks e.g.
threadA has refLock[1] (for bitmap) and wants refLock[2] (for index or
revIndex) and threadB has refLock[2] (for bitmap) and wants refLock[1].
This change introduces separate pools of locks per pack extension
instead of a shared pool. So threads loading bitmap can hold two
locks but with different extensions and no overlap, e.g. threadA holds
refLock[BITMAP_INDEX][1] and refLock[INDEX][2] and threadB holds
refLock[BITMAP_INDEX][2] and refLock[INDEX][1].
More unit tests were added to cover various paralell loading scenarios.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I89704b4721c21548929608d3798ef60925280755
We use BinaryBlobException to signal a binary blob was found and never
make use of its stack trace. Suppress filling in the stack trace to
avoid the performance penalty coming with that.
See https://shipilev.net/blog/2014/exceptional-performance/
Change-Id: Iae1f1c19a1fa8aef4f6569822557171130299958