Each index can be set in the reader from two locations: the dfs cache
callback or the code afterwards. The pack is emitting the load event
in both cases, when the reference is set. This is brittle (right now
it is missing events for BITMAP_INDEX and COMMIT_GRAPH).
Emit the index loaded event only once, after going through the cache
code. The fact that the reference was set in the callback or the main
code is irrelevant. Also, the reader is per-thread, so there shouldn't
be any concurrency involved triggering double counts.
Change-Id: I7f3d078a53741ecc1e81b96353ed8faa8fef3a49
The DfsReader#emitIndexLoad expects a reference to the loaded object,
not to its identity hash. This makes listeneres recalculate an
identity hash over an Int object.
Pass the expected reference to the method.
Change-Id: Iee982fdd0169c93c5c7cd9cfe4260a7a82d23f6a
* stable-6.7:
Prepare 6.7.1-SNAPSHOT builds
Switch back to wagon-ssh-external
Fix generation of maven site
JGit v6.7.0.202309050840-r
Remove unused API problem filters
[releng] Bump japicmp base version to 6.6.0.202305301015-r
Fix list of 3rd party bundles in p2 repo
Add missing source bundle org.osgi.service.cm.source to target platform
Prepare 6.6.2-SNAPSHOT builds
JGit v6.6.1.202309021850-r
Checkout: better directory handling
Document commit-graph options supported by JGit
Fix warning raised for local variable hiding DfsPackFile#index
Suppress boxing warnings in DfsPackFile
Remove unused API problem filters
Suppress boxing warnings in tests
Fix warning about empty block
Update mockito to 5.5.0
Update byte-buddy to 1.14.7
Prepare 6.7.0-SNAPSHOT builds
JGit v6.7.0.202308301100-rc1
Change-Id: Ib2a1000ead5ca6cf3816e9e9496ad5d92f4dc963
- update maven-site-plugin to 4.0.0-M9
- use wagon-ssh extension for deployment
- disable doclint
- don't fail the build for javadoc warnings
Change-Id: I81f7229deaf6892416ce4cebd6c122f00a857d51
The chunk list is calculated before start writing and it is
immutable afterwards.
Make this explicit using an unmodifiable list for the chunks.
Change-Id: I4a5cfb1f3d06a9393d8d93a3fac3c128cf3faec0
The final size of the commit-graph is known before-hand. As a
safety-net, assert the written size matches the expected value.
Change-Id: Ib0828a7cce5bacb33f6325ee3910f4eebd95eb8c
The jgit p2 repo should contain all 3rd party dependencies needed at
runtime but not dependencies only used in tests.
- remove assertj-core since it's only used in tests
- add org.eclipse.osgi and org.osgi.service.cm which are runtime
dependencies
Change-Id: Ie789cb8feab0905e7e23aae1d5378e82a0088992
When checking out a file into the working tree ensure that all parent
directories of the file below the working tree root are actually
directories and do exist before we try to create the file.
When multiple files are to be checked out (or even a whole tree), this
may check the same directories over and over again. Asking the file
system every time for file attributes is a potentially expensive
operation. As a remedy, introduce an in-memory cache of directory
states for a particular check-out operation.
Apply the same fix also in the ResolveMerger, which may also check out
files, and also in the PatchApplier. In PatchApplier, also validate
paths.
Change-Id: Ie12864c54c9f901a2ccee7caddec73027f353111
Signed-off-by: Thomas Wolf <twolf@apache.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
CommitGraphWriter first defines the chunks and then writes them. If at
write time a chunk is unknown, it is ignored. This is brittle: if
somebody adds a chunk to the header but not to the actual writing, the
commit-graph is broken and there is no error reported anywhere.
Throw exception if at write time a chunk is unknown. This can only
happen by a coding error in the writer.
Change-Id: Iade677bb6ce368b6941b75a21c622917afa3b751
* master:
Remove the cbi-snapshots Maven repository
Update Orbit to orbit-aggregation/release/4.29.0
Add target platform for Eclipse 2023-09 (4.29)
Use release p2 repo for Eclipse 2023-06 (4.28)
Update tycho to 4.0.2
Update jmh to 1.37
Update bouncycastle to 1.76
Fix some tests in ConfigTest
Handle global git config $XDG_CONFIG_HOME/git/config
IO: use JDK convenience methods
org.eclipse.jgit.junit.ssh/.settings/.api_filters: fix unclosed tags
ReadChangedPathFilter: fix Non-externalized string literal warning
Introduce core.packedIndexGitUseStrongRefs config key
DfsReader: Make PackLoadListener interface visible to subclasses
DfsGarbageCollector: provide commit graph stats
DfsGarbageCollector: put only GC commits into the commit graph
DfsReader: Expose when indices are loaded
Change-Id: Idd78a0a1bc3cd3db5edb475e235c13354d9087a9
Switch to bundle dependencies for hamcrest 1.3 to avoid issues with
split packages in that version.
Don't allow hamcrest 2.x yet since junit 4.13.2 still requires hamcrest
1.3.
See Orbit restructuring in
https://github.com/orgs/eclipse-orbit/discussions/49
Change-Id: I8faf519b8f2c4e4a6bd255d694d1aa28017acd85
Some of the ConfigTest tests created a FileRepositoryBuilder but didn't
use it to actually create a FileRepository.
Change-Id: I1a4b27891daee1b235a71e7bbf2a7588b9d11a53
C git uses this alternate fallback location if the file exists and
~/.gitconfig does not. Implement this also for JGit.
If both files exist, reading behavior is as if the XDG config was
inserted between the HOME config and the system config. Writing
behaviour is different: all changes will be applied only in the HOME
config. Updates will occur in the XDG config only if the HOME config
does not exist.
This is consistent with the behavior of C git; compare [1], especially
the sections on FILES and SCOPES, and the description of the --global
option.
[1] https://git-scm.com/docs/git-config
Bug: 581875
Change-Id: I2460b9aa963fd2811ed8a5b77b05107d916f2b44
Signed-off-by: Thomas Wolf <twolf@apache.org>
The benefit is that certain InputStreams can override the default
implementation for performance reasons.
Change-Id: I4c924157ec0f0ec63b0eca7cdbdc9325af24cab6
Introduce a core.packedIndexGitUseStrongRefs configuration key, which
defaults to true so that the current behavior does not change. However,
setting it to false allows soft references to be used for Pack indices
instead of strong references so that they can be garbage collected when
there is memory pressure.
Pack objects can be large when associated with pack files with large
object counts, and this memory is not really accounted for or tracked by
the WindowCache and it can be very substantial at times, especially with
many large object count projects. A particularly problematic use case is
Gerrit's ls-projects command which loads very little data in the
WindowCache via ByteWindows, but ends up loading and holding many entire
indices in memory, sometimes even after the ByteWindows for their Pack
objects have already been garbage collected since they won't get cleared
until after a new ByteWindow is loaded. By using SoftReferences, single
use indices can get cleared when there is memory pressure and OOMs can
be easily avoided, drastically reducing the amount of memory required to
perform an ls-projects on large sites with many projects and large
object counts.
On one of our test sites, an ls-projects command with strong index
references requires more than 66GB of heap to complete successfully,
with soft index references it requires less than 23GB.
Change-Id: I3cb3df52f4ce1b8c554d378807218f199077d80b
Signed-off-by: Martin Fick <quic_mfick@quicinc.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A subclass cannot implement a listener with the default access.
Make the interface protected. Not public because so far only
subclasses are interested in this interface. We can widen the
visibility later if needed.
Change-Id: I54e5c0ef1312dfe2fa660bc8fb54e2be35c0f6df
Provide commit graph stats in the same way that we provide reftable
stats.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Ib80c892a26f9b552bc90f3cbe7da83b02ffebdfd
GC puts all commits reachable from heads and tags into the GC pack,
and commits reachable only from other refs (e.g. refs/changes) into
GC_REST. The commit-graph contains all commits in GC and GC_REST. This
produces too big commit graphs in some repos, beating the purpose of
loading the index.
Limit the commit graph to commits reachable from heads and tags
(i.e. commits in the GC pack).
Change-Id: I4962faea5a726d2ea3e548af0aeae370a6cc8588