This change is a fix to http://git.eclipse.org/r/c/jgit/jgit/+/183562
that was reverted in http://git.eclipse.org/r/c/jgit/jgit/+/184978
due to deadlocks. Separate locks in DfsBlockFile are removed to rely
on getting value from DfsBlockCache with region locking in place.
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index
and index could be read from storage in parallel in separate threads.
A unit test is added for parallel index loading.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ic6d9c5a4a254628636aa98a5008447a27a003f69
The file name in META-INF/services must be the fully qualified
interface name; the content the fully qualified implementation class
name.
This was broken in commit 9683bc71.
Add a test for the default factory being found by the ServiceLoader.
Change-Id: I1f180d7f60e5c1e74a39bbd9a5f0099bd8343e21
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Add org.bouncycastle.bcutil to p2 repository
Change-Id: I789e3c84045cc9dcd8a26de34d09fc1e3ccc41e7
* stable-5.12:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
* stable-5.11:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
* stable-5.10:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
* stable-5.9:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
We observed the following issue:
$ git tag -a v1.0 -m v1.0
$ git push origin tag v1.0
$ git ls-remote origin v1.0^{}
... empty result ...
On the server (Gerrit) side run git-gc to pack the refs:
$ git gc
Repeat the ls-remote from the client and the result is correct:
$ git ls-remote origin v1.0^{}
7ad85c810de3ae922903d4bdd17c53cd627260ba refs/tags/v1.0^{}
Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.
It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.
Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Let certain pack extensions that are expensive to load from storage
(e.g. pack index, bitmap index) stay in DFS block cache longer than
others by overriding default cache count through DfsBlockCacheConfig
Don't change default behavior when cache override map is empty. Use int
cacheCount instead of boolean hot for Ref<T>
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I18062784ec9cc14dbba3e4bb8d9509440cf2d44f
If an annotated tag refers to a commit, we should not ignore it.
Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.
Example:
As is shown in the graph below:
1 - 2 - 3 - 4 (side)
\
5 - 6^ (master) - 7 (topic)
Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].
Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:
commit | Before | After | Rel %
47a44d27ca | 29251ms | 6687ms | -77%
90327e7dff | 21388ms | 6256ms | -70%
f85fac0efa | 11150ms | 7338ms | -34%
The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.
Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
* stable-5.8:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
* stable-5.7:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
* stable-5.6:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
To avoid a warning about a name clash with the field.
Change-Id: Ib1950486a9c63647e4240a268a3c9a7d613f3303
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In Java 11 type arguments for anonymous subclasses can be inferred
and don't need to be specified. This resolves a number of compiler
warnings.
Change-Id: I55eff3babb7628aa0627085e65a1b45eb12c2cd3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We updated the baseline target platform to jgit-4.17 corresponding to
Eclipse 4.17 (2020-09). Delete all target platforms older than this
version and the corresponding Orbit releases.
Change-Id: I93b2d8585bdc3c3312264d34657ceaf3f4a27bd5
jgit-4.17 corresponds to Eclipse 2020-09, which is the first Eclipse
version that required Java 11, and which is JGit's new baseline as of
JGit 6.0.
Remove pack200 which is deprecated in Java 11 and isn't supported by
tycho 2.5.0 anymore. It was specified in JSR 200 (J2SE 1.5) [1],
deprecated in JEP 336 (Java SE 11) [2] and removed in JEP 367 (Java SE
14) [3].
[1] https://www.jcp.org/en/jsr/detail?id=200
[2] https://openjdk.java.net/jeps/336
[3] https://openjdk.java.net/jeps/367
Change-Id: I12c196586397779d2be81d2374207703e3fc5dbf
On Java 16 this test throws InaccessibleObjectException, handle and
ignore it similar to IllegalAccessException.
Change-Id: I19b4f577579694a146516861a7ec567141f3464b
The package org.eclipse.jgit.transport was split between
org.eclipse.jgit and org.eclipse.jgit.ssh.jsch.
Bug: 564544
Change-Id: I91d38e67c65ed97a880f8dc8f9559663b9eec33b
CryptlibObjectIdentifiers can't be used because bouncycastle doesn't
export the package org.bouncycastle.asn1.cryptlib.
Bug: 573638
Change-Id: I1f9e2af02d9fec69d2249a7d78301ba4b333a9ba
This ensures the compiler compiles against the public, supported and
documented API for a specific VM version (here 11) [1]. This also means
that
we don't need EE descriptors in Eclipse anymore in order to ensure that
only supported APIs of the selected Java version can be used.
According to [2] if option --release is used --source and --target
options can't be used.
While we are at it also add default value for all new jdt core options
added in Eclipse 4.21.
[1] https://docs.oracle.com/en/java/javase/11/tools/javac.html
[2] https://docs.oracle.com/en/java/javase/14/docs/specs/man/javac.html#option-release
Change-Id: I852a5d7b0a3210751c15d79ec91915b4c01c41e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The reftable format is a block based format, but allows for variably
sized blocks. This obviously happens for reflog blocks (which are zlib
compressed), but is also accepted for index blocks: In the spec, this
is motivated as
To achieve constant O(1) disk seeks for lookups the index must be
a single level, which is permitted to exceed the file's
configured block size, but not the format's max block size of
15.99 MiB.
Hence, when parsing a block, one cannot be sure of its exact size:
after reading a default-size block (eg. 4kb), the block header may
state that the block is in fact larger.
Before, the code would mark the block as `truncated`, noting
// Its OK during sequential scan for an index block to have been
// partially read and be truncated in-memory. This happens when
// the index block is larger than the file's blockSize. Caller
// will break out of its scan loop once it sees the blockType.
This looks like either
* a remnant of never-implemented functionality. There is no reason to
ever sequentially scan an index block.
* alluding to sequential scan of the data blocks before the index
blocks (eg. scanning refs, which ends when we find the first ref index
block, and we can then ignore the index block).
This comment is followed by code that populates the
restartTbl/restartCnt fields relative to the (possibly truncated)
buffer. If the buffer is truncated, this essentially reads garbage,
leading to OOB array access when using the index block.
Fix this by dropping the truncated logic and issuing a second read if
the first read was short.
Add a test.
We have never observed this failure scenario at Google. We use 64kb
blocksize, which requires us to need fewer index entries. The reftable
spec mentions an Android repo of size 36M. With 64kb blocks, that's
just 562 index entries. Even with historical growth, we are long from
requiring an index whose size exceeds a single block.
When adding the analogous test for seeking refs, there was no failure.
This points to another possibility which is that the code tries to
avoid writing large index blocks for refs.
I did not investigate further which one it is.
Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=576250
Bug: 576250
Change-Id: I41ec21fac9e526ef57b3d6fb57b988bd353ee338
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Before, while trying to determine if an object ID was a tag or not,
the reftable conversion would yield an exception.
Change-Id: I3688a0ffa9e774ba27f320e3840ff8cada21ecf0
* master: (38 commits)
Revert "DFS block cache: Refactor to enable parallel index loading"
GitServlet: allow to override default error handlers
Silence API error for new interface method ProtocolV2Hook#onObjectInfo
transport: add object-info capability
Ignore IllegalStateException if JVM is already shutting down
Update orbit to R20210825222808 for 2021-09
Update spotbugs-maven-plugin to 4.3.0
Update ant to 1.10.11 also in pom.xml
DFS block cache: add additional stats to DfsReaderIoStats
Update Orbit to S20210817231813
[gpg] Better GPG home directory determination
FS: cleanup use of final modifier
Ensure FS#searchPath only selects executable files
RevWalk: getMergedInto's result is wrong on the second call
DFS block cache: Refactor to enable parallel index loading
[test] Create keystore with the keytool of the running JDK
[gpg] Update to Bouncy Castle 1.69
[test] Create keystore with the keytool of the running JDK
[sshd] Minor code clean-up
Support commit.template config property
...
Change-Id: I9f99e9a513a23c0c0d252334e79c351512d7355e
Thomas Wolf