Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.
FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.
Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).
To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.
Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
* stable-6.0:
Use slf4j-simple instead of log4j for logging
Update orbit to R20211213173813
Change-Id: I746b7fb71571020ce49f7b50fd675c9864327719
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Use slf4j-simple instead of log4j for logging
Update orbit to R20211213173813
Change-Id: I219ef3901c1d908b91bf9c8f00431b22686ff7a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit uses slf4j-api as logging API.
The libraries
- org.eclipse.jgit.http.test
- org.eclipse.jgit.pgm
- org.eclipse.jgit.ssh.apache.test
- org.eclipse.jgit.test
used the outdated log4j 1.2.15 which is EOL since years.
Since both jgit command line and also the tests don't need sophisticated
logging features replace log4j with the much simpler slf4j-simple log
implementation. The org.slf4j.binding.simple 1.7.30 archive has only
25kB instead of 429kB for log4j 1.2.15
Applications using jgit are free to choose any other log implementation
supporting slf4j API.
Change-Id: I89e85cd3c76e954c3434622510975ce65dc227d4
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.
The check was already performed when a new FileBasedConfig
is created at L158:
// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());
The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.
This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.
Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* stable-6.0:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Id7540d03991cdcf6f405e946b8cbbcc6a9696a31
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Ibd69e9d941ad9262b61dd0c4368e48cb82597a12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".
Follow the example below:
. .
└── src └── src
└── d1 ==> └── d1
└─ file1 ├─ file1
└── file2
c.parents[i] c
The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".
We should add an additional check for no paths matching the filter.
Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
Speed up bitmap creation by loading reverse index in parallel
to reading bitmap from storage. Latency changes from
(time_to_read_bitmap + time_to_load_reverse_index) to
max(time_to_read_bitmap, time_to_load_reverse_index).
Add new option to DfsReaderOptions to control parallel reverse index
loading. Static cached thread pool is added to PackBitmapIndexV1 for
reverse index loading, and when not in use consumes minimal resources.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ia37a1d739631d053e8bddb925ac8b0b81d22379e
Let GC#gc return collection of newly created packs as CompletableFuture
to enable using gc() asynchronously.
Change-Id: I3627014fd458c738cfe54225e631d6f7d9cfb1a7
* stable-6.0:
FS: debug logging only if system config file cannot be found
FS: debug logging only if system config file cannot be found
Update .factorypath used by annotation processor for benchmarks
Use maven-compiler-plugin's release tag instead of source and target
Don't use deprecated Repository#getAllRefs in Repository
Don't use deprecated Repository#getAllRefs in FileRepository
RevListTest: fix warning that method parameter hides field 'git'
Implement RecordingLogger based on org.slf4j.Logger
Let ObjectDatabase implement AutoClosable
Change-Id: Ie6b3cfa66b319033d4448dcf20362b753c0e9d7c
see: http://git-scm.com/docs/git-difftool
* add command line support for "jgit difftool"
* show supported commands with "jgit difftool --help"
* added "git difftool --tool-help" to show the tools (empty now)
* prepare for all other commands
Bug: 356832
Change-Id: Ice0c13ef7953a20feaf25e7746d62b94ff4e89e5
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- AdvertiseRefsHook#advertiseRefs
- ReceivePack#setAdvertisedRefs
- Repository#getAdditionalHaves
Bug: 534731
Change-Id: I85bc6ce5815d40be5f80042c53f4663072d96be5
Provide a public interface PackLock exposing only the unlock() method.
Rename the internal PackLock class to PackLockImpl and have it implement
the new interface.
This way PackParser doesn't expose an internal class via its API
anymore, and client can still unlock pack locks that were created.
Bug: 576340
Change-Id: I976739f4ab28fe1f9ba7f35653a69a913aa68841
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fixes an issue that commit template file could not be found if it has a
relative path instead of absolute path.
Relative path is probably common if git config --local is used.
Bug: 446355
Change-Id: I8ddf2be672647be825fd9c01af82809d31bb8356
With change https://git.eclipse.org/r/c/jgit/jgit/+/186455 a thread
loading a bitmap index could hold two ref locks at the same time (one
for bitmap and one for either index or reverse index). So it is possible
that two threads loading bitmaps end up in a deadlock for ref locks e.g.
threadA has refLock[1] (for bitmap) and wants refLock[2] (for index or
revIndex) and threadB has refLock[2] (for bitmap) and wants refLock[1].
This change introduces separate pools of locks per pack extension
instead of a shared pool. So threads loading bitmap can hold two
locks but with different extensions and no overlap, e.g. threadA holds
refLock[BITMAP_INDEX][1] and refLock[INDEX][2] and threadB holds
refLock[BITMAP_INDEX][2] and refLock[INDEX][1].
More unit tests were added to cover various paralell loading scenarios.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I89704b4721c21548929608d3798ef60925280755
Bring our SSH config parser up-to-date with respect to changes in
OpenSSH. In particular, they fixed[1] the handling of line comments
such that #-characters inside strings are not considered. This means
that we have to parse strings with escaped quotes correctly.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=3288
Change-Id: Ifbd9014127e8d51e7c8792e237f3fc2a9a0719d2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is a prerequisite for removing the JSch support bundle; otherwise
OpenSshConfigFile would be left without tests.
Copy OpenSshConfigTest from the JSch support bundle and adapt all tests
to perform the equivalent checks on OpenSshConfigFile directly. Add a
new lookupDefault() method to the SshConfigStore interface and implement
it so that it behaves the same and the tests work identically.
Change-Id: I046abd9197a8484003e77005024e5d973456f1a3
C git considers not only files containing NUL bytes as binary but also
files containing lone CRs. Implement this also for JGit.
C git additionally counts printable vs. non-printable characters and
considers files that have non_printable_count > printable_count / 128
also as binary. This is not implemented because such counting probably
only makes sense if one looks at the full file or blob content. The
Auto[CR]LF* streams in JGit look only at the first few KiB of a stream
in order not to buffer too much.
For the C git implementation, see [1].
[1] https://github.com/git/git/blob/7e27bd589d/convert.c#L35
Bug: 576971
Change-Id: Ia169b59bdbf1477f32ee2014eeb8406f81d4b1ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Move the code to parse numbers with an optional 'k', 'm', or 'g' suffix
from the config file handling to StringUtils. This enables me to re-use
it in EGit, which has duplicate code in StorageSizeFieldEditor.
As this is generally useful functionality, providing it in the library
makes sense.
Change-Id: I86e4f5f62e14f99b35726b198ba3bbf1669418d9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The various streams used in JGit for text/binary and CR-LF detection
used different buffer sizes. Most used 8000, but one used 8KiB, and one
used 8096 (SIC!) bytes.
Considering only the first 8kB of a file/blob is not sufficient; it
may give behavior incompatible with C git. C git considers the whole
blob; since it uses memory-mapped files it can do so with acceptable
performance. Doing this in JGit would most likely incur a noticeable
performance penalty. But 8kB is a bit small; in the file in bug 576971
the limit was hit before the first CR-LF, which occurred on line 155
at offset 9759 in the file.
Make RawText.FIRST_FEW_BYTES only a default and minimum setting, and
set it to 8KiB. Make the actual buffer size configurable: provide
static methods getBufferSize() and setBuffersize(), and use
getBufferSize() throughout instead of the constant.
This enables users of the JGit library to set their own possibly larger
buffer size.
Bug: 576971
Change-Id: I447762c9a5147a521f73d2864ba59ed89f555d54
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for files having been checked in with mixed LF and CR-LF
line endings, and then being checked out with text or text=auto and
eol=lf or eol=crlf. These test cases were missing, and thus commit
efd1cc05 missed that AutoCRLFOutputStream needs the same detection
as AutoLFOutputStream.
Fix AutoCRLFOutputStream to not convert line endings if the blob in
the repository contains CR-LF.
Bug: 575393
Change-Id: Id0c7ae772e282097e95fddcd3f1f9d82aae31e43
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JEP 400 (Java 18) will change the default character set to UTF-8
unconditionally.[1] Introduce SystemReader.getDefaultCharset() that
provides the locale-dependent charset the way JEP 400 recommends.
Change all code locations using Charset.defaultCharset() to use the
new SystemReader method instead.
[1] https://openjdk.java.net/jeps/400
Change-Id: I986f97a410d2fc70748b6f93228a2d45ff100b2c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This change is a fix to http://git.eclipse.org/r/c/jgit/jgit/+/183562
that was reverted in http://git.eclipse.org/r/c/jgit/jgit/+/184978
due to deadlocks. Separate locks in DfsBlockFile are removed to rely
on getting value from DfsBlockCache with region locking in place.
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index
and index could be read from storage in parallel in separate threads.
A unit test is added for parallel index loading.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ic6d9c5a4a254628636aa98a5008447a27a003f69
* stable-5.13:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Add org.bouncycastle.bcutil to p2 repository
Change-Id: I789e3c84045cc9dcd8a26de34d09fc1e3ccc41e7
* stable-5.12:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
* stable-5.11:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
* stable-5.10:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
* stable-5.9:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
We observed the following issue:
$ git tag -a v1.0 -m v1.0
$ git push origin tag v1.0
$ git ls-remote origin v1.0^{}
... empty result ...
On the server (Gerrit) side run git-gc to pack the refs:
$ git gc
Repeat the ls-remote from the client and the result is correct:
$ git ls-remote origin v1.0^{}
7ad85c810de3ae922903d4bdd17c53cd627260ba refs/tags/v1.0^{}
Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.
It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.
Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Let certain pack extensions that are expensive to load from storage
(e.g. pack index, bitmap index) stay in DFS block cache longer than
others by overriding default cache count through DfsBlockCacheConfig
Don't change default behavior when cache override map is empty. Use int
cacheCount instead of boolean hot for Ref<T>
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I18062784ec9cc14dbba3e4bb8d9509440cf2d44f
If an annotated tag refers to a commit, we should not ignore it.
Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.
Example:
As is shown in the graph below:
1 - 2 - 3 - 4 (side)
\
5 - 6^ (master) - 7 (topic)
Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].
Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:
commit | Before | After | Rel %
47a44d27ca | 29251ms | 6687ms | -77%
90327e7dff | 21388ms | 6256ms | -70%
f85fac0efa | 11150ms | 7338ms | -34%
The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.
Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
* stable-5.8:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
* stable-5.7:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
* stable-5.6:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
In Java 11 type arguments for anonymous subclasses can be inferred
and don't need to be specified. This resolves a number of compiler
warnings.
Change-Id: I55eff3babb7628aa0627085e65a1b45eb12c2cd3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
On Java 16 this test throws InaccessibleObjectException, handle and
ignore it similar to IllegalAccessException.
Change-Id: I19b4f577579694a146516861a7ec567141f3464b
This ensures the compiler compiles against the public, supported and
documented API for a specific VM version (here 11) [1]. This also means
that
we don't need EE descriptors in Eclipse anymore in order to ensure that
only supported APIs of the selected Java version can be used.
According to [2] if option --release is used --source and --target
options can't be used.
While we are at it also add default value for all new jdt core options
added in Eclipse 4.21.
[1] https://docs.oracle.com/en/java/javase/11/tools/javac.html
[2] https://docs.oracle.com/en/java/javase/14/docs/specs/man/javac.html#option-release
Change-Id: I852a5d7b0a3210751c15d79ec91915b4c01c41e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The reftable format is a block based format, but allows for variably
sized blocks. This obviously happens for reflog blocks (which are zlib
compressed), but is also accepted for index blocks: In the spec, this
is motivated as
To achieve constant O(1) disk seeks for lookups the index must be
a single level, which is permitted to exceed the file's
configured block size, but not the format's max block size of
15.99 MiB.
Hence, when parsing a block, one cannot be sure of its exact size:
after reading a default-size block (eg. 4kb), the block header may
state that the block is in fact larger.
Before, the code would mark the block as `truncated`, noting
// Its OK during sequential scan for an index block to have been
// partially read and be truncated in-memory. This happens when
// the index block is larger than the file's blockSize. Caller
// will break out of its scan loop once it sees the blockType.
This looks like either
* a remnant of never-implemented functionality. There is no reason to
ever sequentially scan an index block.
* alluding to sequential scan of the data blocks before the index
blocks (eg. scanning refs, which ends when we find the first ref index
block, and we can then ignore the index block).
This comment is followed by code that populates the
restartTbl/restartCnt fields relative to the (possibly truncated)
buffer. If the buffer is truncated, this essentially reads garbage,
leading to OOB array access when using the index block.
Fix this by dropping the truncated logic and issuing a second read if
the first read was short.
Add a test.
We have never observed this failure scenario at Google. We use 64kb
blocksize, which requires us to need fewer index entries. The reftable
spec mentions an Android repo of size 36M. With 64kb blocks, that's
just 562 index entries. Even with historical growth, we are long from
requiring an index whose size exceeds a single block.
When adding the analogous test for seeking refs, there was no failure.
This points to another possibility which is that the code tries to
avoid writing large index blocks for refs.
I did not investigate further which one it is.
Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=576250
Bug: 576250
Change-Id: I41ec21fac9e526ef57b3d6fb57b988bd353ee338
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Before, while trying to determine if an object ID was a tag or not,
the reftable conversion would yield an exception.
Change-Id: I3688a0ffa9e774ba27f320e3840ff8cada21ecf0
* master: (38 commits)
Revert "DFS block cache: Refactor to enable parallel index loading"
GitServlet: allow to override default error handlers
Silence API error for new interface method ProtocolV2Hook#onObjectInfo
transport: add object-info capability
Ignore IllegalStateException if JVM is already shutting down
Update orbit to R20210825222808 for 2021-09
Update spotbugs-maven-plugin to 4.3.0
Update ant to 1.10.11 also in pom.xml
DFS block cache: add additional stats to DfsReaderIoStats
Update Orbit to S20210817231813
[gpg] Better GPG home directory determination
FS: cleanup use of final modifier
Ensure FS#searchPath only selects executable files
RevWalk: getMergedInto's result is wrong on the second call
DFS block cache: Refactor to enable parallel index loading
[test] Create keystore with the keytool of the running JDK
[gpg] Update to Bouncy Castle 1.69
[test] Create keystore with the keytool of the running JDK
[sshd] Minor code clean-up
Support commit.template config property
...
Change-Id: I9f99e9a513a23c0c0d252334e79c351512d7355e
Sometimes it is useful to obtain metadata associated with an object
without the need to first download it locally. This is specially useful
when using partial clones.
This change implements the object-info capability that allows clients to
query the remote server for object metadata (currently only size). This
is a backport of the same capability that was recently added to the Git
project a2ba162cda (object-info: support for retrieving object info,
2021-04-20).
Signed-off-by: Bruno Albuquerque <bga@google.com>
Change-Id: I4dc9828e1c247f08b0976b8810be92d124366165
Make sure the future user can reset all UNINTERESTING commmits after
this operation.
Signed-off-by: kylezhao <kylezhao@tencent.com>
Change-Id: I7549b9ff67bd31acd5dfc92331cb9a30b47b8278
* stable-5.12:
[test] Create keystore with the keytool of the running JDK
Fix garbage collection failing to delete pack file
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I4b54f4850819736144edb784617ee902f491ffd6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: Ie8db450a1fad05bddb812a55b2ceb03b2805403a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I55f4dd19e9fa3b789bd9a79d256fe9abb55ee7f4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: Ic37426211905d987ddd11480a54d95b86143c94c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.8:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: If09cbb877c674f15261715cecef7a2393bf66fa3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.7:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I32010c6bf45d5138e17143d6c284ac56434eade1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.6:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I1f6b4fc26f6ee6f22cc0aacd032c1e73ba246dbc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Adds functionality to read the git commit.template property. The
template content is read either via a default encoding or, if present,
via encoding specified by i18n.commitEncoding property.
Bug: 446355
Change-Id: I0c45db98e324ddff26a7e0262835f259d6528a86
Signed-off-by: Julian Ruppel <julian.ruppel@sap.com>
We currently cannot use hamcrest 2.2 and assertj 3.20.2 (which requires
hamcrest 2.2) in egit tests since other Eclipse dependencies (e.g.
swtbot) require hamcrest 1.1.
Hence relax version range for these components in jgit so that jgit
tests also work when using the egit target platform.
Change-Id: I521e2ec4491bd8d790609b8a66a8f14511a865a1
and update
- assertj to 3.20.2.v20210706-1104
- hamcrest to 2.2.0.v20210711-0821
- classes which were in org.hamcrest.core 1.3 and org.hamcrest.library
1.3 were all moved to org.hamcrest in 2.2
- the annotation org.hamcrest.Factory was removed and is no longer
needed
- junit 4.13 requires hamcrest-core and hamcrest-library 1.3 therefore
keep them in the target platform
CQ: 23501
Change-Id: Ife871c0343b611be9203aed7f86577e85bbf5c95
* master:
searchForReuse might impact performance in large repositories
Retry loose object read upon "Stale file handle" exception
Ignore missing javadoc in test bundles
Upgrade maven-dependency-plugin to 3.2.0
Upgrade jacoco-maven-plugin to 0.8.7
Upgrade maven-jxr-plugin to 3.1.1
Fix garbage collection failing to delete pack file
Fix PathSuffixFilter: can decide only on full paths
Update jetty to 9.4.42.v20210604
[sshd] Log the full KEX negotiation result
[releng] japicmp: update last release version
CONTRIBUTING: add explicit link to ECA
CONTRIBUTING: Use standard markdown format
Add Eclipse code of conduct and security policy
Change-Id: I9e31cb6e22f5398db6e220f71693e87475c598bd
The search for reuse phase for *all* the objects scans *all*
the packfiles, looking for the best candidate to serve back to the
client.
This can lead to an expensive operation when the number of
packfiles and objects is high.
Add parameter "pack.searchForReuseTimeout" to limit the time spent
on this search.
Change-Id: I54f5cddb6796fdc93ad9585c2ab4b44854fa6c48
When reading loose objects over NFS it is possible that the OS syscall
would fail with ESTALE errors: This happens when the open file
descriptor no longer refers to a valid file.
Notoriously it is possible to hit this scenario when git data is shared
among multiple clients, for example by multiple gerrit instances in HA.
If one of the two clients performs a GC operation that would cause the
packing and then the pruning of loose objects, the other client might
still hold a reference to those objects, which would cause an exception
to bubble up the stack.
The Linux NFS FAQ[1] (at point A.10), suggests that the proper way to
handle such ESTALE scenarios is to:
"[...] close the file or directory where the error occurred, and reopen
it so the NFS client can resolve the pathname again and retrieve the new
file handle."
In case of a stale file handle exception, we now attempt to read the
loose object again (up to 5 times), until we either succeed or encounter
a FileNotFoundException, in which case the search can continue to
Packfiles and alternates.
The limit of 5 provides an arbitrary upper bounds that is consistent to
the one chosen when handling stale file handles for packed-refs
files (see [2] for context).
[1] http://nfs.sourceforge.net/
[2] https://git.eclipse.org/r/c/jgit/jgit/+/54350
Bug: 573791
Change-Id: I9950002f772bbd8afeb9c6108391923be9d0ef51
On a subtree, a PathSuffixFilter must return -1 ("indeterminate"),
not 0 ("include"), otherwise negation goes wrong: an indeterminate
result (-1) is passed on, but a decision (0/1) is inverted.
As a result a negated PathSuffixFilter would skip all folders.
Bug: 574253
Change-Id: I27fe785c0d772392a5b5efe0a7b1c9cafcb6e566
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
From Git commit 9c1e657a8f:
Currently, the packfile negotiation step within a Git fetch cannot be
done independent of sending the packfile, even though there is at
least one application wherein this is useful - push negotiation.
Therefore, make it possible for this negotiation step to be done
independently.
This feature is for protocol v2 only.
In the protocol, the main hindrance towards independent negotiation is
that the server can unilaterally decide to send the packfile. This is
solved by a "wait-for-done" argument: the server will then wait for
the client to say "done". In practice, the client will never say it;
instead it will cease requests once it is satisfied.
Advertising the server capability option "wait-for-done" is behind the
transport config: uploadpack.advertisewaitfordone, which by default is
false.
Change-Id: I5ebd3e99ad76b8943597216e23ced2ed38eb5224
This is similar to change Idbc2c29bd that skipped detecting content
renames for large files. With this change, we added a new option in
RenameDetector called "skipContentRenamesForBinaryFiles", that when set,
causes binary files with any slight modification to be identified as
added/deleted. The default for this boolean is false, so preserving
current behaviour.
Change-Id: I4770b1f69c60b1037025ddd0940ba86df6047299
The "branch" field in the .gitmodules is the signal for gerrit to keep
the superproject autoupdated. Tags are immutable and there is no need to
track them, plus the cgit client requires the field to be a "remote
branch name" but not a tag.
Do not set the "branch" field if the revision is a tag. Keep those tags
in another field ("ref") as they help other tools to find the commit in
the destination repository.
We can still have false negatives when a refname is not fully qualified,
but this check covers e.g. the most common case in android.
Note that the javadoc of #setRecordRemoteBranch already mentions that
"submodules that request a tag will not have branch name recorded".
Change-Id: Ib1c321a4d3b7f8d51ca2ea204f72dc0cfed50c37
Signed-off-by: Ivan Frade <ifrade@google.com>
Check the last line of the last hunk of a file, not the last line of
the whole patch.
Note that C git only checks that this line starts with "\ " and is at
least 12 characters long because of possible different texts when non-
English messages are used.
Change-Id: I0db81699eb3e99ed7b536a3e2b8dc97df1f58a89
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
C git treats completely empty lines as empty context lines (which
traditionally have a single blank). Apparently newer GNU diff may
produce such lines; see [1]. ("Newer" meaning "since 2006"...)
[1] https://github.com/git/git/commit/b507b465f7831
Change-Id: I80c1f030edb17a46289b1dabf11a2648d2660d38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Instead of converting the patch bytes to strings apply the patch on
byte level, like C git does. Converting the input lines and the hunk
lines from bytes to strings and then applying the patch based on
strings may give surprising results if a patch converts a text file
from one encoding to another. Moreover, in the end we don't know which
encoding to use to write the result.
Previous code just wrote the result as UTF-8, which forcibly changed
the encoding if the original input had some other encoding (even if the
patch had the same non-UTF-8 encoding). It was also wrong if the input
was UTF-8, and the patch should have changed the encoding to something
else.
So use ByteBuffers instead of Strings. This has the additional advantage
that all these ByteBuffers can share the underlying byte arrays of the
input and of the patch, so it also reduces memory consumption.
Change-Id: I450975f2ba0e7d0bec8973e3113cc2e7aea187ee
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Implement applying binary patches. Handles both literal and delta
patches. Note that C git also runs binary files through the clean
and smudge filters. Implement the same safeguards against corrupted
patches as in C git: require the full OIDs to be present in the patch
file, and apply a binary patch only if both pre- and post-image hashes
match.
Add tests for applying literal and delta patches.
Bug: 371725
Change-Id: I71dc214fe4145d7cc8e4769384fb78c7d0d6c220
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new BinaryDeltaInputStream that applies a delta provided by
another InputStream to a given base. Because delta application needs
random access to the base, the base itself cannot be yet another
InputStream. But at least this enables streaming of the result.
Add a simple test using delta hunks generated by C git.
Bug: 371725
Change-Id: Ibd26fa2f49860737ad5c5387f7f4870d3e85e628
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add streams that can encode or decode git binary patch data on the fly.
Git writes binary patches base-85 encoded, at most 52 un-encoded bytes,
with the unencoded data length prefixed in a one-character encoding, and
suffixed with a newline character.
Add a test for both the new input and the output stream. The test
roundtrips binary data of different lengths in different ways.
Bug: 371725
Change-Id: Ic3faebaa4637520f5448b3d1acd78d5aaab3907a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add an implementation for base-85 encoding and decoding [1]. Git binary
patches use this format.
Base-85 encoding assembles bytes as 32-bit MSB values, then converts
these values to base-85 numbers (always 5 bytes) encoded as printable
ASCII characters. Decoding base-85 is the reverse operation. Note
that decoding may overflow on invalid input as 85^5 > 2^32. Encodings
always have a length that is a multiple of 5. If input length is not
divisible by 4, padding bytes are (logically) added, which are ignored
when decoding. The encoding for n bytes has thus always exactly length
(n + 3) / 4 * 5 in integer arithmetic (truncating division).
Includes tests.
[1] https://datatracker.ietf.org/doc/html/rfc1924
Bug: 371725
Change-Id: Ib5b9a503cd62cf70e080a4fb38c8cd1eeeaebcfe
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Update tests to record the number of events fired post-setup and only
assert for events fired during BatchRefUpdate.execute. For tests which
use writeLooseRef to setup refs, create new tests which assert the
number of RefsChangedEvent(s) rather than updating the existing ones
to call RefDirectory.exactRef as it changes the code path.
Change-Id: I0187811628d179d9c7e874c9bb8a7ddb44dd9df4
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Applying a patch on Windows failed if the patch had the (normal)
single-LF line endings, but the file on disk had the usual Windows
CR-LF line endings.
Git (and JGit) compute diffs on the git-internal blob, i.e., after
CR-LF transformation and clean filtering. Applying patches to files
directly is thus incorrect and may fail if CR-LF settings don't
match, or if clean/smudge filtering is involved.
Change ApplyCommand to run the file content through the check-in
filters before applying the patch, and run the result through the
check-out filters. This makes patch application succeed even if the
patch has single-LFs, but the file has CR-LF and core.autocrlf is
true.
Add tests for various combinations of line endings in the file and in
the patch, and a test to verify the clean/smudge handling.
See also [1].
Running the file though clean/smudge may give strange results with
LFS-managed files. JGit's DiffFormatter has some extra code and
applies the smudge filter again after having run the file through
the check-in filters (CR-LF and clean). So JGit can actually produce
a diff on LFS-managed files using the normal diff machinery. (If it
doesn't run out of memory, that is. After all, LFS is intended for
_large_ files.) How such a diff would be applied with either C git
or JGit is entirely unclear; neither has any code for this special
case. Compare also [2].
Note that C git just doesn't know about LFS and always diffs after
the check-in filter chain, so for LFS files, it'll produce a diff
of the LFS pointers.
[1] https://github.com/git/git/commit/c24f3abac
[2] https://github.com/git-lfs/git-lfs/issues/440
Bug: 571585
Change-Id: I8f71ff26313b5773ff1da612b0938ad2f18751f5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* master: (34 commits)
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
LockFile: create OutputStream only when needed
Add a cgit interoperability test for LockFile
Add TemporaryBuffer.toString(int limit)
LockFile: create OutputStream only when needed
Prepare 5.12.0-SNAPSHOT builds
JGit v5.12.0.202105051250-m2
Update jetty to 9.4.40.v20210413
[releng] Update eclipse-jarsigner-plugin to 1.3.1
Implement ours/theirs content conflict resolution
ssh: ensure list is modifiable before using Iterator.remove().
Update orbit to S20210406213021 and add 4.20-staging target
Fix typo in test method name
Allow file mode conflicts in virtual base commit on recursive merge.
sshd: don't lock the known_hosts files on reading
Allow info messages in UsernamePasswordCredentialsProvider
ssh config: do environment variable replacement
sshd: implement server-sig-algs SSH extension (client side)
Upgrade ecj to 3.25.0
...
Change-Id: Ibc39a9c4e431d15b67ab4a307241f47a7f3740a9
* stable-5.9:
LockFile: create OutputStream only when needed
Remove ReftableNumbersNotIncreasingException
Fix stamping to produce stable file timestamps
Change-Id: I056382d1d93f3e0a95838bdd1f0be89711c8a722
Don't create the stream eagerly in lock(); that may cause JGit to
exceed OS or JVM limits on open file descriptors if many locks need
to be created, for instance when creating many refs. Instead create
the output stream only when one really needs to write something.
Bug: 573328
Change-Id: If9441ed40494d46f594a896d34a5c4f56f91ebf4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Don't create the stream eagerly in lock(); that may cause JGit to
exceed OS or JVM limits on open file descriptors if many locks need
to be created, for instance when creating many refs. Instead create
the output stream only when one really needs to write something.
Bug: 573328
Change-Id: If9441ed40494d46f594a896d34a5c4f56f91ebf4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Git has different conflict resolution strategies:
* There is a tree merge strategy "ours" which just ignores any changes
from theirs ("-s ours"). JGit also has the mirror strategy "theirs"
ignoring any changes from "ours". (This doesn't exist in C git.)
Adapt StashApplyCommand and CherrypickCommand to be able to use those
tree merge strategies.
* For the resolve/recursive tree merge strategies, there are content
conflict resolution strategies "ours" and "theirs", which resolve
any conflict hunks by taking the "ours" or "theirs" hunk. In C git
those correspond to "-Xours" or -Xtheirs". Implement that in
MergeAlgorithm, and add API to set and pass through such a strategy
for resolving content conflicts.
* The "ours/theirs" content conflict resolution strategies also apply
for binary files. Handle these cases in ResolveMerger.
Note that the content conflict resolution strategies ("-X ours/theirs")
do _not_ apply to modify/delete or delete/modify conflicts. Such
conflicts are always reported as conflicts by C git. They do apply,
however, if one side completely clears a file's content.
Bug: 501111
Change-Id: I2c9c170c61c440a2ab9c387991e7a0c3ab960e07
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Similar to https://git.eclipse.org/r/c/jgit/jgit/+/175166, ignore
path that have conflicts on attributes, so that the virtual base could
be used by RecursiveMerger.
Change-Id: I99c95445a305558d55bbb9c9e97446caaf61c154
Signed-off-by: Marija Savtchouk <mariasavtchouk@google.com>
* master: (27 commits)
Optimize RevWalkUtils.findBranchesReachableFrom()
Introduce getMergedInto(RevCommit commit, Collection<Ref> refs)
Skip detecting content renames for large files
Remove unused API problem filters
Document http options supported by JGit
HTTP cookies: do tilde expansion on http.cookieFile
Prepare 5.12.0-SNAPSHOT builds
Update Orbit to R20210223232630
Prepare 5.11.1-SNAPSHOT builds
JGit v5.11.0.202103091610-r
Manually set status of jmh dependencies
Update DEPENDENCIES report for 5.11.0
Add dependency to dash-licenses
PackFile: Add id + ext based constructors
GC: deleteOrphans: Use PackFile
PackExt: Convert to Enum
Restore preserved packs during missing object seeks
Pack: Replace extensions bitset with bitmapIdx PackFile
PackDirectory: Use PackFile to ensure we find preserved packs
GC: Use PackFile to de-dup logic
...
Change-Id: I2326d4d728fbde3090a5b87b0e273db46e0c5f62
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In cases where we need to determine if a given commit is merged
into many refs, using isMergedInto(base, tip) for each ref would
cause multiple unwanted walks.
getMergedInto() marks the unreachable commits as uninteresting
which would then avoid walking that same path again.
Using the same api, also introduce isMergedIntoAny() and
isMergedIntoAll()
Change-Id: I65de9873dce67af9c415d1d236bf52d31b67e8fe
Signed-off-by: Adithya Chakilam <quic_achakila@quicinc.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There are two code paths for detecting renames: one on tree diffs
(using DiffFormatter#scan) and the other on single file diffs (using
DiffFormatter#format). The latter skips binary and large files
for rename detection - check [1], but the former doesn't.
This change skips content rename detection for the tree diffs case for
large files. This is essential to avoid expensive computations while
reading the file, especially for callers who don't want to pay that
cost. Content renames are those which involve files with slightly
modified content. Exact renames will still be identified.
The default threshold for file sizes is reused from
PackConfig.DEFAULT_BIG_FILE_THRESHOLD: 50 MB.
[1] 232876421d/org.eclipse.jgit/src/org/eclipse/jgit/diff/RawText.java (386)
Change-Id: Idbc2c29bd381c6e387185204638f76fda47df41e
Signed-off-by: Youssef Elghareeb <ghareeb@google.com>
Add new constructors to PackFile to improve a common use case where
callers know the directory, id, and extension, but previously needed to
construct a valid file name (with prefix, '.', etc) to create a
PackFile. Most callers can use the variant that has id as an ObjectId,
but provide an id as String variant too.
Change-Id: I39e4466abe8c9509f5916d5bfe675066570b8585
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
Provide a recovery path for objects being referenced during the pack
pruning race. Due to the pack pruning race, it is possible for objects
to become referenced after a pack has been deemed safe to prune, but
before it actually gets pruned. If this happened previously, the newly
referenced objects would be missing and potentially result in a
corrupted ref.
Add the ability to recover from this situation when an object is missing
but happens to still be available in a pack in the "preserved"
directory. This is likely only useful when used in conjunction with the
--preserve-old-packs GC option, which prunes packs by hard-linking to
the preserved directory. If an object is missing and found in a pack in
the preserved directory, immediately recover that pack and its
associated files (idx, bitmaps...) by moving them back to the original
pack directory, and then retry the operation that would have failed due
to the missing object. This retry can now succeed and the repository
may avoid corruption. This approach should drastically reduce the
chance of a corrupt repository during pack pruning at very little extra
cost. This extra cost should only be incurred when objects are missing
and a failure would normally occur.
Change-Id: I2a704e3276b88cc892159d9bfe2455c6eec64252
Signed-off-by: Martin Fick <quic_mfick@quicinc.com>
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
The only extension that was ever consulted from the bitmap was the
bitmap index. We can simplify the Pack code as well as the code of
all the callers if we focus on just that usage.
Change-Id: I799ddfdee93142af67ce5081d14a430d36aa4c15
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
The PackFile class is intended to be a central place to do all
common pack filename manipulation and parsing to help reduce repeated
code and bugs. Use the PackFile class in the Pack class and in many
tests to ensure it works well in a variety of situations. Later changes
will expand use of PackFiles to even more areas.
Change-Id: I921b30f865759162bae46ddd2c6d669de06add4a
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A cookie file stores the expiration in seconds since the Linux Epoch,
not in milliseconds. Correct reading and writing cookie files; with
a backwards-compatibility hack to read files that contain a millisecond
timestamp.
Add a test, and fix tests not to rely on the actual current time so
that they will also run successfully after 2030-01-01 noon.
Bug: 571574
Change-Id: If3ba68391e574520701cdee119544eedc42a1ff2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* master: (143 commits)
Prepare 5.11.0-SNAPSHOT builds
JGit v5.11.0.202102240950-m3
[releng] japicmp: update last release version
IgnoreNode: include path to file for invalid .gitignore patterns
FastIgnoreRule: include bad pattern in log message
init: add config option to set default for the initial branch name
init: allow specifying the initial branch name for the new repository
Fail clone if initial branch doesn't exist in remote repository
GPG: fix reading unprotected old-format secret keys
Update Orbit to S20210216215844
Add missing bazel dependency for o.e.j.gpg.bc.test
GPG: handle extended private key format
dfs: handle short copies
[GPG] Provide a factory for the BouncyCastleGpgSigner
Fix boxing warnings
GPG: compute the keygrip to find a secret key
GPG signature verification via BouncyCastle
Post commit hook failure should not cause commit failure
Allow to define additional Hook classes outside JGit
GitHook: use default charset for output and error streams
...
Change-Id: I689f4070e79f4a0ac1c02b35698ccaab68ad2f34
We introduced the option --initial-branch=<branch-name> to allow
initializing a new repository with a different initial branch.
To allow users to override the initial branch name more permanently
(i.e. without having to specify the name manually for each 'git init'),
introduce the 'init.defaultBranch' option.
This option was added to git in 2.28.0.
See https://git-scm.com/docs/git-config#Documentation/git-config.txt-initdefaultBranch
Bug: 564794
Change-Id: I679b14057a54cd3d19e44460c4a5bd3a368ec848
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add option --initial-branch/-b to InitCommand and the CLI init command.
This is the first step to implement support for the new option
init.defaultBranch. Both were added to git in release 2.28.
See https://git-scm.com/docs/git-init#Documentation/git-init.txt--bltbranch-namegt
Bug: 564794
Change-Id: Ia383b3f90b5549db80f99b2310450a7faf6bce4c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Pack better represents the purpose of the object and paves the way to
add a PackFile object that extends File.
Change-Id: I39b4f697902d395e9b6df5e8ce53078ce72fcea3
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
If RecursiveMerger finds multiple base commits, it tries to compute
the virtual ancestor to use as a base for the three way merge.
Currently, the content conflicts between ancestors are ignored (file
staged with the conflict markers). If the path is a file in one ancestor
and a dir in the other, it results in NoMergeBaseException
(CONFLICTS_DURING_MERGE_BASE_CALCULATION).
Allow these conflicts by ignoring this unmerged path in the virtual
base. The merger will compute diff in the children instead and it
can be further fixed manually if needed.
Change-Id: Id59648ae1d6bdf300b26fff513c3204317b755ab
Signed-off-by: Marija Savtchouk <mariasavtchouk@google.com>
DateRevQueue is expected to give out the commits that have higher
commit time. But in case of tie(same commit time), it should give
the commit that is inserted first. This is inferred from the
testInsertTie test case written for DateRevQueue. Also that test
case, right now uses just two commits which caused it not to fail
with the current implementation, so added another commit to make
the test more robust.
By fixing the DateRevQueue, we would also match the behaviour of
LogCommand.addRange(c1,c2) with git log c1..c2. A test case for
the same is added to show that current behaviour is not the
expected one.
By fixing addRange(), the order in which commits are applied during
a rebase is altered. Rebase logic should have never depended upon
LogCommand.addRange() since the intended order of addRange() is not
the order a rebase should use. So, modify the RebaseCommand to use
RevWalk directly with TopoNonIntermixSortGenerator.
Add a new LogCommandTest.addRangeWithMerge() test case which creates
commits in the following order:
A - B - C - M
\ /
-D-
Using git 2.30.0, git log B..M outputs: M C D
LogCommand.addRange(B, M) without this fix outputs: M D C
LogCommand.addRange(B, M) with this fix outputs: M C D
Change-Id: I30cc3ba6c97f0960f64e9e021df96ff276f63db7
Signed-off-by: Adithya Chakilam <achakila@codeaurora.org>
This would run into an endless loop if the offset given was not zero.
Fix the logic to exit the read loop when the buffer is full.
Luckily all existing uses of this method call it only with offset zero.
Change-Id: I0ec2a4fb43efe4a605d06ac2e88cf155d50e2f1e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reachability checkers are retrieved from RevWalk and ObjectWalk objects:
* RevWalk.createReachabilityChecker()
* ObjectWalk.createObjectReachabilityChecker()
Since RevWalks and ObjectWalks are themselves directly instantiated
in hundreds of places (e.g. UploadPack...) overriding them in a
consistent way requires overloading 100s of methods, which isn't
feasible. Moving reachability checker generation to a more central
place solves that problem.
The ObjectReader object seems a good place from which to get
reachability checkers, because reachability checkers return
information about relationships between objects. ObjectDatabases
delegate many operations to ObjectReaders, and reachability bitmaps
are attached to ObjectReaders.
The Bitmapped and Pedestrian reachability checker objects were
package private in the org.eclipse.jgit.revwalk package. This change
makes them public and moves them to the
org.eclipse.jgit.internal.revwalk package. Corresponding tests are
also moved.
Motivation:
1) Reachability checking algorithms need to scale. One of the
internal Android repositories has ~2.4 million refs/changes/*
references, causing bad long tail performance in reachability
checks.
2) Reachability check performance is impacted by repository
topography: number of refs, number of objects, amounts of
related vs. unrelated history.
3) Reachability check performance is also affected by per-branch
access (Gerrit branch permissions) since different users can
see different branches.
4) Reachability check performance isn't affected by any state in a
RevWalk or ObjectWalk.
I don't yet know if a single algorithm will work for all cases in #2
and #3. We may need to evolve the ReachabilityChecker interfaces
over time to solve the Gerrit branch permissions case, or use
Gerrit-specific identity information to solve that in an efficient
way.
This change takes the existing public API and moves it to the
ObjectReader/whole repository level, which is where we can do
consistent customizations for #2 and #3. We intend to upstream the
best of whatever works, but anticipate the need for multiple rounds
of experimentation.
Change-Id: I9185feff43551fb387957c436112d5250486833d
Signed-off-by: Terry Parker <tparker@google.com>
Different archive formats support a compression level in the range
[0-9]. The value 0 is for lowest compressions and 9 for highest. Highest
levels produce output files of smaller sizes but require more memory to
do the compression.
This change allows passing a "compression-level" option to the git
archive command and implements using it for different file formats.
Change-Id: I5758f691c37ba630dbac24db67bb7da827bbc8e1
Signed-off-by: Youssef Elghareeb <ghareeb@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* changes:
Compare getting all refs except specific refs with seek and with filter
Add getsRefsByPrefixWithSkips (excluding prefixes) to ReftableDatabase
Add seekPastPrefix method to RefCursor
We sometimes want to get all the refs except specific prefixes,
similarly to getRefsByPrefix that gets all the refs of a specific
prefix.
We now create a new method that gets all refs matching a prefix except a
set of specific prefixes.
One use-case is for Gerrit to be able to get all the refs except
refs/changes; in Gerrit we often have lots of refs/changes, but very
little other refs. Currently, to get all the refs except refs/changes we
need to get all the refs and then filter the refs/changes, which is very
inefficient. With this method, we can simply skip the unneeded prefix so
that we don't have to go over all the elements.
RefDirectory still uses the inefficient implementation, since there
isn't a simple way to use Refcursor to achieve the efficient
implementation (as done in ReftableDatabase).
Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I8c5db581acdeb6698e3d3a2abde8da32f70c854c
This method will be used by the follow-up change. This useful if we want
to go over all the changes after a specific ref.
For example, the new method allows us to create a follow-up that would
go over all the refs until we reach a specific ref (e.g refs/changes/),
and then we use seekPastPrefix(refs/changes/) to read the rest of the refs,
thus basically we return all refs except a specific prefix.
When seeking past a prefix, the previous condition that created the
RefCursor still applies. E.g, if the cursor was created by
seekRefsWithPrefix, we can skip some refs but we will not return refs
that are not starting with this prefix.
Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I2c02e89c877fe90da8619cb8a4a9a0c865f238ef
Heap always copied whole blocks, which leads to AIOOBEs. LocalFile
didn't overwrite the method and thus caused NPEs.
Change-Id: Ia37d4a875df9f25d4825e6bc95fed7f0dff42afb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.
Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If a file exists in head, merge, and the working tree, but not in
the index, and we're doing a force checkout, the checkout must be
an "update", not a "keep".
This is a follow-up on If3a9b9e60064459d187c7db04eb4471a72c6cece.
Bug: 569962
Change-Id: I59a7ac41898ddc1dd90e86b09b621a41fdf45667
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We want complete control over clone behavior and don't want to use
Object's clone implementation.
Change-Id: I34a4e967f8aa3879c69a4978768bd35712c4105c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Path#getFileName can return null. Fix the warning by asserting the file
name isn't null.
Change-Id: I7f2fe75b46113d8be1d14e3f18dd77da27df25ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
File#listFiles can return null. Use Files#list instead to fix the
problem.
Change-Id: I74e0b49aa6dae370219507c64aa43be4d8aa7b82
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Path#getParent can return null. Fix the warning by implementing a helper
method which asserts the parent is not null.
Change-Id: Ib4f8dff0674b74bc891f15f08bd9755c5ea728dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
File#listFiles can return null, assert it is not null to fix the
warning.
Change-Id: I28fc668fee760d39965e6e039003ac9f85fd461b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This was experimental code and never used in production.
Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Some clients may wish to allow NO_CHANGE lightweight tag updates
without setting the force flag. (For instance EGit does so.)
Command-line git does not allow this.
Propagate the RefUpdate result via the RefAlreadyExistsException.
That way a client has the possibility to catch it and check the
failure reason without having to parse the exception message, and
take appropriate action, like ignoring the exception on NO_CHANGE.
Change-Id: I60e7a15a3c309db4106cab87847a19b6d24866f6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit treated a NO_CHANGE RefUpdate as an error in all cases. But when
updating a lightweight tag, this is a successful result if -f was
specified.
Change-Id: Iddfa6d6a6dc8bf8fed81138a008ebc32d5f960bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn