The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.
The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.
The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.
The fix processes a situation when some trees reached the end, while
others are still point to an entry.
bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86
Keys used for identifying reverse indexes in the DfsBlockCache use a
custom subclass ForReverseIndex because there was no PackExt for them.
This conflates BlockCacheMetrics for reverse indexes with those for
packs, since the key falls back onto 0 when there is no extension.
Replace the custom ForReverseIndex with a DfsStreamKey usage to bring
keys for the new REVERSE_INDEX extension in line with INDEX and BITMAP
and separate reverse index and pack BlockCacheMetrics.
Change-Id: I305e2c16d2a8cb2a824855ea92e0c9a9b188fce5
Signed-off-by: Anna Papitto <annapapitto@google.com>
Make sure we always get consistent results, whether or not we have the
full data in the buffer.
Change-Id: Ieb379a0c375ad3dd352e63ac2f23bda6ef16c215
Signed-off-by: Thomas Wolf <twolf@apache.org>
* stable-6.3:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Fix API breakage caused by extracting WorkTreeUpdater
Extract Exception -> HTTP status code mapping for reuse
Don't handle internal git errors as an HTTP error
Ignore IllegalStateException if JVM is already shutting down
Allow to perform PackedBatchRefUpdate without locking loose refs
Change-Id: Ib58879be292c54a2a7f4936ac0986997985c822b
* stable-6.2:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa
* stable-6.1:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286
* stable-6.0:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
* stable-5.13:
[benchmarks] Remove profiler configuration
Add SHA1 benchmark
[benchmarks] Set version of maven-compiler-plugin to 3.8.1
Fix running JMH benchmarks
Add option to allow using JDK's SHA1 implementation
Ignore IllegalStateException if JVM is already shutting down
Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.
When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.
Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e
and update
- com.sun.jna to 5.12.1.v20221103-2317
- com.sun.jna.platform to 5.12.1.v20221103-2317
- org.bouncycastle.bcpg to 1.72.0.v20221013-1810
- org.bouncycastle.bcpkix to 1.72.0.v20221013-1810
- org.bouncycastle.bcprov to 1.72.0.v20221013-1810
- org.bouncycastle.bcutil to 1.72.0.v20221013-1810
- org.mockito.mockito-core to 4.8.1.v20221103-2317
- org.objenesis to 3.3.0.v20221103-2317
Change-Id: If00094d23e51d5f66928f83c1334aa6b18b98dfe
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.
The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.
The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.
The fix processes a situation when some trees reached the end, while
others are still point to an entry.
bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86
Before this change JGit did not support the session-id capability
implemented by native Git in UploadPack. This change implements
advertising the capability from the server and parsing the session-id
received from the client during an UploadPack operation.
Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the UploadPack object.
This change does not add the capability to send a session-id from the
JGit client.
https://git-scm.com/docs/gitprotocol-capabilities#_session_idsession_id
Change-Id: Ib1b6929ff1b3a4528e767925b5e5c44b5d18182f
Signed-off-by: Josh Brown <sjoshbrown@google.com>
The config setting to enable advertising the session-id capability is
currently read in the ReceivePack class. This change moves it to a
common location in the TransferConfig class so that it can be reused
in other places like UploadPack. TransferConfig is also a more logical
place for the setting as it resides in the `transfer` config section.
Set the transfer.advertisesid setting to true to send the session-id
capability to the client.
Change-Id: If68ecb5e68b59f5c452a7992d02e3688b0a86747
Signed-off-by: Josh Brown <sjoshbrown@google.com>
In protocol V0 the client capabilities are appended to the first line.
Parsing session-id is currently only supported during a ReceivePack
operation. This change will parse the client session-id capability if
it has been sent by the client.
If the server sends the session-id capability to the client. The client
may respond with a session ID of its own. FirstWant.fromLine will now
parse the ID and make it available via the getClientSID method.
This change does not add support to send the session-id capability from
the server. The change is necessary to support session-id in UploadPack.
Change-Id: Id3fe44fdf9a72984ee3de9cf40cc4e71d434df4a
Signed-off-by: Josh Brown <sjoshbrown@google.com>
Before this change JGit did not support the session-id capability
implemented by native Git. This change implements advertising the
capability from the server and parsing the session-id received from
the client during a ReceivePack operation.
Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the ReceivePack object. All capabilities
in the form `capability=value` are now split into key value pairs at the
first `=` character. This change replaces specific handling for the
agent capability.
This change does not add advertisement or parsing to UploadPack. This
change also does not add the ability to send a session ID from the JGit
client.
https://git-scm.com/docs/protocol-v2/2.33.0#_session_idsession_id
Change-Id: I56fb115e843b11b27e128c4ac427b05d5ec129d0
Signed-off-by: Josh Brown <sjoshbrown@google.com>
Reachability bitmaps are designed to speed up the "counting objects"
phase of generating a pack during a clone or fetch. They are not
optimized for Git clients sending a small topic branch via "git push".
In some cases (see [1]), using reachability bitmaps during "git push"
can cause significant performance regressions.
Add PushCommand#setUseBitmaps(boolean) to allow users to tell "git push"
not to use bitmaps.
[1]: https://lore.kernel.org/git/87zhoz8b9o.fsf@evledraar.gmail.com/
Change-Id: I7fb7d26084ec63ddfa7249cf58abb85929b30e56
Signed-off-by: kylezhao <kylezhao@tencent.com>
Fix and complete the implementation of calling the pre-push hook.
Add the missing error stream redirect, and add the missing setters
in Transport and in PushCommand. In Transport, delay setting up a
PrePushHook such that it happens only on a push. Previously, the
hook was set up also for fetches.
Bug: 549246
Change-Id: I64a576dfc6b139426f05d9ea6654027ab805734e
Signed-off-by: Thomas Wolf <twolf@apache.org>
We need the full size of the objects to populate the object-size index
of a pack. This size is not always the one encoded in the object header
in the pack (e.g. for deltas).
Populate the full size of PackedObjectInfos in the PackParser, which is
invoked when receiving a pack e.g. in a push.
Change-Id: I102c20901aefb5e85047e2e526c0d733f82ff74b
CloneCommand, when setNoCheckout(true) was set, did not set HEAD.
With C git, "git clone --no-checkout" does.
Change-Id: Ief3df7e904ce90829a6345a6c3e9ee6a68486ab0
Signed-off-by: Thomas Wolf <twolf@apache.org>
PatchApplier now routes updates through the index. This has two
results:
* we can now execute patches in-memory.
* the JGit apply command will now always update the
index to match the working tree.
Change-Id: Id60a88232f05d0367787d038d2518c670cdb543f
Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>
Co-authored-by: Nitzan Gur-Furman <nitzan@google.com>
* stable-6.3:
Prepare 6.3.1-SNAPSHOT builds
JGit v6.3.0.202209071007-r
JGit v6.3.0.2022009070944-r
[merge] Fix merge conflicts with symlinks
Update DEPENDENCIES for 6.3.0
Update tycho to 2.7.5
Revert "Adds FilteredRevCommit that can overwrites its parents in the DAG."
Revert "Option to pass start RevCommit to be blamed on to the BlameGenerator."
Prepare 6.3.0-SNAPSHOT builds
JGit v6.3.0.202208161710-m3
Change-Id: Ia9430fb516dca795e25064a190704b70689af364
Add a test case to make sure that the bundle writer writes objects in
GC_REST packs as well.
Signed-off-by: Yunjie Li <yunjieli@google.com>
Change-Id: Iba4d88c573aa1cda4505afbe2b83581a09a343df
Previous code would do a content merge on symlinks, and write the merge
result to the working tree as a file. C git doesn't do this; it leaves
a symlink in the working tree unchanged, or in a delete-modify conflict
it would check out "theirs".
Moreover, previous code would write the merge result to the link target,
not to the link. This would overwrite an existing link target, or fail
if the link pointed to a directory.
In link/file conflicts or file/link conflicts, C git always puts the
file into the working tree.
Change conflict handling accordingly. Add tests for all the conflict
cases.
Bug: 580347
Change-Id: I3cffcb4bcf8e336a85186031fff23f0c4b6ee19d
Signed-off-by: Thomas Wolf <twolf@apache.org>
DAG."
This reverts commit 6297491e8a.
This is done as a quick fix for the failure of egit tests caused by the
introduction of FilteredRevCommit.
Bug: 580690
Change-Id: Ia6b651dd11b0a4b02d5e52247eb4bf13adf94e27
BlameGenerator."
This reverts commit 5747bba48b.
This is done as a quick fix for the failure of egit tests caused by the
introduction of FilteredRevCommit.
Bug: 580690
Change-Id: Ia0178bc2de4fc825a81207bbd7979bf3a386c955
An invalid path in the manifest (e.g. '.') is reported by DirCache in a
runtime exception. In server context this becomes a 500 instead of a user error.
Wrap the runtime invalid path exception into a checked ManifestErrorException that
caller can handle.
Change-Id: I61a2104922765506ae232334891057bb06141d97
On executing a copy, mark the destination as updated.
On executing a rename, mark both source and destination as updated.
Change-Id: Ied5b9b0e5a14eac59a06cdd0961e25e143f50ff0
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Change-Id: Ie127cb710d004079e9f53a5802130afdb49a7de1
Previous code loaded the WorkingTreeOptions afresh for every single
file being checked out. This checked the git config (all three files,
repo, user and system config) for having been modified every time.
These checks can be costly, for instance on Windows, or if one of the
three config files is not on a local disk, or on an otherwise slow
storage.
Improve this by loading the options and thus checking the git config
only once before the checkout.
Bug: 579715
Change-Id: I21cd5a808f9d90b5ca2d022f91f0eeb8ca26091c
Signed-off-by: Thomas Wolf <twolf@apache.org>
With core.symlinks=false, symlinks are checked out as plain files.
When such a file is re-added to the index, and the index already
contains a symlink there, add the file as a symlink. Previous code
changed the index entry to a regular file.
Bug: 580412
Change-Id: I5497bedc3da89c8b10120b8077c56bc5b67cb791
Signed-off-by: Thomas Wolf <twolf@apache.org>
String.startsWith() is not a valid test for file path prefixes:
directory "a" is _not_ a prefix of a file "ab", only of "a/b".
Add a proper Paths.isEqualOrPrefix() method and use it in CleanCommand.
Bug: 580478
Change-Id: I6863e6ba94a8ffba6561835cc57044a0945d2770
Signed-off-by: Thomas Wolf <twolf@apache.org>
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.
Change-Id: I5dab25301d6aef7df6a0bc25a4c553c730199272
This makes RevCommit extensible to allow having different structure of
child-parent relationship. This change is a pre-requsite for having a
FilteredRevCommit that overrides parents from the RevCommit. That then
provides a cheaper way to walk over a subset of RevCommits instead of
an expensive way that applies filters while walking over selected
commits. Useful with Blame which works on a single file and that can be
made performant, if we know all the commits needed by the Blame
algorithm. So Blame algorithm can avoid walking over finding what
commits to blame on.
This change makes parents field on RevCommit private and exposes it
thrrough overrideable methods such as getParents, getParent at index,
getParentCount and setParents. All other files other than RevCommit are
updating the usages of accessing RevCommits parents.
Change-Id: I2d13b001c599cc4ebc92d1ab6e07b07acb3b7fe5
The FilteredRenameDetectorTest was added in [1], but bazel build rules
were not updated.
[1] https://git.eclipse.org/r/c/jgit/jgit/+/194200
Change-Id: I7fc713e19b4768176eb84e9768137431b33a805e
This adds support for shallow cloning. The CloneCommand and the
FetchCommand now have the new methods setDepth, setShallowSince and
addShallowExclude to tell the server that the client doesn't want to
download the complete history.
Bug: 475615
Change-Id: Ic80fb6efb5474543ae59be590ebe385bec21cc0d
The NameConflictTreeWalk class is used in 3-way merge for iterating over
entries in 3 different commits. The class provides information about a
current entry and a state of the entry in commits (e.g entry is file,
entry is directory, entry is missing). In rare cases, the tree walker
can mix information about entries with different name.
The problem appears, because git uses unusual sorting order for
files. Example (this is a simplified real-life example):
Commit 1:
* gradle.properties - file
* gradle - directory (with nested files)
* gradle/file - file in gradle directory
Commit 2:
* gradle.properties - file
* no entry with the name gradle
Commit 3:
* gradle.properties - file
* gradle - file
Here the names are ordered like this:
"gradle" file <"gradle.properties" file < "gradle/file" file.
NameConflictTreeWalk iterator already have code for processing
git sorting order, however in the example above the code doesn't
work correctly. Before the fix, NameConflictTreeWalk returns:
#next()
"gradle - directory" | "gradle.properties" | "gradle - file" - which is
wrong. The expected result is
#next()
"gradle - directory | MISSED_FILE | "gradle - file"
#next()
"gradle.properties"|"gradle.properties"|"gradle.properties"
Ensure that the "matches" field of tree iterators (which contains the
current path) is kept in sync in the case above.
Change-Id: Ief5aa06d80b358f4080043c8694aa0fd7c60045b
Signed-off-by: Dmitrii Filippov <dmfilippov@google.com>
The RootIgnoreNode in a WorkingTreeIterator must _not_ add the rules
from .git/info/exclude or from the file designated by git config
core.excludesFile to the list of rules read from the root .gitignore.
These really must be separate nodes in a hierarchy, otherwise the
precedence rules from [1] are violated and the outcome is not the
same as in C git.
[1] https://git-scm.com/docs/gitignore
Bug: 580381
Change-Id: I57802ba7bbbe4f183504c882b6c77a78cc3a9b99
Signed-off-by: Thomas Wolf <twolf@apache.org>
* stable-6.2:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Change-Id: Ia2280b848cea7bdfa5bc48356484e040d159845d
* stable-6.1:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Change-Id: Ib3ff6a038e1b92a931a7f2624c19bbefd8cb95fd
* stable-6.0:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Change-Id: I08734ee2c8f3296d908da6a29d53ed87c4b48eb2
* stable-5.13:
UploadPack: don't prematurely terminate timer in case of error
Do not create reflog for remote tracking branches during clone
UploadPack: do not check reachability of visible SHA1s
Add missing package import javax.management to org.eclipse.jgit
Change-Id: I6db0a4d74399fde892eeec62efd2946f97547a5d
Adjusted BlameGenerator to filter rename detection with the blame path.
This reduces the running time of the blame computation significantly,
for repositories with massive commits involving renames.
The filtered rename detection is made (internally) available with:
org.eclipse.jgit.internal.diff.FilteredRenameDetector
Bug: 578900
Change-Id: I6580004e81102d685081b8180da1587a35073d36
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
When using JGit on a non-bare repository, the CloneCommand
it previously created local reflogs for all branches including remote
tracking ones, causing the generation of a potentially large
number of files on the local filesystem.
The creation of the remote-tracking branches (refs/remotes/*) during
clone is not an issue for the local filesystem because all of them are
stored in a single packed-refs file. However, the creation of a large
number of ref logs on a local filesystem IS an issue because it
may not be tuned or initialised in term of inodes to contain a very
large number of files.
When a user (or a CI system) performs the CloneCommand against
a potentially large repository (e.g., millions of branches), it is
interested in working or validating a single branch or tag and is
unlikely to work with all the remote-tracking branches.
The eager creation of a reflogs for all the remote-tracking branches is
not just a performance issue but may also compromise the ability to
use JGit for cloning a large repository.
The behaviour implemented in this change is also consistent with the
optimisation done in the C code-base [1].
We differentiate between clone and fetch commands using --branch
<initialBranch> option, that is only available in clone command,
and is set as HEAD per default.
[1] 58f233ce1e
Bug: 579805
Change-Id: I58d0d36a8a4ce42e0f59b8bf063747c4b81bd859
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
* stable-6.2:
Prepare 5.13.2-SNAPSHOT builds
JGit v5.13.1.202206130422-r
AmazonS3: Add support for AWS API signature version 4
Change-Id: I13514595f3a710b0d6e3c2de62f6508b950d22f9
* stable-6.1:
Prepare 5.13.2-SNAPSHOT builds
JGit v5.13.1.202206130422-r
AmazonS3: Add support for AWS API signature version 4
Change-Id: Id4965aacd4e2ea1e8575a2c1bd4845729db6049a
* stable-6.0:
Prepare 5.13.2-SNAPSHOT builds
JGit v5.13.1.202206130422-r
AmazonS3: Add support for AWS API signature version 4
Change-Id: Ie9c38ab8033fe1283e8b444b6acd3f4298062bf3
* stable-5.13:
Prepare 5.13.2-SNAPSHOT builds
JGit v5.13.1.202206130422-r
AmazonS3: Add support for AWS API signature version 4
Change-Id: Ibd663a1d874d1aac274abc3dd44354fd99f64c39
Updating the AmazonS3 class to support AWS Signature version 4 because
version 2 is no longer supported in all AWS regions. The version can be
selected with the new 'aws.api.signature.version' property (defaults to
2 for backwards compatibility). When set to '4', the user must also
specify the AWS region via the 'region' property. The 'region' property
must match the region that the 'domain' property resolves to.
Bug: 579907
Change-Id: If289dbc6d0f57323cfeaac2624c4eb5028f78d13
* master:
Adapt diff- and merge tool code for PGM and EGit usage
Teach JGit to handle external diff/merge tools defined in
.gitattributes
Change-Id: I3aefc14160caaac859bd3548460dd755ebe42fc5
Adds API that allows UI to find (and handle) diff/merge tools, specific
for the given path. The assumption is that user can specify file type
specific diff/merge tools via gitattributes.
Bug: 552840
Change-Id: I1daa091e9afa542a9ebb5417853dff0452ed52dd
Signed-off-by: Mykola Zakharchuk <zakharchuk.vn@gmail.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
see: https://git-scm.com/docs/git-mergetool
* DiffTools and MergeTools
* store FS, gitDir and workTree for usage without
git repository (for EGit preferences)
* add getUserDefinedToolNames() and getPredefinedToolNames()
* replace getToolNames() with getAllToolNames() that combines the two
lists and put default tool name (diff.tool or merge.tool) as first
element (for EGit preferences)
* FileElement: refactoring of getFile() and friends to have midName
(LOCAL, REMOTE etc.) always added to the temp file name (also for EGit)
* FileElement: added directory attribute that is used in getFile() to
return path with workDir as parent
* DiffTool and MergeTool
* added errw.flush(), because sometimes stderr is not printed in case
of die()
* print e.getMessage() always to stderr
* Moved toolname and prompt logic into managers
* Exported internal packages required for egit.ui
Bug: 356832
Change-Id: I71e7f4dc362169a7612ca4f6546a021bc4b2b5f4
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
see: https://git-scm.com/docs/git-difftool
see: https://git-scm.com/docs/git-mergetool
* now all available tools are printed with "--tool-help"
* if no diff.tool or merge.tool is defined the first available
pre-defined tool is used
TODO:
- add mergetools to difftools --> extra change or merge to this
- return the exit-code of the tool to jgit / java runtime
Bug: 356832
Change-Id: I20fb04e71ced981f5625020f461bbac24e6cec70
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
see: https://git-scm.com/docs/git-mergetool
* refactoring of content (FileElement) handling
* now the temporary files are already filled with filtered content in
the calling classes (PGM), that can be used with EGit content too
TODO:
* keep the temporaries when no change detected and the user answers no
to the question if the merge was successful
Bug: 356832
Change-Id: I86a0a052d059957d4d152c1bb94c262902c377d2
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
NOTE: port back from master branch.
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Bug: 579445
Change-Id: Iaee2301eb14e6201406398a90228ad10cfea6098
see: http://git-scm.com/docs/git-difftool
* add CommandExecutor that handles tool execution with help of "jgit.FS"
* it handles tool execution with temporary created "command file" -->
for for all "command interpreters" and parameters with spaces etc.
* using of external bash.exe at Windows (MinGW) if shell-script is
used as difftool command. It can be enabled with parameter
"jgit.usemsys2bash=auto" that checks if command contains ".sh" or
enabled / disabled with "jgit.usemsys2bash=true|false"
* added special handling for empty files (e.g. deleted, added etc.) that
are named "/dev/null"
* added creation and deletion of temporary files needed for compare
* added own Exception class for reporting to pgm / command line / EGit
* added prompt option handling before executing difftool
* reworked trustExitCode option for specific difftool and override for
all difftools from config and command line
* tested with command line options "--[no]-trust-exit-code",
"--tool=<toolname>", "--[no]-gui", --[no]-prompt
* ContentSource
* added close() methods to close / cleanup used resources
(like ObjectReader TreeWalk etc.)
* added isWorkingTreeSource() methods to check if file can be used
from working tree instead of copy from "ObjectLoader / ObjectReader" to
temporary file (fixes "difftool <commit> <commit>")
Bug: 356832
Change-Id: I5462fb6dbe4ecfd9da7c74117fce4070bbfd4d7a
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Bug: 579445
Change-Id: Ie27aca1fdaddfa487ebe072c3913d78355910df8
Implement negative refspecs in JGit fetch, following C Git. Git
supports negative refspecs in source only while this change supports
them in both source and destination.
If one branch is equal to any branch or matches any pattern in the
negative refspecs collection, the branch will not be fetched even if
it's in the toFetch collection.
With this feature, users can express more complex patterns during fetch.
Change-Id: Iaa1cd4de5c08c273e198b72e12e3dadae7be709f
Sign-off-by: Yunjie Li<yunjieli@google.com>
This concerns committing, creating merge conflict messages and creating
and editing squash messages. In a squash message, once the comment
character has been determined initially is always the first character.
Note that if core.commentChar=auto and there is a sequence of squashes,
it may be necessary to change the comment character when a new message
is added.
Bug: 579325
Change-Id: Idca19284a0240cd322e7512ea299a03658e1b2c1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Provide access to the core.commentChar git config, and provide a
utility method to determine an unused comment character if the setting
is "auto".
Bug: 579325
Change-Id: I1ec7e4deffea6ac5929a8538a624d73bb59e4ecc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the last sideband progress message didn't end in \r or \n, there
may still be a buffered message at the end of a fetch or push. Ensure
that message gets written, too, even if it may be only partial.
Bug: 575629
Change-Id: I38edccb5cffb89e00e468480b43c7d951fb63e8e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If LsRemoteCommand is used without repository, the remote URI was not
translated. If the git user config contains e.g.
[url "ssh://git@github.com/"]
insteadOf = https://github.com/
and LsRemoteCommand is called with an HTTPS GitHub repository URL, the
command should actually rewrite this to an SSH URI and use the SSH
transport.
Actually this same problem may exist everywhere Transport is used with
an URIish instead of with a remote name. However, doing this translation
in Transport.open(URIish) and in Transport.open(Repository, URIish,
String) if no remote name is given would change the behavior and might
break assumptions made in existing clients. For instance, EGit's
PushOperation assumes that the URI obtained from PushResult.getURI()
was the same as was passed in to Transport.open(Repository, URIish).
URIs obtained from a RemoteConfig have this translation applied
transparently, and in Transport we cannot know for sure whether or
not a URI has already been translated, if needed. So doing this in
Transport might also lead to translating URIs twice.
Therefore this commit does the translation in LsRemoteCommand, where
we can be sure that it won't affect other use cases. If other cases
besides LsRemoteCommand are found where such a URI translation is
missing, it'll have to be done at higher levels, possibly even in
client code directly.
Bug: 544769
Change-Id: I5df54a925d30b55d98e21f37f2851fe79649b064
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-6.1:
Prepare 6.1.1-SNAPSHOT builds
JGit v6.1.0.202203080745-r
[checkout] Use .gitattributes from the commit to be checked out
Don't use final for method parameters
[push] support the "matching" RefSpecs ":" and "+:"
[push] Call the pre-push hook later in the push process
IndexDiff: use tree filter also for SubmoduleWalk
Run license check with option -Ddash.projectId=technology.jgit
Exclude transitive dependencies of sshd-sftp
Update DEPENDENCIES for 6.1.0 release
Add dependency to dash-licenses
Fix typos of some keys in LfsText
Sort LfsText entries alphabetically
Support for "lfs.url" from ".lfsconfig"
Change-Id: I1b9f0c0ed647837e00b9640d235dbfab2329c5a6
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.
These attributes must be used to determine whether the working tree is
dirty.
But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.
Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.
Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.
The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)
Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.
Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.
Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?
Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.
This RefSpec can be expanded only once the advertisement from the
remote has been received.
Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.
Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.
Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.
Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If core.abbrev is unset or "auto" estimate abbreviation length like C
git does:
- Estimate repository's object count by only considering packed objects,
round up to next power of 2
- With the order of 2^len objects, we expect a collision at 2^(len/2).
But we also care about hex chars, not bits, and there are 4 bits per
hex. So all together we need to divide by 2; but we also want to round
odd numbers up, hence adding one before dividing.
- For small repos use at least 7 hexdigits
- If object database fails to determine object count use 7 hexdigits as
fallback
If it is set to "no" do not abbreviate object-ids.
Otherwise set it to the configured value capped to the range between 4
and length of an unabbreviated object-id.
Change-Id: I425f9724b69813dbb57872466bf2d2e1d6dc72c6
Error Prone is flagging Date-API as obsolete and recommends to migrate
to Instant and LocalDate. Given that more JGit users starting to migrate
to new Time API, offer ctors that accept Instant type and also add new
getter that returns when attribute as Instant type.
Change-Id: I64a36bf40f191495c6889c1dff314ede06848880
potentiallyUnclosedCloseable currently is not checked and it creates
a warning when compiling with Maven.
Change-Id: I0715ead433270937f77f56a19aa203f8d6f5322f
The following warning was raised by Eclipse:
"At least one of the problems in category
'unused' is not analysed due to a compiler option being ignored"
The org.eclipse.jdt.core.compiler.problem.unusedTypeParameter compiler
option is set to ignore, hence the warning suppression is redundant.
Change-Id: I5fd6efc9968be133c71e95c53803c44310d75e85
Add ConfigConstants and expose branch.<name>.pushRemote in the
BranchConfig. Use the branch configuration and remote.pushDefault
if no remote is given explicitly. If nothing is configured, fall
back to "origin".
Bug: 578676
Change-Id: I6bb141ff02c8b04980ec34b26ef248b72614c3c9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When no RefSpecs are given, PushCommand until now simply fell back to
pushing the current branch to an upstream branch of the same name. This
corresponds to push.default=current. Any setting from the git config
for push.default was simply ignored.
Implement the other modes (nothing, matching, upstream, and simple),
too. Add a setter and getter for the PushDefault so that an application
can force a particular mode to be used. For backwards compatibility,
use "current" as the default setting; to figure out the value from the
git config, which defaults to "simple", call setPushDefault(null).
Bug: 351314
Change-Id: I86c5402318771e47d80b137e99947762e1150bb4
Signed-off-by: Rolf Theunissen <rolf.theunissen@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Enhance the (unused!?) PushConfig; include a PushDefault enumeration.
Add simple tests for this PushConfig.
Bug: 351314
Change-Id: Ibc5656a2a1fccf70d00c5e15de8ed3dd8add6337
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
BasePackConnection::readAdvertisedRefsImpl was creating an exception by
calling `noRepository`, and then blindly calling `initCause` on it. As
`noRepository` can be overridden, it's not guaranteed to be missing a
cause.
BasePackPushConnection overrides `noRepository` and initiates a fetch,
which may throw a `NoRemoteRepositoryException` with a cause.
In this case calling `initCause` threw an `IllegalStateException`.
In order to throw the correct exception, we now return the
BasePackPushConnection exception and suppress the one thrown by
BasePackConnection
Bug: 578511
Change-Id: Ic1018b214be1e83d895979ee6c7cbce3f6765f6f
Set the string encoding when converting a byte array into String,
avoiding the build-time warning on platform-dependent encoding.
See https://errorprone.info/bugpattern/DefaultCharset
Change-Id: I1f920043a8f303da43a8278793c38453e8773d69
Repository wasn't closed in FetchAndPullCommandsRecurseSubmodulesTest.
This caused a "Resource leak: 'g' is never closed" warning during
compilation.
Bug: 578546
Change-Id: I591ef1e286bcd9a7103c016fd47015e77fa95cbb
CancellableDigestOutputStream resources are never closed in the tests.
This causes a "Resource leak: 'out' is never closed" warning
at compile time.
Suppress it by using a try with resources.
Bug: 578544
Change-Id: I0dc7de9162b8e3ac6fcaabe3002423f545baddb8
JGit accumulated in MESSAGE_FIXUP commit messages of a fixup sequence,
just like it did in MESSAGE_SQUASH, and on the last step of a sequence
of fixups used that file, after stripping all comment lines, as the
commit message. That also stripped any lines from the original commit
message that happened to start with the comment character.
This is not how this is supposed to work. MESSAGE_FIXUP must contain
the original commit message of the base commit that is amended, and
the file contains the verbatim commit message for the final fixup.[1]
Change the implementation accordingly, and add new tests.
[1] https://github.com/git/git/blob/df3c41adeb/sequencer.c#L86 ff.
Bug: 513726
Change-Id: I885a2b7f10d6c74460a8693aa6cbf867ee0494a1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reload the stack _before_ trying to delete the files. This ensures we
don't trip over our own open file handles when deleting compacted
tables.
If there is another process reading the file, it may be impossible to
delete the compacted tables. In this case, ignore the failure.
For cleaning the garbage in this case, the protocol as described in
https://www.git-scm.com/docs/reftable#_windows should be implemented.
This is left for another commit.
Bug: 578454
Change-Id: I7aa43508450041eb9376d9f67a0262ff7cc53c73
* changes:
RepoCommand: Offer to set extra files in the destination repository
RepoCommand: Move bare/regular superproject writing to their own classes
On Windows eol=native, which is also the default if nothing else
overrides it, means convert text files to CRLF on checkout.
Adapt tests for this. Also don't fiddle with the real system property
"line.separator", use the mocked SystemReader for this.
Bug: 550111
Change-Id: Ie19b80fc543fa4970d6d9f181041c5f4d6ef3ed4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Escape paths when writing them to a config file to ensure they work
with backslashes and unusual characters.
Bug: 550111
Change-Id: Iedc5c0f2c0c02ac6cadf43cdae0f0d19578aed91
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Ensure that this test works on Windows.
Bug: 550111
Change-Id: Ib3ca803f4d9521a5f2a0e1344bfbc0d2beccc9cc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
On Windows, java.io.File.canExecute is always true. Guard assertions
testing the executable bit with FS.DETECTED.supportsExecute().
Bug: 550111
Change-Id: I2704d122f5b1086d01a0503a8c047a02ecbc1d4a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We want to save in the destination repository what manifest created its
structure. This helps to detect and debug failures in the manifest ->
superproject translations. The src commit should be easily readable from
the superproject tip.
Offer an API to write a file in the destination repository. RepoCommand
callers (e.g. gerrit supermanifest plugin) can use this to add a
file with the repo/ref/hash of the manifest.
Alternatives considered to write the source repo/ref/hash:
* .gitattributes of the .gitmodules file. Some updates in the manifest
don't touch the .gitmodules (e.g. a linkfile change), so it can fall
out of sync.
* commit message. Caller would need to follow the commit history to
find the latest modification by repo command. This is not helpful
e.g. for build bots that want to get the value in one call.
Change-Id: I113662734a7ccd39cbc60b46ad3f73038c807682
RepoCommand parses the manifest to get a list of projects, clears up
conflicts and then writes to the superproject. The first steps are
common but the writing is completely different for bare or "regular"
(with working dir) repository.
Split writing to bare and regular repos into its own classes. This
simplifies RepoCommand class and makes clearer what happens on each side
(e.g. many options apply only to bare repos).
Change-Id: I256e15729bd53ee15fc56de88bce86a2edb2417a
C git also does so. Note that currently the comment character is
hard-coded as the hash '#' throughout JGit.
Bug: 548529
Change-Id: I4a5597694082a9e5b07412b365cfaf41fa034cfa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Use CommitConfig.CleanupMode to implement git commit --cleanup. Add
setters for the clean-up mode, the comment character, and for the
default default clean-up mode.
Behavior of existing client code is unchanged as the default clean-up
mode is set to "verbatim". To use git config defaults, one can call
setCleanupMode(CleanupMode.DEFAULT). The default comment character
is hard-coded as '#' for now, as in other parts of JGit. Implementing
full support for core.commentChar shall be done in a separate change.
Bug: 553065
Change-Id: I470785e464a762d3f409f163f1cbdbb98dd81aaf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add an enumeration for the possible values, and a method to resolve the
"default" value. Give CommitConfig a static method to process a text
according to a given clean-up mode and comment character.
(The core.commentChar is not yet handled by JGit; it's hard-coded as #.)
Bug: 553065
Change-Id: If6e384522275f73b713fbc29ffcaa1753c239dea
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fix tests failing on Windows because Repository instance is created but
not closed on tear down.
Fix repositories closed twice, except in tests that test this behavior
explicitly.
Name the temporary directories the tests run in after the test method;
that makes it easier to figure out in which tests repositories are
closed twice if it should occur again in the future.
Bug: 550111
Change-Id: I9398b58f0f36d2c29236d2a9a8599117d9083980
Signed-off-by: Nail Samatov <sanail@yandex.ru>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
With eol=native, we expect LF on Unixes, and CR-LF on Windows. One test
didn't account for this and always expected LF, and thus failed on
Windows.
Bug: 550111
Change-Id: I69354ac691c464d1b6003812ddb4510c5ab5e77a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Test.txt should not be in the "removed" list if it can't be deleted
but only in the "not deleted" list. The test was wrong.
Bug: 550111
Change-Id: I3ecede4278014c15015c8c24089647fa3db3742f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Some tests checked whether or not a file is executable via
java.io.File.canExecute(). But that always returns true on Windows.
Use FS.DETECTED.canExecute() instead, and guard all such assertions to
run only of file systems that do support the "execute" flag.
Bug: 550111
Change-Id: Iacb9e414b612359fcecb61312c3dfb830801cd36
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The tests assumed that a path like "/tmp" was an absolute path, and
also compared against strings with forward slashes. On Windows, "/tmp"
is not an absolute path and thus resolved against the current directory,
and the separator is a backslash.
Change the tests to use ~/ notation, and test paths resolved against
the (mocked) user home directory. That way, the tests are independent
of the file system used.
Bug: 550111
Change-Id: I1c31608ca83c8d8586256d1586a792e4a33cfaa4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add parsing of the config. Implement the SSH agent protocol for adding
a key. In the pubkey authentication, add keys to the agent as soon as
they've been loaded successfully, before even attempting to use them
for authentication. OpenSSH does the same.
Bug: 577052
Change-Id: Id1c08d9676a74652256b22281c2f8fa0b6508fa6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
OpenSSH has (for legacy reasons?) the option of specifying the default
environment variable directly, instead of using ${SSH_AUTH_SOCK}. Make
sure the plain variable name is not taken as a relative path name.
Bug: 577053
Change-Id: If8f550dffc43887254f71aa0b487c50fa14d0627
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.
Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.
Pull up that functionality to a superclass, so it can be reused.
Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.
Add a custom consumer to report loading events and eviction events when
enabled.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.
FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.
Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).
To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.
Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
* stable-6.0:
Use slf4j-simple instead of log4j for logging
Update orbit to R20211213173813
Change-Id: I746b7fb71571020ce49f7b50fd675c9864327719
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Use slf4j-simple instead of log4j for logging
Update orbit to R20211213173813
Change-Id: I219ef3901c1d908b91bf9c8f00431b22686ff7a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit uses slf4j-api as logging API.
The libraries
- org.eclipse.jgit.http.test
- org.eclipse.jgit.pgm
- org.eclipse.jgit.ssh.apache.test
- org.eclipse.jgit.test
used the outdated log4j 1.2.15 which is EOL since years.
Since both jgit command line and also the tests don't need sophisticated
logging features replace log4j with the much simpler slf4j-simple log
implementation. The org.slf4j.binding.simple 1.7.30 archive has only
25kB instead of 429kB for log4j 1.2.15
Applications using jgit are free to choose any other log implementation
supporting slf4j API.
Change-Id: I89e85cd3c76e954c3434622510975ce65dc227d4
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.
The check was already performed when a new FileBasedConfig
is created at L158:
// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());
The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.
This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.
Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* stable-6.0:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Id7540d03991cdcf6f405e946b8cbbcc6a9696a31
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Ibd69e9d941ad9262b61dd0c4368e48cb82597a12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".
Follow the example below:
. .
└── src └── src
└── d1 ==> └── d1
└─ file1 ├─ file1
└── file2
c.parents[i] c
The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".
We should add an additional check for no paths matching the filter.
Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
Speed up bitmap creation by loading reverse index in parallel
to reading bitmap from storage. Latency changes from
(time_to_read_bitmap + time_to_load_reverse_index) to
max(time_to_read_bitmap, time_to_load_reverse_index).
Add new option to DfsReaderOptions to control parallel reverse index
loading. Static cached thread pool is added to PackBitmapIndexV1 for
reverse index loading, and when not in use consumes minimal resources.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ia37a1d739631d053e8bddb925ac8b0b81d22379e
Let GC#gc return collection of newly created packs as CompletableFuture
to enable using gc() asynchronously.
Change-Id: I3627014fd458c738cfe54225e631d6f7d9cfb1a7
* stable-6.0:
FS: debug logging only if system config file cannot be found
FS: debug logging only if system config file cannot be found
Update .factorypath used by annotation processor for benchmarks
Use maven-compiler-plugin's release tag instead of source and target
Don't use deprecated Repository#getAllRefs in Repository
Don't use deprecated Repository#getAllRefs in FileRepository
RevListTest: fix warning that method parameter hides field 'git'
Implement RecordingLogger based on org.slf4j.Logger
Let ObjectDatabase implement AutoClosable
Change-Id: Ie6b3cfa66b319033d4448dcf20362b753c0e9d7c
see: http://git-scm.com/docs/git-difftool
* add command line support for "jgit difftool"
* show supported commands with "jgit difftool --help"
* added "git difftool --tool-help" to show the tools (empty now)
* prepare for all other commands
Bug: 356832
Change-Id: Ice0c13ef7953a20feaf25e7746d62b94ff4e89e5
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- AdvertiseRefsHook#advertiseRefs
- ReceivePack#setAdvertisedRefs
- Repository#getAdditionalHaves
Bug: 534731
Change-Id: I85bc6ce5815d40be5f80042c53f4663072d96be5
Provide a public interface PackLock exposing only the unlock() method.
Rename the internal PackLock class to PackLockImpl and have it implement
the new interface.
This way PackParser doesn't expose an internal class via its API
anymore, and client can still unlock pack locks that were created.
Bug: 576340
Change-Id: I976739f4ab28fe1f9ba7f35653a69a913aa68841
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fixes an issue that commit template file could not be found if it has a
relative path instead of absolute path.
Relative path is probably common if git config --local is used.
Bug: 446355
Change-Id: I8ddf2be672647be825fd9c01af82809d31bb8356
With change https://git.eclipse.org/r/c/jgit/jgit/+/186455 a thread
loading a bitmap index could hold two ref locks at the same time (one
for bitmap and one for either index or reverse index). So it is possible
that two threads loading bitmaps end up in a deadlock for ref locks e.g.
threadA has refLock[1] (for bitmap) and wants refLock[2] (for index or
revIndex) and threadB has refLock[2] (for bitmap) and wants refLock[1].
This change introduces separate pools of locks per pack extension
instead of a shared pool. So threads loading bitmap can hold two
locks but with different extensions and no overlap, e.g. threadA holds
refLock[BITMAP_INDEX][1] and refLock[INDEX][2] and threadB holds
refLock[BITMAP_INDEX][2] and refLock[INDEX][1].
More unit tests were added to cover various paralell loading scenarios.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I89704b4721c21548929608d3798ef60925280755
Bring our SSH config parser up-to-date with respect to changes in
OpenSSH. In particular, they fixed[1] the handling of line comments
such that #-characters inside strings are not considered. This means
that we have to parse strings with escaped quotes correctly.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=3288
Change-Id: Ifbd9014127e8d51e7c8792e237f3fc2a9a0719d2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is a prerequisite for removing the JSch support bundle; otherwise
OpenSshConfigFile would be left without tests.
Copy OpenSshConfigTest from the JSch support bundle and adapt all tests
to perform the equivalent checks on OpenSshConfigFile directly. Add a
new lookupDefault() method to the SshConfigStore interface and implement
it so that it behaves the same and the tests work identically.
Change-Id: I046abd9197a8484003e77005024e5d973456f1a3
C git considers not only files containing NUL bytes as binary but also
files containing lone CRs. Implement this also for JGit.
C git additionally counts printable vs. non-printable characters and
considers files that have non_printable_count > printable_count / 128
also as binary. This is not implemented because such counting probably
only makes sense if one looks at the full file or blob content. The
Auto[CR]LF* streams in JGit look only at the first few KiB of a stream
in order not to buffer too much.
For the C git implementation, see [1].
[1] https://github.com/git/git/blob/7e27bd589d/convert.c#L35
Bug: 576971
Change-Id: Ia169b59bdbf1477f32ee2014eeb8406f81d4b1ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Move the code to parse numbers with an optional 'k', 'm', or 'g' suffix
from the config file handling to StringUtils. This enables me to re-use
it in EGit, which has duplicate code in StorageSizeFieldEditor.
As this is generally useful functionality, providing it in the library
makes sense.
Change-Id: I86e4f5f62e14f99b35726b198ba3bbf1669418d9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The various streams used in JGit for text/binary and CR-LF detection
used different buffer sizes. Most used 8000, but one used 8KiB, and one
used 8096 (SIC!) bytes.
Considering only the first 8kB of a file/blob is not sufficient; it
may give behavior incompatible with C git. C git considers the whole
blob; since it uses memory-mapped files it can do so with acceptable
performance. Doing this in JGit would most likely incur a noticeable
performance penalty. But 8kB is a bit small; in the file in bug 576971
the limit was hit before the first CR-LF, which occurred on line 155
at offset 9759 in the file.
Make RawText.FIRST_FEW_BYTES only a default and minimum setting, and
set it to 8KiB. Make the actual buffer size configurable: provide
static methods getBufferSize() and setBuffersize(), and use
getBufferSize() throughout instead of the constant.
This enables users of the JGit library to set their own possibly larger
buffer size.
Bug: 576971
Change-Id: I447762c9a5147a521f73d2864ba59ed89f555d54
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for files having been checked in with mixed LF and CR-LF
line endings, and then being checked out with text or text=auto and
eol=lf or eol=crlf. These test cases were missing, and thus commit
efd1cc05 missed that AutoCRLFOutputStream needs the same detection
as AutoLFOutputStream.
Fix AutoCRLFOutputStream to not convert line endings if the blob in
the repository contains CR-LF.
Bug: 575393
Change-Id: Id0c7ae772e282097e95fddcd3f1f9d82aae31e43
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JEP 400 (Java 18) will change the default character set to UTF-8
unconditionally.[1] Introduce SystemReader.getDefaultCharset() that
provides the locale-dependent charset the way JEP 400 recommends.
Change all code locations using Charset.defaultCharset() to use the
new SystemReader method instead.
[1] https://openjdk.java.net/jeps/400
Change-Id: I986f97a410d2fc70748b6f93228a2d45ff100b2c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This change is a fix to http://git.eclipse.org/r/c/jgit/jgit/+/183562
that was reverted in http://git.eclipse.org/r/c/jgit/jgit/+/184978
due to deadlocks. Separate locks in DfsBlockFile are removed to rely
on getting value from DfsBlockCache with region locking in place.
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index
and index could be read from storage in parallel in separate threads.
A unit test is added for parallel index loading.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ic6d9c5a4a254628636aa98a5008447a27a003f69
* stable-5.13:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Add org.bouncycastle.bcutil to p2 repository
Change-Id: I789e3c84045cc9dcd8a26de34d09fc1e3ccc41e7
* stable-5.12:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
* stable-5.11:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
* stable-5.10:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
* stable-5.9:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
We observed the following issue:
$ git tag -a v1.0 -m v1.0
$ git push origin tag v1.0
$ git ls-remote origin v1.0^{}
... empty result ...
On the server (Gerrit) side run git-gc to pack the refs:
$ git gc
Repeat the ls-remote from the client and the result is correct:
$ git ls-remote origin v1.0^{}
7ad85c810de3ae922903d4bdd17c53cd627260ba refs/tags/v1.0^{}
Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.
It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.
Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Let certain pack extensions that are expensive to load from storage
(e.g. pack index, bitmap index) stay in DFS block cache longer than
others by overriding default cache count through DfsBlockCacheConfig
Don't change default behavior when cache override map is empty. Use int
cacheCount instead of boolean hot for Ref<T>
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I18062784ec9cc14dbba3e4bb8d9509440cf2d44f
If an annotated tag refers to a commit, we should not ignore it.
Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.
Example:
As is shown in the graph below:
1 - 2 - 3 - 4 (side)
\
5 - 6^ (master) - 7 (topic)
Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].
Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:
commit | Before | After | Rel %
47a44d27ca | 29251ms | 6687ms | -77%
90327e7dff | 21388ms | 6256ms | -70%
f85fac0efa | 11150ms | 7338ms | -34%
The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.
Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
* stable-5.8:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
* stable-5.7:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
* stable-5.6:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
In Java 11 type arguments for anonymous subclasses can be inferred
and don't need to be specified. This resolves a number of compiler
warnings.
Change-Id: I55eff3babb7628aa0627085e65a1b45eb12c2cd3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
On Java 16 this test throws InaccessibleObjectException, handle and
ignore it similar to IllegalAccessException.
Change-Id: I19b4f577579694a146516861a7ec567141f3464b
This ensures the compiler compiles against the public, supported and
documented API for a specific VM version (here 11) [1]. This also means
that
we don't need EE descriptors in Eclipse anymore in order to ensure that
only supported APIs of the selected Java version can be used.
According to [2] if option --release is used --source and --target
options can't be used.
While we are at it also add default value for all new jdt core options
added in Eclipse 4.21.
[1] https://docs.oracle.com/en/java/javase/11/tools/javac.html
[2] https://docs.oracle.com/en/java/javase/14/docs/specs/man/javac.html#option-release
Change-Id: I852a5d7b0a3210751c15d79ec91915b4c01c41e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The reftable format is a block based format, but allows for variably
sized blocks. This obviously happens for reflog blocks (which are zlib
compressed), but is also accepted for index blocks: In the spec, this
is motivated as
To achieve constant O(1) disk seeks for lookups the index must be
a single level, which is permitted to exceed the file's
configured block size, but not the format's max block size of
15.99 MiB.
Hence, when parsing a block, one cannot be sure of its exact size:
after reading a default-size block (eg. 4kb), the block header may
state that the block is in fact larger.
Before, the code would mark the block as `truncated`, noting
// Its OK during sequential scan for an index block to have been
// partially read and be truncated in-memory. This happens when
// the index block is larger than the file's blockSize. Caller
// will break out of its scan loop once it sees the blockType.
This looks like either
* a remnant of never-implemented functionality. There is no reason to
ever sequentially scan an index block.
* alluding to sequential scan of the data blocks before the index
blocks (eg. scanning refs, which ends when we find the first ref index
block, and we can then ignore the index block).
This comment is followed by code that populates the
restartTbl/restartCnt fields relative to the (possibly truncated)
buffer. If the buffer is truncated, this essentially reads garbage,
leading to OOB array access when using the index block.
Fix this by dropping the truncated logic and issuing a second read if
the first read was short.
Add a test.
We have never observed this failure scenario at Google. We use 64kb
blocksize, which requires us to need fewer index entries. The reftable
spec mentions an Android repo of size 36M. With 64kb blocks, that's
just 562 index entries. Even with historical growth, we are long from
requiring an index whose size exceeds a single block.
When adding the analogous test for seeking refs, there was no failure.
This points to another possibility which is that the code tries to
avoid writing large index blocks for refs.
I did not investigate further which one it is.
Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=576250
Bug: 576250
Change-Id: I41ec21fac9e526ef57b3d6fb57b988bd353ee338
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Before, while trying to determine if an object ID was a tag or not,
the reftable conversion would yield an exception.
Change-Id: I3688a0ffa9e774ba27f320e3840ff8cada21ecf0
* master: (38 commits)
Revert "DFS block cache: Refactor to enable parallel index loading"
GitServlet: allow to override default error handlers
Silence API error for new interface method ProtocolV2Hook#onObjectInfo
transport: add object-info capability
Ignore IllegalStateException if JVM is already shutting down
Update orbit to R20210825222808 for 2021-09
Update spotbugs-maven-plugin to 4.3.0
Update ant to 1.10.11 also in pom.xml
DFS block cache: add additional stats to DfsReaderIoStats
Update Orbit to S20210817231813
[gpg] Better GPG home directory determination
FS: cleanup use of final modifier
Ensure FS#searchPath only selects executable files
RevWalk: getMergedInto's result is wrong on the second call
DFS block cache: Refactor to enable parallel index loading
[test] Create keystore with the keytool of the running JDK
[gpg] Update to Bouncy Castle 1.69
[test] Create keystore with the keytool of the running JDK
[sshd] Minor code clean-up
Support commit.template config property
...
Change-Id: I9f99e9a513a23c0c0d252334e79c351512d7355e
Sometimes it is useful to obtain metadata associated with an object
without the need to first download it locally. This is specially useful
when using partial clones.
This change implements the object-info capability that allows clients to
query the remote server for object metadata (currently only size). This
is a backport of the same capability that was recently added to the Git
project a2ba162cda (object-info: support for retrieving object info,
2021-04-20).
Signed-off-by: Bruno Albuquerque <bga@google.com>
Change-Id: I4dc9828e1c247f08b0976b8810be92d124366165
Make sure the future user can reset all UNINTERESTING commmits after
this operation.
Signed-off-by: kylezhao <kylezhao@tencent.com>
Change-Id: I7549b9ff67bd31acd5dfc92331cb9a30b47b8278
* stable-5.12:
[test] Create keystore with the keytool of the running JDK
Fix garbage collection failing to delete pack file
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I4b54f4850819736144edb784617ee902f491ffd6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: Ie8db450a1fad05bddb812a55b2ceb03b2805403a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
[test] Create keystore with the keytool of the running JDK
ReachabilityCheckerTestCase: fix reachable from self test case
Change-Id: I55f4dd19e9fa3b789bd9a79d256fe9abb55ee7f4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Matthias Sohn