JGit accumulated in MESSAGE_FIXUP commit messages of a fixup sequence,
just like it did in MESSAGE_SQUASH, and on the last step of a sequence
of fixups used that file, after stripping all comment lines, as the
commit message. That also stripped any lines from the original commit
message that happened to start with the comment character.
This is not how this is supposed to work. MESSAGE_FIXUP must contain
the original commit message of the base commit that is amended, and
the file contains the verbatim commit message for the final fixup.[1]
Change the implementation accordingly, and add new tests.
[1] https://github.com/git/git/blob/df3c41adeb/sequencer.c#L86 ff.
Bug: 513726
Change-Id: I885a2b7f10d6c74460a8693aa6cbf867ee0494a1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reload the stack _before_ trying to delete the files. This ensures we
don't trip over our own open file handles when deleting compacted
tables.
If there is another process reading the file, it may be impossible to
delete the compacted tables. In this case, ignore the failure.
For cleaning the garbage in this case, the protocol as described in
https://www.git-scm.com/docs/reftable#_windows should be implemented.
This is left for another commit.
Bug: 578454
Change-Id: I7aa43508450041eb9376d9f67a0262ff7cc53c73
* changes:
RepoCommand: Offer to set extra files in the destination repository
RepoCommand: Move bare/regular superproject writing to their own classes
FS.list() is supposed to return an empty array if the File given is not
a directory.
Bug: 550111
Change-Id: I245da5f1f2bdafd9dfb38fb8d7eff27d900cd5a8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We want to save in the destination repository what manifest created its
structure. This helps to detect and debug failures in the manifest ->
superproject translations. The src commit should be easily readable from
the superproject tip.
Offer an API to write a file in the destination repository. RepoCommand
callers (e.g. gerrit supermanifest plugin) can use this to add a
file with the repo/ref/hash of the manifest.
Alternatives considered to write the source repo/ref/hash:
* .gitattributes of the .gitmodules file. Some updates in the manifest
don't touch the .gitmodules (e.g. a linkfile change), so it can fall
out of sync.
* commit message. Caller would need to follow the commit history to
find the latest modification by repo command. This is not helpful
e.g. for build bots that want to get the value in one call.
Change-Id: I113662734a7ccd39cbc60b46ad3f73038c807682
RepoCommand parses the manifest to get a list of projects, clears up
conflicts and then writes to the superproject. The first steps are
common but the writing is completely different for bare or "regular"
(with working dir) repository.
Split writing to bare and regular repos into its own classes. This
simplifies RepoCommand class and makes clearer what happens on each side
(e.g. many options apply only to bare repos).
Change-Id: I256e15729bd53ee15fc56de88bce86a2edb2417a
C git also does so. Note that currently the comment character is
hard-coded as the hash '#' throughout JGit.
Bug: 548529
Change-Id: I4a5597694082a9e5b07412b365cfaf41fa034cfa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Respect git config commit.cleanup for rewording. Note that by default
this is CleanupMode.STRIP, whereas before this change, JGit would take
the reworded message verbatim.
Squashing was the only place in JGit where it automatically and
unconditionally removed comment lines from commit messages. In other
places it didn't do so, and client code needed to do so.
Unconditionally removing comments is problematic if the commit message
_should_ contain some line starting with a hash, which can easily occur
with the way Github, Gitlab, and other git web servers link to issues
or PRs: they all allow the short-hand "#<number>".
Introduce a new InteractiveHandler2 extension interface, which can
return the edited message _and_ a clean-up mode. This way, client code
can decide on its own how to clean the message, and if JGit shouldn't
do any further cleaning, it can return CleanupMode.VERBATIM. Or
CleanupMode.WHITESPACE. (In the case of SQUASH, it is then of course
the client's responsibility to remove the squash comment lines.)
If the old InteractiveHandler interface is used, CleanupMode.STRIP is
applied unconditionally for squashing, as before.
Bug: 578173
Change-Id: Ia0040c247884e684587dd45d6cb85f8b72a4b876
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Use CommitConfig.CleanupMode to implement git commit --cleanup. Add
setters for the clean-up mode, the comment character, and for the
default default clean-up mode.
Behavior of existing client code is unchanged as the default clean-up
mode is set to "verbatim". To use git config defaults, one can call
setCleanupMode(CleanupMode.DEFAULT). The default comment character
is hard-coded as '#' for now, as in other parts of JGit. Implementing
full support for core.commentChar shall be done in a separate change.
Bug: 553065
Change-Id: I470785e464a762d3f409f163f1cbdbb98dd81aaf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add an enumeration for the possible values, and a method to resolve the
"default" value. Give CommitConfig a static method to process a text
according to a given clean-up mode and comment character.
(The core.commentChar is not yet handled by JGit; it's hard-coded as #.)
Bug: 553065
Change-Id: If6e384522275f73b713fbc29ffcaa1753c239dea
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the walk is created via ObjectWalk(Repository), it creates a new
ObjectReader. This reader was closed only on dispose(). If such an
ObjectWalk was used in a try-with-resource statement the reader might
not get closed.
Bug: 578458
Change-Id: I1be31829dc466530f23006a53c29b657fd5fb410
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Let's be lenient and accept hunk lines terminated by CR-LF, too, not
just lines terminated by LF.
Bug: 550111
Change-Id: I7f796df666300ab56cc6c07f22eda45fbf4c941e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Parse the value from the ssh config and if set use it when connecting.
Change-Id: I85b44c9468a5027602375706612c46ea7a99b2bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add parsing of the config. Implement the SSH agent protocol for adding
a key. In the pubkey authentication, add keys to the agent as soon as
they've been loaded successfully, before even attempting to use them
for authentication. OpenSSH does the same.
Bug: 577052
Change-Id: Id1c08d9676a74652256b22281c2f8fa0b6508fa6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
OpenSSH has (for legacy reasons?) the option of specifying the default
environment variable directly, instead of using ${SSH_AUTH_SOCK}. Make
sure the plain variable name is not taken as a relative path name.
Bug: 577053
Change-Id: If8f550dffc43887254f71aa0b487c50fa14d0627
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.
Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.
Pull up that functionality to a superclass, so it can be reused.
Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.
Add a custom consumer to report loading events and eviction events when
enabled.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).
To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.
Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.
The check was already performed when a new FileBasedConfig
is created at L158:
// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());
The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.
This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.
Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* stable-6.0:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Id7540d03991cdcf6f405e946b8cbbcc6a9696a31
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Ibd69e9d941ad9262b61dd0c4368e48cb82597a12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.12:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I6576872cc0f5dd452252fa6e4526086cdee65c28
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ib80336a42e22da729b9db1e573772504cc0a3e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I9e79ea2a0c554a184e4ce3b13e375eac8b7a4ac5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2a84c838a886d1d6383c34f50b418baa743c57b0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.8:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I88a629e571fec5a9820114ebf5765b5d94a276bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.7:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ied786ab5e3c0dd05f701705fce2d4ad85502c4d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.6:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I454622dae6eb95aedbd858e3b12da72282d36673
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.5:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2622f1d384a88a556ba9d88f0d08a37af69e530c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.4:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ia1665dd92ccc3811a6116f41421a05aca10fc6eb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.3:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I52a57a17abe60e30e3d7615f8cb4d0c5e6aebd9b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.2:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Id37f47a5ef2e3c8329eca30c171941f7e5606a85
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.1:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I625667c2718ab31ae7df907c3dd6024a933913b8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This reverts commit f829f5f838.
Using MISSING_FILEKEY as indicator for a non-existing file doesn't work
on Windows.
Bug: 577954
Change-Id: I92102a3d259f6cc0f367096a3213cfa794466817
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".
Follow the example below:
. .
└── src └── src
└── d1 ==> └── d1
└─ file1 ├─ file1
└── file2
c.parents[i] c
The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".
We should add an additional check for no paths matching the filter.
Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
Speed up bitmap creation by loading reverse index in parallel
to reading bitmap from storage. Latency changes from
(time_to_read_bitmap + time_to_load_reverse_index) to
max(time_to_read_bitmap, time_to_load_reverse_index).
Add new option to DfsReaderOptions to control parallel reverse index
loading. Static cached thread pool is added to PackBitmapIndexV1 for
reverse index loading, and when not in use consumes minimal resources.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ia37a1d739631d053e8bddb925ac8b0b81d22379e
Return immediately in scanRef if the loose ref was identified as
missing when a snapshot was attempted for the ref. This will help
performance of scanRef when the ref is packed but has a corresponding
empty dir in 'refs/'.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~2.5s, where as with this change it takes ~1.5s.
Change-Id: I926bc41b9ae89a1a792b1b5ec9a17b05271c906b
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Doing a getFileStoreAttributes call even when the file doesn't
exist is unnecessary. This call is particularly slow on some
filesystems. Instead, do it only when the file exists and load
the appropriate cache.
This update can help speed up RefDirectory.exactRef when the ref
is packed, but has a corresponding empty dir for it under 'refs/'.
This scenario can happen when an atomic 'BatchRefUpdate' creates
new sharded refs.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~30s, where as with this change it takes ~2.5s.
Change-Id: I4a5d4c6a652dbeed1f4bc3b4f2b2f1416f7ca0e7
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Let GC#gc return collection of newly created packs as CompletableFuture
to enable using gc() asynchronously.
Change-Id: I3627014fd458c738cfe54225e631d6f7d9cfb1a7
* stable-6.0:
FS: debug logging only if system config file cannot be found
FS: debug logging only if system config file cannot be found
Update .factorypath used by annotation processor for benchmarks
Use maven-compiler-plugin's release tag instead of source and target
Don't use deprecated Repository#getAllRefs in Repository
Don't use deprecated Repository#getAllRefs in FileRepository
RevListTest: fix warning that method parameter hides field 'git'
Implement RecordingLogger based on org.slf4j.Logger
Let ObjectDatabase implement AutoClosable
Change-Id: Ie6b3cfa66b319033d4448dcf20362b753c0e9d7c
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
org.eclipse.jgit.internal.diffmergetool
org.eclipse.jgit.internal.fsck
This was broken with changes for bug 356832.
Bug: 356832
Change-Id: Id008d058769b4923d545a9373b45ceb3a3d27a08
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
see: http://git-scm.com/docs/git-difftool
* add command line support for "jgit difftool"
* show supported commands with "jgit difftool --help"
* added "git difftool --tool-help" to show the tools (empty now)
* prepare for all other commands
Bug: 356832
Change-Id: Ice0c13ef7953a20feaf25e7746d62b94ff4e89e5
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- Repository#getAllRefsByPeeledObjectId
Bug: 534731
Change-Id: Id6956ff112560e6314d4335238494708346f2338
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- AdvertiseRefsHook#advertiseRefs
- ReceivePack#setAdvertisedRefs
- Repository#getAdditionalHaves
Bug: 534731
Change-Id: I85bc6ce5815d40be5f80042c53f4663072d96be5
Provide a public interface PackLock exposing only the unlock() method.
Rename the internal PackLock class to PackLockImpl and have it implement
the new interface.
This way PackParser doesn't expose an internal class via its API
anymore, and client can still unlock pack locks that were created.
Bug: 576340
Change-Id: I976739f4ab28fe1f9ba7f35653a69a913aa68841
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Remove the unused parameter, which had a non-API type anyway.
Bug: 576340
Change-Id: Id71c01a643e1f31a8ff61ff69f7915c373db3263
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
While building the commit for the destination project, RepoCommand
catches GitApiExceptions and wraps them into ManifestErrorException
(a subclass of GitApiException). This hides the real exception from the
caller and prevent them to do a fine-grained catch.
Specifically this is problematic for gerrit's supermanifest plugin, that
won't see ConcurrentRefUpdate exceptions to detect lock-failures.
Use ManifestErrorException to wrap non-GitApiExceptions, let
GitApiExceptions pass through as they are.
Change-Id: Ia2cda244e65993bb07c89cd6435507d5d0754dd4
* stable-5.12:
Better git system config finding
Change-Id: Iebbae08b6bb6ef510ca07329df77223bc2128ec1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Better git system config finding
Change-Id: I47946b91658cb2e38709179b7a8513b1211431e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Better git system config finding
Change-Id: I460d855ea7878b279dbaffa6eb7ce5ca93f4c12c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Better git system config finding
Change-Id: I1022d003e0d3b9d452abfed9ac49663d0a93c6e6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 9446e62733)
Use the GitAPIException CanceledException instead of IOException
CancelledException in the rename detector.
Change-Id: I5121719eb714a123ec57769fc113c84857bd3c6c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Skip javadoc generation for test bundles.
Use character entities < and > for < and > outside of
code-formatted spans.
Change-Id: I66e1a1dc98881c61f93c9e5561c5513896b2ba01
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
CommitConfig.getCommitTemplateContent() has changed API.
Change-Id: If72ed79c43c167b99f356b69cc095d6bbf4d66e8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fixes an issue that commit template file could not be found if it has a
relative path instead of absolute path.
Relative path is probably common if git config --local is used.
Bug: 446355
Change-Id: I8ddf2be672647be825fd9c01af82809d31bb8356
With change https://git.eclipse.org/r/c/jgit/jgit/+/186455 a thread
loading a bitmap index could hold two ref locks at the same time (one
for bitmap and one for either index or reverse index). So it is possible
that two threads loading bitmaps end up in a deadlock for ref locks e.g.
threadA has refLock[1] (for bitmap) and wants refLock[2] (for index or
revIndex) and threadB has refLock[2] (for bitmap) and wants refLock[1].
This change introduces separate pools of locks per pack extension
instead of a shared pool. So threads loading bitmap can hold two
locks but with different extensions and no overlap, e.g. threadA holds
refLock[BITMAP_INDEX][1] and refLock[INDEX][2] and threadB holds
refLock[BITMAP_INDEX][2] and refLock[INDEX][1].
More unit tests were added to cover various paralell loading scenarios.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I89704b4721c21548929608d3798ef60925280755
We use BinaryBlobException to signal a binary blob was found and never
make use of its stack trace. Suppress filling in the stack trace to
avoid the performance penalty coming with that.
See https://shipilev.net/blog/2014/exceptional-performance/
Change-Id: Iae1f1c19a1fa8aef4f6569822557171130299958
Since OpenSSH 7.8, the ProxyJump directive accepts the value "none"[1]
to override and clear a setting that might otherwise be contributed by
another (wildcard) host entry.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=2869
Change-Id: Ia35e82c6f8c58d5c6b8040cda7a07b220f43fc21
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Bring our SSH config parser up-to-date with respect to changes in
OpenSSH. In particular, they fixed[1] the handling of line comments
such that #-characters inside strings are not considered. This means
that we have to parse strings with escaped quotes correctly.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=3288
Change-Id: Ifbd9014127e8d51e7c8792e237f3fc2a9a0719d2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
It appears that the OpenSSH documentation[1] has changed; it now allows
more flags for a number of keys.
[1] https://man.openbsd.org/ssh_config.5#TOKENS
Change-Id: I55df174f86a3fd4a6ef22687dc433ac9f9ad181d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Once a factory supports different SSH agents on the same platform,
which is planned for Windows once we use Apache MINA sshd 2.8.0,
client code may need to have a way to specify which SSH agent shall
be used when the SSH config doesn't define anything.
Add a mechanism by which a ConnectorFactory can tell what Connectors
it may provide. Client code can use this to set the identityAgent
parameter of ConnectorFactory.create() to the wanted default if it
would be null otherwise.
A ConnectorDescriptor is a pair of strings: an internal name, and a
display name. The latter is included because client code might want to
communicate agent names to the user, be it in error messages or in some
chooser dialog where a user could define which of several alternative
SSH agents should be used as default. The internal name is intended to
be used in the IdentityAgent directive in ~/.ssh/config.
Also make the ConnectorFactory discovered via the ServiceLoader
accessible and overrideable. Provide static get/setDefault() methods,
similar to the SshSessionFactory itself.
Change-Id: Ie3d077395d32dfddc72bc8627e92b23636938182
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is a prerequisite for removing the JSch support bundle; otherwise
OpenSshConfigFile would be left without tests.
Copy OpenSshConfigTest from the JSch support bundle and adapt all tests
to perform the equivalent checks on OpenSshConfigFile directly. Add a
new lookupDefault() method to the SshConfigStore interface and implement
it so that it behaves the same and the tests work identically.
Change-Id: I046abd9197a8484003e77005024e5d973456f1a3
Make the default toolchain use Java 11, and fix two errorprone findings
introduced recently.
Change-Id: Iff51206fe8bdf096cb7d88cb1a499002550766cd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
C git considers not only files containing NUL bytes as binary but also
files containing lone CRs. Implement this also for JGit.
C git additionally counts printable vs. non-printable characters and
considers files that have non_printable_count > printable_count / 128
also as binary. This is not implemented because such counting probably
only makes sense if one looks at the full file or blob content. The
Auto[CR]LF* streams in JGit look only at the first few KiB of a stream
in order not to buffer too much.
For the C git implementation, see [1].
[1] https://github.com/git/git/blob/7e27bd589d/convert.c#L35
Bug: 576971
Change-Id: Ia169b59bdbf1477f32ee2014eeb8406f81d4b1ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Move the code to parse numbers with an optional 'k', 'm', or 'g' suffix
from the config file handling to StringUtils. This enables me to re-use
it in EGit, which has duplicate code in StorageSizeFieldEditor.
As this is generally useful functionality, providing it in the library
makes sense.
Change-Id: I86e4f5f62e14f99b35726b198ba3bbf1669418d9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The various streams used in JGit for text/binary and CR-LF detection
used different buffer sizes. Most used 8000, but one used 8KiB, and one
used 8096 (SIC!) bytes.
Considering only the first 8kB of a file/blob is not sufficient; it
may give behavior incompatible with C git. C git considers the whole
blob; since it uses memory-mapped files it can do so with acceptable
performance. Doing this in JGit would most likely incur a noticeable
performance penalty. But 8kB is a bit small; in the file in bug 576971
the limit was hit before the first CR-LF, which occurred on line 155
at offset 9759 in the file.
Make RawText.FIRST_FEW_BYTES only a default and minimum setting, and
set it to 8KiB. Make the actual buffer size configurable: provide
static methods getBufferSize() and setBuffersize(), and use
getBufferSize() throughout instead of the constant.
This enables users of the JGit library to set their own possibly larger
buffer size.
Bug: 576971
Change-Id: I447762c9a5147a521f73d2864ba59ed89f555d54
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for files having been checked in with mixed LF and CR-LF
line endings, and then being checked out with text or text=auto and
eol=lf or eol=crlf. These test cases were missing, and thus commit
efd1cc05 missed that AutoCRLFOutputStream needs the same detection
as AutoLFOutputStream.
Fix AutoCRLFOutputStream to not convert line endings if the blob in
the repository contains CR-LF.
Bug: 575393
Change-Id: Id0c7ae772e282097e95fddcd3f1f9d82aae31e43
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JEP 400 (Java 18) will change the default character set to UTF-8
unconditionally.[1] Introduce SystemReader.getDefaultCharset() that
provides the locale-dependent charset the way JEP 400 recommends.
Change all code locations using Charset.defaultCharset() to use the
new SystemReader method instead.
[1] https://openjdk.java.net/jeps/400
Change-Id: I986f97a410d2fc70748b6f93228a2d45ff100b2c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This change is a fix to http://git.eclipse.org/r/c/jgit/jgit/+/183562
that was reverted in http://git.eclipse.org/r/c/jgit/jgit/+/184978
due to deadlocks. Separate locks in DfsBlockFile are removed to rely
on getting value from DfsBlockCache with region locking in place.
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index
and index could be read from storage in parallel in separate threads.
A unit test is added for parallel index loading.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ic6d9c5a4a254628636aa98a5008447a27a003f69
* stable-5.13:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Add org.bouncycastle.bcutil to p2 repository
Change-Id: I789e3c84045cc9dcd8a26de34d09fc1e3ccc41e7
* stable-5.12:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
* stable-5.11:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
* stable-5.10:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
* stable-5.9:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
We observed the following issue:
$ git tag -a v1.0 -m v1.0
$ git push origin tag v1.0
$ git ls-remote origin v1.0^{}
... empty result ...
On the server (Gerrit) side run git-gc to pack the refs:
$ git gc
Repeat the ls-remote from the client and the result is correct:
$ git ls-remote origin v1.0^{}
7ad85c810de3ae922903d4bdd17c53cd627260ba refs/tags/v1.0^{}
Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.
It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.
Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Let certain pack extensions that are expensive to load from storage
(e.g. pack index, bitmap index) stay in DFS block cache longer than
others by overriding default cache count through DfsBlockCacheConfig
Don't change default behavior when cache override map is empty. Use int
cacheCount instead of boolean hot for Ref<T>
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I18062784ec9cc14dbba3e4bb8d9509440cf2d44f
If an annotated tag refers to a commit, we should not ignore it.
Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.
Example:
As is shown in the graph below:
1 - 2 - 3 - 4 (side)
\
5 - 6^ (master) - 7 (topic)
Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].
Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:
commit | Before | After | Rel %
47a44d27ca | 29251ms | 6687ms | -77%
90327e7dff | 21388ms | 6256ms | -70%
f85fac0efa | 11150ms | 7338ms | -34%
The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.
Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
* stable-5.8:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
* stable-5.7:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
* stable-5.6:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
To avoid a warning about a name clash with the field.
Change-Id: Ib1950486a9c63647e4240a268a3c9a7d613f3303
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In Java 11 type arguments for anonymous subclasses can be inferred
and don't need to be specified. This resolves a number of compiler
warnings.
Change-Id: I55eff3babb7628aa0627085e65a1b45eb12c2cd3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Sebastian Schuberth