see: https://git-scm.com/docs/git-difftool
see: https://git-scm.com/docs/git-mergetool
* now all available tools are printed with "--tool-help"
* if no diff.tool or merge.tool is defined the first available
pre-defined tool is used
TODO:
- add mergetools to difftools --> extra change or merge to this
- return the exit-code of the tool to jgit / java runtime
Bug: 356832
Change-Id: I20fb04e71ced981f5625020f461bbac24e6cec70
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
see: https://git-scm.com/docs/git-mergetool
* refactoring of content (FileElement) handling
* now the temporary files are already filled with filtered content in
the calling classes (PGM), that can be used with EGit content too
TODO:
* keep the temporaries when no change detected and the user answers no
to the question if the merge was successful
Bug: 356832
Change-Id: I86a0a052d059957d4d152c1bb94c262902c377d2
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
NOTE: port back from master branch.
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Bug: 579445
Change-Id: Iaee2301eb14e6201406398a90228ad10cfea6098
see: http://git-scm.com/docs/git-difftool
* add CommandExecutor that handles tool execution with help of "jgit.FS"
* it handles tool execution with temporary created "command file" -->
for for all "command interpreters" and parameters with spaces etc.
* using of external bash.exe at Windows (MinGW) if shell-script is
used as difftool command. It can be enabled with parameter
"jgit.usemsys2bash=auto" that checks if command contains ".sh" or
enabled / disabled with "jgit.usemsys2bash=true|false"
* added special handling for empty files (e.g. deleted, added etc.) that
are named "/dev/null"
* added creation and deletion of temporary files needed for compare
* added own Exception class for reporting to pgm / command line / EGit
* added prompt option handling before executing difftool
* reworked trustExitCode option for specific difftool and override for
all difftools from config and command line
* tested with command line options "--[no]-trust-exit-code",
"--tool=<toolname>", "--[no]-gui", --[no]-prompt
* ContentSource
* added close() methods to close / cleanup used resources
(like ObjectReader TreeWalk etc.)
* added isWorkingTreeSource() methods to check if file can be used
from working tree instead of copy from "ObjectLoader / ObjectReader" to
temporary file (fixes "difftool <commit> <commit>")
Bug: 356832
Change-Id: I5462fb6dbe4ecfd9da7c74117fce4070bbfd4d7a
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Fully buffering by the previous generator of the input for the RewriteGenerator is no longer necessary.
Bug: 577948
Signed-off-by: Simon Sohrt <sohrt@his.de>
Change-Id: I59c7a7c7f3766e97627764608bc8dc733804274c
Errorprone raises the following warning: "[ReferenceEquality] Comparison
using reference equality instead of value equality"
Added Equality#isSameInstance util method to be reused in similar cases.
Change-Id: Ifc9885c9806dcafa1c5ee1351d3095bf1517dbf5
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.
Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.
Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.
Bug: 579445
Change-Id: Ie27aca1fdaddfa487ebe072c3913d78355910df8
Use the holder pattern to de-couple the loading of super classes from
the ServiceLoader calls to set up global instances. This prevents
potential lock inversions.
Bug: 579550
Change-Id: Ie8284e4d6d680ddd4cc6a486bbefe8ed00266240
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If we're not signing the commit, we don't need the GpgSigner at all.
Change-Id: Ica35651c73d82b7604c8a67487b0d9474c908aae
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Don't store the default signer in a static field of the abstract
superclass GpgSigner. This many lead to a lock inversion on the class
initialization locks if there are concurrent loads of the GpgSigner
class and of one of its subclasses, and that subclass happens to be
the one the ServiceLoader wants to load.
Use the holder pattern to de-couple the loading of class GpgSigner
from the ServiceLoader call.
Bug: 579550
Change-Id: Ifac0ea0c8985a09fe0518d0dabc072fafd6db907
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Implement negative refspecs in JGit fetch, following C Git. Git
supports negative refspecs in source only while this change supports
them in both source and destination.
If one branch is equal to any branch or matches any pattern in the
negative refspecs collection, the branch will not be fetched even if
it's in the toFetch collection.
With this feature, users can express more complex patterns during fetch.
Change-Id: Iaa1cd4de5c08c273e198b72e12e3dadae7be709f
Sign-off-by: Yunjie Li<yunjieli@google.com>
Errorprone raises the following warning:
"[ReferenceEquality] Comparison using reference equality
instead of value equality"
Change-Id: Ib7542506dcdf5a0c7775a99a2c0972930d622879
Errorprone raises the following warning:
"[ReferenceEquality] Comparison using reference equality
instead of value equality"
Change-Id: Idfe3f044730019fb49dccee25a0563c6fc01fc6a
Errorprone raises the following warning:
"[ReferenceEquality] Comparison using reference equality
instead of value equality"
Change-Id: Ia0dc2df68c77e40ff509a2c39568abce92525ee9
Errorprone raises the following warning:
"[ReferenceEquality] Comparison using reference equality
instead of value equality"
A single pointer comparison is fine here.
A comparison with 'equals' would complicate the code and
also impact performance.
Change-Id: I6677dfc483acb95a64afbf433fd2b9460e408264
This concerns committing, creating merge conflict messages and creating
and editing squash messages. In a squash message, once the comment
character has been determined initially is always the first character.
Note that if core.commentChar=auto and there is a sequence of squashes,
it may be necessary to change the comment character when a new message
is added.
Bug: 579325
Change-Id: Idca19284a0240cd322e7512ea299a03658e1b2c1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Provide access to the core.commentChar git config, and provide a
utility method to determine an unused comment character if the setting
is "auto".
Bug: 579325
Change-Id: I1ec7e4deffea6ac5929a8538a624d73bb59e4ecc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is a feature that does not exist in C git: an external pre-push
hook doesn't know whether the push is run as a dry run. But for
internal hooks written in Java it is entirely possible to give a hook
this information.
In JGit with its internal LFS implementation, this enables us to not
perform LFS uploads in a dry run. This is kind of important because
EGit frequently does a dry-run before doing the actual push to give the
user a way to review what would be pushed before it actually happens.
Doing an LFS upload of potentially huge files during a dry-run is
wasteful, makes the dry run not actually a dry run, and leads to
uploading the same file twice if followed by a real push.
Use the information in the LfsPrePushHook to only do the initial call
to the LFS server, but then skipping the actual upload if the push is
a dry run. That way, a failure to contact the LFS server leads to an
error in the dry run, as it should.
Bug: 546567
Change-Id: I155430f27c4979d91096ba72fd95c3775dd3f28b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This guarantees that updates are checked out in git order, which
is important for LFS if a .lfsconfig file is used. That file comes
early in git order, and the LFS smudge filter will consider the
working tree version. To ensure that on branch switches the correct
version of that file is used, the checkout order must be stable and
should be the git order.
Change-Id: I20f6d11bf08558f9d5adfd2be71e36321460038c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Git and JGit are very careful to replace git files atomically when
writing. The normal mechanism for this is to write to a temporary
file and then to rename it atomically to the final destination. This
works fine on POSIX-compliant systems, but on systems where renaming
may not be atomic, exceptions may be thrown if code tries to read
the file while the rename is still ongoing. This happens in particular
on Windows, where the typical symptom is that a FileNotFoundException
with message "The process cannot access the file because it is being
used by another process" is thrown, but file.isFile() == true at the
same time.
In FileBasedConfig, a re-try was already implemented for this case.
But the same problem can also occur in other places, for instance
in RefDirectory when reading loose or packed refs. Additionally,
JGit has similar re-tries when a stale NFS file handle is detected,
but that mechanism wasn't used consistently (only for git configs
and packed refs, but not for loose refs).
Factor out the general re-try mechanism for reading into a new method
FileUtils.readWithRetry() and use that in all three places. The
re-try parameters are hardcoded: at most 5 times for stale NFS handles,
and at most 5 times with increasing backoff delays (50, 100, 200, 400,
and 800ms) for the above concurrent write case.
Bug: 579116
Change-Id: If0c2ad367446d3c0f32b509274cf8e814aca12cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the last sideband progress message didn't end in \r or \n, there
may still be a buffered message at the end of a fetch or push. Ensure
that message gets written, too, even if it may be only partial.
Bug: 575629
Change-Id: I38edccb5cffb89e00e468480b43c7d951fb63e8e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If LsRemoteCommand is used without repository, the remote URI was not
translated. If the git user config contains e.g.
[url "ssh://git@github.com/"]
insteadOf = https://github.com/
and LsRemoteCommand is called with an HTTPS GitHub repository URL, the
command should actually rewrite this to an SSH URI and use the SSH
transport.
Actually this same problem may exist everywhere Transport is used with
an URIish instead of with a remote name. However, doing this translation
in Transport.open(URIish) and in Transport.open(Repository, URIish,
String) if no remote name is given would change the behavior and might
break assumptions made in existing clients. For instance, EGit's
PushOperation assumes that the URI obtained from PushResult.getURI()
was the same as was passed in to Transport.open(Repository, URIish).
URIs obtained from a RemoteConfig have this translation applied
transparently, and in Transport we cannot know for sure whether or
not a URI has already been translated, if needed. So doing this in
Transport might also lead to translating URIs twice.
Therefore this commit does the translation in LsRemoteCommand, where
we can be sure that it won't affect other use cases. If other cases
besides LsRemoteCommand are found where such a URI translation is
missing, it'll have to be done at higher levels, possibly even in
client code directly.
Bug: 544769
Change-Id: I5df54a925d30b55d98e21f37f2851fe79649b064
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new UrlConfig that encapsulates the basic URL replacement logic
for git configs url.<base>.insteadOf and pushInsteadof.
Change-Id: Iecc61d1c5e6089533552afa1d1e684ae72393b38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.
These attributes must be used to determine whether the working tree is
dirty.
But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.
Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.
Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.
The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)
Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.
Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.
Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?
Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.
This RefSpec can be expanded only once the advertisement from the
remote has been received.
Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.
Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.
Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.
Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If core.abbrev is unset or "auto" estimate abbreviation length like C
git does:
- Estimate repository's object count by only considering packed objects,
round up to next power of 2
- With the order of 2^len objects, we expect a collision at 2^(len/2).
But we also care about hex chars, not bits, and there are 4 bits per
hex. So all together we need to divide by 2; but we also want to round
odd numbers up, hence adding one before dividing.
- For small repos use at least 7 hexdigits
- If object database fails to determine object count use 7 hexdigits as
fallback
If it is set to "no" do not abbreviate object-ids.
Otherwise set it to the configured value capped to the range between 4
and length of an unabbreviated object-id.
Change-Id: I425f9724b69813dbb57872466bf2d2e1d6dc72c6
Error Prone is flagging Date-API as obsolete and recommends to migrate
to Instant and LocalDate. Given that more JGit users starting to migrate
to new Time API, offer ctors that accept Instant type and also add new
getter that returns when attribute as Instant type.
Change-Id: I64a36bf40f191495c6889c1dff314ede06848880
Errorprone was failing the build with: "[OperatorPrecedence]. Use
grouping parenthesis to make the operator precedence explicit"
Add parentheses to silence it.
Change-Id: I81f1f249e38fd2543f5412b3501b0179d0759f55
All the filtering in WorkingTreeIterator is for check-in, i.e., clean
filtering. The implementation was in some parts too general, passing
around an OperationType. But since it's always CHECKIN_OP, that's not
actually necessary.
Change-Id: I73f8bc059e485a073e456962868f52b3a3db4fc1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
There is no need to load all refs if there are no wildcard push
refspecs. Load them lazily on the first wildcard refspec encountered
instead of loading them up-front.
Change-Id: I6d0e981f9ed4997dbdefeb7f83f37ff4f33e06a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add ConfigConstants and expose branch.<name>.pushRemote in the
BranchConfig. Use the branch configuration and remote.pushDefault
if no remote is given explicitly. If nothing is configured, fall
back to "origin".
Bug: 578676
Change-Id: I6bb141ff02c8b04980ec34b26ef248b72614c3c9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When no RefSpecs are given, PushCommand until now simply fell back to
pushing the current branch to an upstream branch of the same name. This
corresponds to push.default=current. Any setting from the git config
for push.default was simply ignored.
Implement the other modes (nothing, matching, upstream, and simple),
too. Add a setter and getter for the PushDefault so that an application
can force a particular mode to be used. For backwards compatibility,
use "current" as the default setting; to figure out the value from the
git config, which defaults to "simple", call setPushDefault(null).
Bug: 351314
Change-Id: I86c5402318771e47d80b137e99947762e1150bb4
Signed-off-by: Rolf Theunissen <rolf.theunissen@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We checked if the command instance is called but missed to set the flag
tracking if it actually was used.
Change-Id: I0f9fb85c47945a4d91eab01ec5e1abc1ab83332a
Enhance the (unused!?) PushConfig; include a PushDefault enumeration.
Add simple tests for this PushConfig.
Bug: 351314
Change-Id: Ibc5656a2a1fccf70d00c5e15de8ed3dd8add6337
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
BasePackConnection::readAdvertisedRefsImpl was creating an exception by
calling `noRepository`, and then blindly calling `initCause` on it. As
`noRepository` can be overridden, it's not guaranteed to be missing a
cause.
BasePackPushConnection overrides `noRepository` and initiates a fetch,
which may throw a `NoRemoteRepositoryException` with a cause.
In this case calling `initCause` threw an `IllegalStateException`.
In order to throw the correct exception, we now return the
BasePackPushConnection exception and suppress the one thrown by
BasePackConnection
Bug: 578511
Change-Id: Ic1018b214be1e83d895979ee6c7cbce3f6765f6f
According to Git LFS documentation, URLs with and without .git suffix
should be supported. By default, Git LFS will append .git/info/lfs to
the end of a Git remote URL. To build the LFS server URL it will use:
Git Remote: https://git-server.com/foo/bar
LFS Server: https://git-server.com/foo/bar.git/info/lfs
Git Remote: https://git-server.com/foo/bar.git
LFS Server: https://git-server.com/foo/bar.git/info/lfs
Fix the LfsConnectionFactory accordingly. Move a utility method to
add the ".git" suffix if not present yet from FileResolver to
StringUtils and use it.
Bug: 578621
Change-Id: I8d3645872d5f03bb8e82c9c73647adb3e81ce484
Signed-off-by: Nail Samatov <sanail@yandex.ru>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a way for the handler to tell whether the commit should generate a
Gerrit Change-Id. Augment the ModifyResult interface, and set the flag
on the CommitCommand.
This enables users to have a Change-ID be generated when squashing or
rewording commits. A possibly already existing Change-Id will remain
unchanged.
Bug: 440211
Change-Id: I66a72e0646876d162a7011235cca969e20acf060
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit accumulated in MESSAGE_FIXUP commit messages of a fixup sequence,
just like it did in MESSAGE_SQUASH, and on the last step of a sequence
of fixups used that file, after stripping all comment lines, as the
commit message. That also stripped any lines from the original commit
message that happened to start with the comment character.
This is not how this is supposed to work. MESSAGE_FIXUP must contain
the original commit message of the base commit that is amended, and
the file contains the verbatim commit message for the final fixup.[1]
Change the implementation accordingly, and add new tests.
[1] https://github.com/git/git/blob/df3c41adeb/sequencer.c#L86 ff.
Bug: 513726
Change-Id: I885a2b7f10d6c74460a8693aa6cbf867ee0494a1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reload the stack _before_ trying to delete the files. This ensures we
don't trip over our own open file handles when deleting compacted
tables.
If there is another process reading the file, it may be impossible to
delete the compacted tables. In this case, ignore the failure.
For cleaning the garbage in this case, the protocol as described in
https://www.git-scm.com/docs/reftable#_windows should be implemented.
This is left for another commit.
Bug: 578454
Change-Id: I7aa43508450041eb9376d9f67a0262ff7cc53c73
* changes:
RepoCommand: Offer to set extra files in the destination repository
RepoCommand: Move bare/regular superproject writing to their own classes
FS.list() is supposed to return an empty array if the File given is not
a directory.
Bug: 550111
Change-Id: I245da5f1f2bdafd9dfb38fb8d7eff27d900cd5a8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We want to save in the destination repository what manifest created its
structure. This helps to detect and debug failures in the manifest ->
superproject translations. The src commit should be easily readable from
the superproject tip.
Offer an API to write a file in the destination repository. RepoCommand
callers (e.g. gerrit supermanifest plugin) can use this to add a
file with the repo/ref/hash of the manifest.
Alternatives considered to write the source repo/ref/hash:
* .gitattributes of the .gitmodules file. Some updates in the manifest
don't touch the .gitmodules (e.g. a linkfile change), so it can fall
out of sync.
* commit message. Caller would need to follow the commit history to
find the latest modification by repo command. This is not helpful
e.g. for build bots that want to get the value in one call.
Change-Id: I113662734a7ccd39cbc60b46ad3f73038c807682
RepoCommand parses the manifest to get a list of projects, clears up
conflicts and then writes to the superproject. The first steps are
common but the writing is completely different for bare or "regular"
(with working dir) repository.
Split writing to bare and regular repos into its own classes. This
simplifies RepoCommand class and makes clearer what happens on each side
(e.g. many options apply only to bare repos).
Change-Id: I256e15729bd53ee15fc56de88bce86a2edb2417a
C git also does so. Note that currently the comment character is
hard-coded as the hash '#' throughout JGit.
Bug: 548529
Change-Id: I4a5597694082a9e5b07412b365cfaf41fa034cfa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Respect git config commit.cleanup for rewording. Note that by default
this is CleanupMode.STRIP, whereas before this change, JGit would take
the reworded message verbatim.
Squashing was the only place in JGit where it automatically and
unconditionally removed comment lines from commit messages. In other
places it didn't do so, and client code needed to do so.
Unconditionally removing comments is problematic if the commit message
_should_ contain some line starting with a hash, which can easily occur
with the way Github, Gitlab, and other git web servers link to issues
or PRs: they all allow the short-hand "#<number>".
Introduce a new InteractiveHandler2 extension interface, which can
return the edited message _and_ a clean-up mode. This way, client code
can decide on its own how to clean the message, and if JGit shouldn't
do any further cleaning, it can return CleanupMode.VERBATIM. Or
CleanupMode.WHITESPACE. (In the case of SQUASH, it is then of course
the client's responsibility to remove the squash comment lines.)
If the old InteractiveHandler interface is used, CleanupMode.STRIP is
applied unconditionally for squashing, as before.
Bug: 578173
Change-Id: Ia0040c247884e684587dd45d6cb85f8b72a4b876
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Use CommitConfig.CleanupMode to implement git commit --cleanup. Add
setters for the clean-up mode, the comment character, and for the
default default clean-up mode.
Behavior of existing client code is unchanged as the default clean-up
mode is set to "verbatim". To use git config defaults, one can call
setCleanupMode(CleanupMode.DEFAULT). The default comment character
is hard-coded as '#' for now, as in other parts of JGit. Implementing
full support for core.commentChar shall be done in a separate change.
Bug: 553065
Change-Id: I470785e464a762d3f409f163f1cbdbb98dd81aaf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add an enumeration for the possible values, and a method to resolve the
"default" value. Give CommitConfig a static method to process a text
according to a given clean-up mode and comment character.
(The core.commentChar is not yet handled by JGit; it's hard-coded as #.)
Bug: 553065
Change-Id: If6e384522275f73b713fbc29ffcaa1753c239dea
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the walk is created via ObjectWalk(Repository), it creates a new
ObjectReader. This reader was closed only on dispose(). If such an
ObjectWalk was used in a try-with-resource statement the reader might
not get closed.
Bug: 578458
Change-Id: I1be31829dc466530f23006a53c29b657fd5fb410
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Let's be lenient and accept hunk lines terminated by CR-LF, too, not
just lines terminated by LF.
Bug: 550111
Change-Id: I7f796df666300ab56cc6c07f22eda45fbf4c941e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Parse the value from the ssh config and if set use it when connecting.
Change-Id: I85b44c9468a5027602375706612c46ea7a99b2bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add parsing of the config. Implement the SSH agent protocol for adding
a key. In the pubkey authentication, add keys to the agent as soon as
they've been loaded successfully, before even attempting to use them
for authentication. OpenSSH does the same.
Bug: 577052
Change-Id: Id1c08d9676a74652256b22281c2f8fa0b6508fa6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
OpenSSH has (for legacy reasons?) the option of specifying the default
environment variable directly, instead of using ${SSH_AUTH_SOCK}. Make
sure the plain variable name is not taken as a relative path name.
Bug: 577053
Change-Id: If8f550dffc43887254f71aa0b487c50fa14d0627
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.
Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.
Pull up that functionality to a superclass, so it can be reused.
Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.
Add a custom consumer to report loading events and eviction events when
enabled.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).
To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.
Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.
The check was already performed when a new FileBasedConfig
is created at L158:
// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());
The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.
This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.
Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
* stable-6.0:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Id7540d03991cdcf6f405e946b8cbbcc6a9696a31
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.13:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
TreeRevFilter: fix wrong stop when the given path disappears
Change-Id: Ibd69e9d941ad9262b61dd0c4368e48cb82597a12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.12:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I6576872cc0f5dd452252fa6e4526086cdee65c28
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ib80336a42e22da729b9db1e573772504cc0a3e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I9e79ea2a0c554a184e4ce3b13e375eac8b7a4ac5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2a84c838a886d1d6383c34f50b418baa743c57b0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.8:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I88a629e571fec5a9820114ebf5765b5d94a276bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.7:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ied786ab5e3c0dd05f701705fce2d4ad85502c4d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.6:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I454622dae6eb95aedbd858e3b12da72282d36673
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.5:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I2622f1d384a88a556ba9d88f0d08a37af69e530c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.4:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Ia1665dd92ccc3811a6116f41421a05aca10fc6eb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.3:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I52a57a17abe60e30e3d7615f8cb4d0c5e6aebd9b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.2:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: Id37f47a5ef2e3c8329eca30c171941f7e5606a85
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.1:
Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
Change-Id: I625667c2718ab31ae7df907c3dd6024a933913b8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This reverts commit f829f5f838.
Using MISSING_FILEKEY as indicator for a non-existing file doesn't work
on Windows.
Bug: 577954
Change-Id: I92102a3d259f6cc0f367096a3213cfa794466817
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".
Follow the example below:
. .
└── src └── src
└── d1 ==> └── d1
└─ file1 ├─ file1
└── file2
c.parents[i] c
The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".
We should add an additional check for no paths matching the filter.
Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
Speed up bitmap creation by loading reverse index in parallel
to reading bitmap from storage. Latency changes from
(time_to_read_bitmap + time_to_load_reverse_index) to
max(time_to_read_bitmap, time_to_load_reverse_index).
Add new option to DfsReaderOptions to control parallel reverse index
loading. Static cached thread pool is added to PackBitmapIndexV1 for
reverse index loading, and when not in use consumes minimal resources.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ia37a1d739631d053e8bddb925ac8b0b81d22379e
Return immediately in scanRef if the loose ref was identified as
missing when a snapshot was attempted for the ref. This will help
performance of scanRef when the ref is packed but has a corresponding
empty dir in 'refs/'.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~2.5s, where as with this change it takes ~1.5s.
Change-Id: I926bc41b9ae89a1a792b1b5ec9a17b05271c906b
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Doing a getFileStoreAttributes call even when the file doesn't
exist is unnecessary. This call is particularly slow on some
filesystems. Instead, do it only when the file exists and load
the appropriate cache.
This update can help speed up RefDirectory.exactRef when the ref
is packed, but has a corresponding empty dir for it under 'refs/'.
This scenario can happen when an atomic 'BatchRefUpdate' creates
new sharded refs.
For example, consider the case where we create 50k sharded refs in
a new namespace called 'new-refs' using an atomic 'BatchRefUpdate'.
The refs are named like 'refs/new-refs/01/1/1', 'refs/new-refs/01/1/2',
'refs/new-refs/01/1/3' and so on. After the refs are created, the
'new-refs' namespace looks like below:
$ find refs/new-refs -type f | wc -l
0
$ find refs/new-refs -type d | wc -l
5101
At this point, an 'exactRef' call on each of the 50k refs without
this change takes ~30s, where as with this change it takes ~2.5s.
Change-Id: I4a5d4c6a652dbeed1f4bc3b4f2b2f1416f7ca0e7
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Let GC#gc return collection of newly created packs as CompletableFuture
to enable using gc() asynchronously.
Change-Id: I3627014fd458c738cfe54225e631d6f7d9cfb1a7
* stable-6.0:
FS: debug logging only if system config file cannot be found
FS: debug logging only if system config file cannot be found
Update .factorypath used by annotation processor for benchmarks
Use maven-compiler-plugin's release tag instead of source and target
Don't use deprecated Repository#getAllRefs in Repository
Don't use deprecated Repository#getAllRefs in FileRepository
RevListTest: fix warning that method parameter hides field 'git'
Implement RecordingLogger based on org.slf4j.Logger
Let ObjectDatabase implement AutoClosable
Change-Id: Ie6b3cfa66b319033d4448dcf20362b753c0e9d7c
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.
Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
see: http://git-scm.com/docs/git-difftool
* add command line support for "jgit difftool"
* show supported commands with "jgit difftool --help"
* added "git difftool --tool-help" to show the tools (empty now)
* prepare for all other commands
Bug: 356832
Change-Id: Ice0c13ef7953a20feaf25e7746d62b94ff4e89e5
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- Repository#getAllRefsByPeeledObjectId
Bug: 534731
Change-Id: Id6956ff112560e6314d4335238494708346f2338
Also expose the potentially IOException thrown by RefDatabase#getRefs.
Hence the following methods now potentially throw IOException:
- AdvertiseRefsHook#advertiseRefs
- ReceivePack#setAdvertisedRefs
- Repository#getAdditionalHaves
Bug: 534731
Change-Id: I85bc6ce5815d40be5f80042c53f4663072d96be5
Provide a public interface PackLock exposing only the unlock() method.
Rename the internal PackLock class to PackLockImpl and have it implement
the new interface.
This way PackParser doesn't expose an internal class via its API
anymore, and client can still unlock pack locks that were created.
Bug: 576340
Change-Id: I976739f4ab28fe1f9ba7f35653a69a913aa68841
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Remove the unused parameter, which had a non-API type anyway.
Bug: 576340
Change-Id: Id71c01a643e1f31a8ff61ff69f7915c373db3263
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
While building the commit for the destination project, RepoCommand
catches GitApiExceptions and wraps them into ManifestErrorException
(a subclass of GitApiException). This hides the real exception from the
caller and prevent them to do a fine-grained catch.
Specifically this is problematic for gerrit's supermanifest plugin, that
won't see ConcurrentRefUpdate exceptions to detect lock-failures.
Use ManifestErrorException to wrap non-GitApiExceptions, let
GitApiExceptions pass through as they are.
Change-Id: Ia2cda244e65993bb07c89cd6435507d5d0754dd4
* stable-5.12:
Better git system config finding
Change-Id: Iebbae08b6bb6ef510ca07329df77223bc2128ec1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Better git system config finding
Change-Id: I47946b91658cb2e38709179b7a8513b1211431e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.10:
Better git system config finding
Change-Id: I460d855ea7878b279dbaffa6eb7ce5ca93f4c12c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.9:
Better git system config finding
Change-Id: I1022d003e0d3b9d452abfed9ac49663d0a93c6e6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 9446e62733)
Use the GitAPIException CanceledException instead of IOException
CancelledException in the rename detector.
Change-Id: I5121719eb714a123ec57769fc113c84857bd3c6c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Skip javadoc generation for test bundles.
Use character entities < and > for < and > outside of
code-formatted spans.
Change-Id: I66e1a1dc98881c61f93c9e5561c5513896b2ba01
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.
Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.
Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of
git config --system --show-origin --list -z
"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)
Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
CommitConfig.getCommitTemplateContent() has changed API.
Change-Id: If72ed79c43c167b99f356b69cc095d6bbf4d66e8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Fixes an issue that commit template file could not be found if it has a
relative path instead of absolute path.
Relative path is probably common if git config --local is used.
Bug: 446355
Change-Id: I8ddf2be672647be825fd9c01af82809d31bb8356
With change https://git.eclipse.org/r/c/jgit/jgit/+/186455 a thread
loading a bitmap index could hold two ref locks at the same time (one
for bitmap and one for either index or reverse index). So it is possible
that two threads loading bitmaps end up in a deadlock for ref locks e.g.
threadA has refLock[1] (for bitmap) and wants refLock[2] (for index or
revIndex) and threadB has refLock[2] (for bitmap) and wants refLock[1].
This change introduces separate pools of locks per pack extension
instead of a shared pool. So threads loading bitmap can hold two
locks but with different extensions and no overlap, e.g. threadA holds
refLock[BITMAP_INDEX][1] and refLock[INDEX][2] and threadB holds
refLock[BITMAP_INDEX][2] and refLock[INDEX][1].
More unit tests were added to cover various paralell loading scenarios.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I89704b4721c21548929608d3798ef60925280755
We use BinaryBlobException to signal a binary blob was found and never
make use of its stack trace. Suppress filling in the stack trace to
avoid the performance penalty coming with that.
See https://shipilev.net/blog/2014/exceptional-performance/
Change-Id: Iae1f1c19a1fa8aef4f6569822557171130299958
Since OpenSSH 7.8, the ProxyJump directive accepts the value "none"[1]
to override and clear a setting that might otherwise be contributed by
another (wildcard) host entry.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=2869
Change-Id: Ia35e82c6f8c58d5c6b8040cda7a07b220f43fc21
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Bring our SSH config parser up-to-date with respect to changes in
OpenSSH. In particular, they fixed[1] the handling of line comments
such that #-characters inside strings are not considered. This means
that we have to parse strings with escaped quotes correctly.
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=3288
Change-Id: Ifbd9014127e8d51e7c8792e237f3fc2a9a0719d2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
It appears that the OpenSSH documentation[1] has changed; it now allows
more flags for a number of keys.
[1] https://man.openbsd.org/ssh_config.5#TOKENS
Change-Id: I55df174f86a3fd4a6ef22687dc433ac9f9ad181d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Once a factory supports different SSH agents on the same platform,
which is planned for Windows once we use Apache MINA sshd 2.8.0,
client code may need to have a way to specify which SSH agent shall
be used when the SSH config doesn't define anything.
Add a mechanism by which a ConnectorFactory can tell what Connectors
it may provide. Client code can use this to set the identityAgent
parameter of ConnectorFactory.create() to the wanted default if it
would be null otherwise.
A ConnectorDescriptor is a pair of strings: an internal name, and a
display name. The latter is included because client code might want to
communicate agent names to the user, be it in error messages or in some
chooser dialog where a user could define which of several alternative
SSH agents should be used as default. The internal name is intended to
be used in the IdentityAgent directive in ~/.ssh/config.
Also make the ConnectorFactory discovered via the ServiceLoader
accessible and overrideable. Provide static get/setDefault() methods,
similar to the SshSessionFactory itself.
Change-Id: Ie3d077395d32dfddc72bc8627e92b23636938182
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is a prerequisite for removing the JSch support bundle; otherwise
OpenSshConfigFile would be left without tests.
Copy OpenSshConfigTest from the JSch support bundle and adapt all tests
to perform the equivalent checks on OpenSshConfigFile directly. Add a
new lookupDefault() method to the SshConfigStore interface and implement
it so that it behaves the same and the tests work identically.
Change-Id: I046abd9197a8484003e77005024e5d973456f1a3
Make the default toolchain use Java 11, and fix two errorprone findings
introduced recently.
Change-Id: Iff51206fe8bdf096cb7d88cb1a499002550766cd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
C git considers not only files containing NUL bytes as binary but also
files containing lone CRs. Implement this also for JGit.
C git additionally counts printable vs. non-printable characters and
considers files that have non_printable_count > printable_count / 128
also as binary. This is not implemented because such counting probably
only makes sense if one looks at the full file or blob content. The
Auto[CR]LF* streams in JGit look only at the first few KiB of a stream
in order not to buffer too much.
For the C git implementation, see [1].
[1] https://github.com/git/git/blob/7e27bd589d/convert.c#L35
Bug: 576971
Change-Id: Ia169b59bdbf1477f32ee2014eeb8406f81d4b1ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Move the code to parse numbers with an optional 'k', 'm', or 'g' suffix
from the config file handling to StringUtils. This enables me to re-use
it in EGit, which has duplicate code in StorageSizeFieldEditor.
As this is generally useful functionality, providing it in the library
makes sense.
Change-Id: I86e4f5f62e14f99b35726b198ba3bbf1669418d9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The various streams used in JGit for text/binary and CR-LF detection
used different buffer sizes. Most used 8000, but one used 8KiB, and one
used 8096 (SIC!) bytes.
Considering only the first 8kB of a file/blob is not sufficient; it
may give behavior incompatible with C git. C git considers the whole
blob; since it uses memory-mapped files it can do so with acceptable
performance. Doing this in JGit would most likely incur a noticeable
performance penalty. But 8kB is a bit small; in the file in bug 576971
the limit was hit before the first CR-LF, which occurred on line 155
at offset 9759 in the file.
Make RawText.FIRST_FEW_BYTES only a default and minimum setting, and
set it to 8KiB. Make the actual buffer size configurable: provide
static methods getBufferSize() and setBuffersize(), and use
getBufferSize() throughout instead of the constant.
This enables users of the JGit library to set their own possibly larger
buffer size.
Bug: 576971
Change-Id: I447762c9a5147a521f73d2864ba59ed89f555d54
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for files having been checked in with mixed LF and CR-LF
line endings, and then being checked out with text or text=auto and
eol=lf or eol=crlf. These test cases were missing, and thus commit
efd1cc05 missed that AutoCRLFOutputStream needs the same detection
as AutoLFOutputStream.
Fix AutoCRLFOutputStream to not convert line endings if the blob in
the repository contains CR-LF.
Bug: 575393
Change-Id: Id0c7ae772e282097e95fddcd3f1f9d82aae31e43
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JEP 400 (Java 18) will change the default character set to UTF-8
unconditionally.[1] Introduce SystemReader.getDefaultCharset() that
provides the locale-dependent charset the way JEP 400 recommends.
Change all code locations using Charset.defaultCharset() to use the
new SystemReader method instead.
[1] https://openjdk.java.net/jeps/400
Change-Id: I986f97a410d2fc70748b6f93228a2d45ff100b2c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This change is a fix to http://git.eclipse.org/r/c/jgit/jgit/+/183562
that was reverted in http://git.eclipse.org/r/c/jgit/jgit/+/184978
due to deadlocks. Separate locks in DfsBlockFile are removed to rely
on getting value from DfsBlockCache with region locking in place.
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index
and index could be read from storage in parallel in separate threads.
A unit test is added for parallel index loading.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ic6d9c5a4a254628636aa98a5008447a27a003f69
* stable-5.13:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Add org.bouncycastle.bcutil to p2 repository
Change-Id: I789e3c84045cc9dcd8a26de34d09fc1e3ccc41e7
* stable-5.12:
Fix missing peel-part in lsRefsV2 for loose annotated tags
Fix RevWalk.getMergedInto() ignores annotated tags
Optimize RevWalk.getMergedInto()
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
* stable-5.11:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
* stable-5.10:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
* stable-5.9:
Fix missing peel-part in lsRefsV2 for loose annotated tags
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Fix running benchmarks from bazel
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
We observed the following issue:
$ git tag -a v1.0 -m v1.0
$ git push origin tag v1.0
$ git ls-remote origin v1.0^{}
... empty result ...
On the server (Gerrit) side run git-gc to pack the refs:
$ git gc
Repeat the ls-remote from the client and the result is correct:
$ git ls-remote origin v1.0^{}
7ad85c810de3ae922903d4bdd17c53cd627260ba refs/tags/v1.0^{}
Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.
It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.
Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Let certain pack extensions that are expensive to load from storage
(e.g. pack index, bitmap index) stay in DFS block cache longer than
others by overriding default cache count through DfsBlockCacheConfig
Don't change default behavior when cache override map is empty. Use int
cacheCount instead of boolean hot for Ref<T>
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I18062784ec9cc14dbba3e4bb8d9509440cf2d44f
If an annotated tag refers to a commit, we should not ignore it.
Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.
Example:
As is shown in the graph below:
1 - 2 - 3 - 4 (side)
\
5 - 6^ (master) - 7 (topic)
Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].
Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:
commit | Before | After | Rel %
47a44d27ca | 29251ms | 6687ms | -77%
90327e7dff | 21388ms | 6256ms | -70%
f85fac0efa | 11150ms | 7338ms | -34%
The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.
Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
* stable-5.8:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
* stable-5.7:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
* stable-5.6:
reftable: drop code for truncated reads
reftable: pass on invalid object ID in conversion
Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
To avoid a warning about a name clash with the field.
Change-Id: Ib1950486a9c63647e4240a268a3c9a7d613f3303
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In Java 11 type arguments for anonymous subclasses can be inferred
and don't need to be specified. This resolves a number of compiler
warnings.
Change-Id: I55eff3babb7628aa0627085e65a1b45eb12c2cd3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The reftable format is a block based format, but allows for variably
sized blocks. This obviously happens for reflog blocks (which are zlib
compressed), but is also accepted for index blocks: In the spec, this
is motivated as
To achieve constant O(1) disk seeks for lookups the index must be
a single level, which is permitted to exceed the file's
configured block size, but not the format's max block size of
15.99 MiB.
Hence, when parsing a block, one cannot be sure of its exact size:
after reading a default-size block (eg. 4kb), the block header may
state that the block is in fact larger.
Before, the code would mark the block as `truncated`, noting
// Its OK during sequential scan for an index block to have been
// partially read and be truncated in-memory. This happens when
// the index block is larger than the file's blockSize. Caller
// will break out of its scan loop once it sees the blockType.
This looks like either
* a remnant of never-implemented functionality. There is no reason to
ever sequentially scan an index block.
* alluding to sequential scan of the data blocks before the index
blocks (eg. scanning refs, which ends when we find the first ref index
block, and we can then ignore the index block).
This comment is followed by code that populates the
restartTbl/restartCnt fields relative to the (possibly truncated)
buffer. If the buffer is truncated, this essentially reads garbage,
leading to OOB array access when using the index block.
Fix this by dropping the truncated logic and issuing a second read if
the first read was short.
Add a test.
We have never observed this failure scenario at Google. We use 64kb
blocksize, which requires us to need fewer index entries. The reftable
spec mentions an Android repo of size 36M. With 64kb blocks, that's
just 562 index entries. Even with historical growth, we are long from
requiring an index whose size exceeds a single block.
When adding the analogous test for seeking refs, there was no failure.
This points to another possibility which is that the code tries to
avoid writing large index blocks for refs.
I did not investigate further which one it is.
Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=576250
Bug: 576250
Change-Id: I41ec21fac9e526ef57b3d6fb57b988bd353ee338
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Before, while trying to determine if an object ID was a tag or not,
the reftable conversion would yield an exception.
Change-Id: I3688a0ffa9e774ba27f320e3840ff8cada21ecf0
* master: (38 commits)
Revert "DFS block cache: Refactor to enable parallel index loading"
GitServlet: allow to override default error handlers
Silence API error for new interface method ProtocolV2Hook#onObjectInfo
transport: add object-info capability
Ignore IllegalStateException if JVM is already shutting down
Update orbit to R20210825222808 for 2021-09
Update spotbugs-maven-plugin to 4.3.0
Update ant to 1.10.11 also in pom.xml
DFS block cache: add additional stats to DfsReaderIoStats
Update Orbit to S20210817231813
[gpg] Better GPG home directory determination
FS: cleanup use of final modifier
Ensure FS#searchPath only selects executable files
RevWalk: getMergedInto's result is wrong on the second call
DFS block cache: Refactor to enable parallel index loading
[test] Create keystore with the keytool of the running JDK
[gpg] Update to Bouncy Castle 1.69
[test] Create keystore with the keytool of the running JDK
[sshd] Minor code clean-up
Support commit.template config property
...
Change-Id: I9f99e9a513a23c0c0d252334e79c351512d7355e
Sometimes it is useful to obtain metadata associated with an object
without the need to first download it locally. This is specially useful
when using partial clones.
This change implements the object-info capability that allows clients to
query the remote server for object metadata (currently only size). This
is a backport of the same capability that was recently added to the Git
project a2ba162cda (object-info: support for retrieving object info,
2021-04-20).
Signed-off-by: Bruno Albuquerque <bga@google.com>
Change-Id: I4dc9828e1c247f08b0976b8810be92d124366165
Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.
Bug: 575367
Change-Id: Ic967c16c7f566c84778795315ab369e76668b364
New stats are populated in DfsPackFile to provide details about loading
pack, bitmap and reverse indexes.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Ib6e8384ecc31821261e5099d0768c0b1227e8364
On Posix non executable files on the path should be ignored, on Windows
File#canExecute always returns true.
While we are here also add missing braces.
Bug: 575385
Change-Id: I80cd5057bdf9632a17f103db6f1356e975b6e150
Make sure the future user can reset all UNINTERESTING commmits after
this operation.
Signed-off-by: kylezhao <kylezhao@tencent.com>
Change-Id: I7549b9ff67bd31acd5dfc92331cb9a30b47b8278
With this change bitmap index creation is not blocked on index and
reverse index full initialization in DfsPackFile. Now bitmap index and index could be
read from storage in parallel in separate threads.
Update PackBitmapIndexV1 constructor to get packIndex and reverseIndex
suppliers instead of actual objects. Inner class IdxPositionBitmap was added to initialize pack objects after bitmap
index is fully read from storage. Update DfsPackFile and PackBitmapIndex
accordingly.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: Iea59ab58501b2acbbf9263412982ec9c6898a7ee
Adds functionality to read the git commit.template property. The
template content is read either via a default encoding or, if present,
via encoding specified by i18n.commitEncoding property.
Bug: 446355
Change-Id: I0c45db98e324ddff26a7e0262835f259d6528a86
Signed-off-by: Julian Ruppel <julian.ruppel@sap.com>
* master:
searchForReuse might impact performance in large repositories
Retry loose object read upon "Stale file handle" exception
Ignore missing javadoc in test bundles
Upgrade maven-dependency-plugin to 3.2.0
Upgrade jacoco-maven-plugin to 0.8.7
Upgrade maven-jxr-plugin to 3.1.1
Fix garbage collection failing to delete pack file
Fix PathSuffixFilter: can decide only on full paths
Update jetty to 9.4.42.v20210604
[sshd] Log the full KEX negotiation result
[releng] japicmp: update last release version
CONTRIBUTING: add explicit link to ECA
CONTRIBUTING: Use standard markdown format
Add Eclipse code of conduct and security policy
Change-Id: I9e31cb6e22f5398db6e220f71693e87475c598bd
Repository.getAllRefs() is deprecated and should not
be used anymore. Leverage the ref-db and the retrieval
of refs by prefix and adapt the result to the expected
refname/ref map.
Bug: 534731
Change-Id: I37a9092859f220ddc4e5063d01544f3e82208be8
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
The search for reuse phase for *all* the objects scans *all*
the packfiles, looking for the best candidate to serve back to the
client.
This can lead to an expensive operation when the number of
packfiles and objects is high.
Add parameter "pack.searchForReuseTimeout" to limit the time spent
on this search.
Change-Id: I54f5cddb6796fdc93ad9585c2ab4b44854fa6c48
When reading loose objects over NFS it is possible that the OS syscall
would fail with ESTALE errors: This happens when the open file
descriptor no longer refers to a valid file.
Notoriously it is possible to hit this scenario when git data is shared
among multiple clients, for example by multiple gerrit instances in HA.
If one of the two clients performs a GC operation that would cause the
packing and then the pruning of loose objects, the other client might
still hold a reference to those objects, which would cause an exception
to bubble up the stack.
The Linux NFS FAQ[1] (at point A.10), suggests that the proper way to
handle such ESTALE scenarios is to:
"[...] close the file or directory where the error occurred, and reopen
it so the NFS client can resolve the pathname again and retrieve the new
file handle."
In case of a stale file handle exception, we now attempt to read the
loose object again (up to 5 times), until we either succeed or encounter
a FileNotFoundException, in which case the search can continue to
Packfiles and alternates.
The limit of 5 provides an arbitrary upper bounds that is consistent to
the one chosen when handling stale file handles for packed-refs
files (see [2] for context).
[1] http://nfs.sourceforge.net/
[2] https://git.eclipse.org/r/c/jgit/jgit/+/54350
Bug: 573791
Change-Id: I9950002f772bbd8afeb9c6108391923be9d0ef51
The loosen() method has opened pack file and the open pack file handle
may prevent it from being deleted e.g. on Windows. Fix this by closing
the pack file only after loosen() finished.
Bug: 574178
Change-Id: Icd59931a218d84c9c97b450eea87b21ed01248ff
Signed-off-by: andrew.xian2000@gmail.com
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The loosen() method has opened pack file and the open pack file handle
may prevent it from being deleted e.g. on Windows. Fix this by closing
the pack file only after loosen() finished.
Bug: 574178
Change-Id: Icd59931a218d84c9c97b450eea87b21ed01248ff
Signed-off-by: andrew.xian2000@gmail.com
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
On a subtree, a PathSuffixFilter must return -1 ("indeterminate"),
not 0 ("include"), otherwise negation goes wrong: an indeterminate
result (-1) is passed on, but a decision (0/1) is inverted.
As a result a negated PathSuffixFilter would skip all folders.
Bug: 574253
Change-Id: I27fe785c0d772392a5b5efe0a7b1c9cafcb6e566
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
That commit was submitted on master between the 5.12.0 release and
the 5.13.0 version bump.
Change-Id: I679e818bfc5a4695b66548add9a83a22c89a4ffc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
From Git commit 9c1e657a8f:
Currently, the packfile negotiation step within a Git fetch cannot be
done independent of sending the packfile, even though there is at
least one application wherein this is useful - push negotiation.
Therefore, make it possible for this negotiation step to be done
independently.
This feature is for protocol v2 only.
In the protocol, the main hindrance towards independent negotiation is
that the server can unilaterally decide to send the packfile. This is
solved by a "wait-for-done" argument: the server will then wait for
the client to say "done". In practice, the client will never say it;
instead it will cease requests once it is satisfied.
Advertising the server capability option "wait-for-done" is behind the
transport config: uploadpack.advertisewaitfordone, which by default is
false.
Change-Id: I5ebd3e99ad76b8943597216e23ced2ed38eb5224
HostEntry class was public with empty constructor, so adding
constructors with default visibility actually reduced visibility of
constructor.
Change-Id: I4c996c0559102084946ba49a71afe10dda5e0f95
This is similar to change Idbc2c29bd that skipped detecting content
renames for large files. With this change, we added a new option in
RenameDetector called "skipContentRenamesForBinaryFiles", that when set,
causes binary files with any slight modification to be identified as
added/deleted. The default for this boolean is false, so preserving
current behaviour.
Change-Id: I4770b1f69c60b1037025ddd0940ba86df6047299
When the target repository is receiving commits from other sources,
the repo command commit can fail with a LOCK_FAILURE. We could let
callers retry, but then the command needs to redo all the work (opening
all subrepos to recreate the tree).
Retry the commit in LOCK_FAILURE inside the command. The commit
rewrites the whole tree, so it shouldn't have merge errors. Use an
exponential delay with jitter for the retries.
Change-Id: I517b6f2afd16a4b695e6cf471b5d6cf492024ec4
Signed-off-by: Ivan Frade <ifrade@google.com>
The "branch" field in the .gitmodules is the signal for gerrit to keep
the superproject autoupdated. Tags are immutable and there is no need to
track them, plus the cgit client requires the field to be a "remote
branch name" but not a tag.
Do not set the "branch" field if the revision is a tag. Keep those tags
in another field ("ref") as they help other tools to find the commit in
the destination repository.
We can still have false negatives when a refname is not fully qualified,
but this check covers e.g. the most common case in android.
Note that the javadoc of #setRecordRemoteBranch already mentions that
"submodules that request a tag will not have branch name recorded".
Change-Id: Ib1c321a4d3b7f8d51ca2ea204f72dc0cfed50c37
Signed-off-by: Ivan Frade <ifrade@google.com>
Check the last line of the last hunk of a file, not the last line of
the whole patch.
Note that C git only checks that this line starts with "\ " and is at
least 12 characters long because of possible different texts when non-
English messages are used.
Change-Id: I0db81699eb3e99ed7b536a3e2b8dc97df1f58a89
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
C git treats completely empty lines as empty context lines (which
traditionally have a single blank). Apparently newer GNU diff may
produce such lines; see [1]. ("Newer" meaning "since 2006"...)
[1] https://github.com/git/git/commit/b507b465f7831
Change-Id: I80c1f030edb17a46289b1dabf11a2648d2660d38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Instead of converting the patch bytes to strings apply the patch on
byte level, like C git does. Converting the input lines and the hunk
lines from bytes to strings and then applying the patch based on
strings may give surprising results if a patch converts a text file
from one encoding to another. Moreover, in the end we don't know which
encoding to use to write the result.
Previous code just wrote the result as UTF-8, which forcibly changed
the encoding if the original input had some other encoding (even if the
patch had the same non-UTF-8 encoding). It was also wrong if the input
was UTF-8, and the patch should have changed the encoding to something
else.
So use ByteBuffers instead of Strings. This has the additional advantage
that all these ByteBuffers can share the underlying byte arrays of the
input and of the patch, so it also reduces memory consumption.
Change-Id: I450975f2ba0e7d0bec8973e3113cc2e7aea187ee
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Implement applying binary patches. Handles both literal and delta
patches. Note that C git also runs binary files through the clean
and smudge filters. Implement the same safeguards against corrupted
patches as in C git: require the full OIDs to be present in the patch
file, and apply a binary patch only if both pre- and post-image hashes
match.
Add tests for applying literal and delta patches.
Bug: 371725
Change-Id: I71dc214fe4145d7cc8e4769384fb78c7d0d6c220
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new BinaryDeltaInputStream that applies a delta provided by
another InputStream to a given base. Because delta application needs
random access to the base, the base itself cannot be yet another
InputStream. But at least this enables streaming of the result.
Add a simple test using delta hunks generated by C git.
Bug: 371725
Change-Id: Ibd26fa2f49860737ad5c5387f7f4870d3e85e628
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add streams that can encode or decode git binary patch data on the fly.
Git writes binary patches base-85 encoded, at most 52 un-encoded bytes,
with the unencoded data length prefixed in a one-character encoding, and
suffixed with a newline character.
Add a test for both the new input and the output stream. The test
roundtrips binary data of different lengths in different ways.
Bug: 371725
Change-Id: Ic3faebaa4637520f5448b3d1acd78d5aaab3907a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add an implementation for base-85 encoding and decoding [1]. Git binary
patches use this format.
Base-85 encoding assembles bytes as 32-bit MSB values, then converts
these values to base-85 numbers (always 5 bytes) encoded as printable
ASCII characters. Decoding base-85 is the reverse operation. Note
that decoding may overflow on invalid input as 85^5 > 2^32. Encodings
always have a length that is a multiple of 5. If input length is not
divisible by 4, padding bytes are (logically) added, which are ignored
when decoding. The encoding for n bytes has thus always exactly length
(n + 3) / 4 * 5 in integer arithmetic (truncating division).
Includes tests.
[1] https://datatracker.ietf.org/doc/html/rfc1924
Bug: 371725
Change-Id: Ib5b9a503cd62cf70e080a4fb38c8cd1eeeaebcfe
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Rather than getting all ref names and prefixes and saving them
in memory to perform the check for conflicting names, rely on
RefDirectory.isNameConflicting as it is no longer an expensive
call after it was optimized in Ie994fc.
The old optimization to save ref names and prefixes in memory
was targeted towards making clones faster. With this change,
the clone performance is unaffected when tests were done with
repos containing many(~500k) refs.
Here are few recorded elapsed times for creating 10 branches
using BatchRefUpdate on NFS based repositories with varying
loose refs count. As seen here, this change helps improve the
BatchRefUpdate performance from O(n^2) to O(1).
loose_refs_count with_change without_change
50 241 ms 310 ms
300 263 ms 1502 ms
1k 181 ms 4241 ms
2k 204 ms 6440 ms
9k 158 ms 25930 ms
20k 154 ms 60443 ms
50k 171 ms 135199 ms
110k 157 ms 329450 ms
160k 209 ms 396328 ms
This update improves the Gerrit notedb migration performance
as it uses BatchRefUpdate to write change meta refs similar to
the test performed above.
Change-Id: I853ac6c7feb4b39c3156c01876b38cbd182accfe
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Avoid having to scan over ALL loose refs to determine if the
name is nested within or is a container of an existing reference.
This can get really expensive if there are too many loose refs.
Instead use exactRef and getRefsByPrefix which scan based on a
prefix.
With a simple shell script(like below) using jgit client to create
1k refs in a new repository on NFS, this change brings down the time
from 12mins to 7mins.
for ref in $(seq 1 1000); do
jgit branch "$ref"
done
Here are few recorded elapsed times to create a new branch on NFS
based repositories with varying loose refs count. As we see here,
this change improves the name conflicting check from O(n^2) to O(1).
loose_refs_count with_change without_change
50 44 ms 164 ms
300 45 ms 1193 ms
1k 38 ms 2610 ms
2k 44 ms 6003 ms
9k 46 ms 27860 ms
20k 45 ms 48591 ms
50k 51 ms 135471 ms
110k 43 ms 294252 ms
160k 52 ms 430976 ms
Change-Id: Ie994fc184b8f82811bfb37b111eb9733dbe3e6e0
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
Applying a patch on Windows failed if the patch had the (normal)
single-LF line endings, but the file on disk had the usual Windows
CR-LF line endings.
Git (and JGit) compute diffs on the git-internal blob, i.e., after
CR-LF transformation and clean filtering. Applying patches to files
directly is thus incorrect and may fail if CR-LF settings don't
match, or if clean/smudge filtering is involved.
Change ApplyCommand to run the file content through the check-in
filters before applying the patch, and run the result through the
check-out filters. This makes patch application succeed even if the
patch has single-LFs, but the file has CR-LF and core.autocrlf is
true.
Add tests for various combinations of line endings in the file and in
the patch, and a test to verify the clean/smudge handling.
See also [1].
Running the file though clean/smudge may give strange results with
LFS-managed files. JGit's DiffFormatter has some extra code and
applies the smudge filter again after having run the file through
the check-in filters (CR-LF and clean). So JGit can actually produce
a diff on LFS-managed files using the normal diff machinery. (If it
doesn't run out of memory, that is. After all, LFS is intended for
_large_ files.) How such a diff would be applied with either C git
or JGit is entirely unclear; neither has any code for this special
case. Compare also [2].
Note that C git just doesn't know about LFS and always diffs after
the check-in filter chain, so for LFS files, it'll produce a diff
of the LFS pointers.
[1] https://github.com/git/git/commit/c24f3abac
[2] https://github.com/git-lfs/git-lfs/issues/440
Bug: 571585
Change-Id: I8f71ff26313b5773ff1da612b0938ad2f18751f5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Use Character.isWhitespace() instead of Character.isSpaceChar() to
treat TABs as whitespace, too.
Change-Id: Iffc59c13357d981ede6a1e0feb6ea6ff03fb3064
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Negated patterns were handled wrongly. According to the OpenBSD
ssh_config man page,[1] a negated pattern never matches. Negated
patterns make only sense if there are positive patterns; the
negated pattern then can define exceptions for the positive
patterns.
OpenSshConfigFile did this wrongly. It handled "!foo" as "matching
everything but foo", but actually the semantics is "if the input is
"foo", this entry doesn't apply. If the input is anything else,
other patterns determine whether the entry may apply.".
[1] https://man.openbsd.org/ssh_config
Change-Id: I50f6e46581b7ece4c949eddf62f4a265573ec29e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* master: (34 commits)
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
LockFile: create OutputStream only when needed
Add a cgit interoperability test for LockFile
Add TemporaryBuffer.toString(int limit)
LockFile: create OutputStream only when needed
Prepare 5.12.0-SNAPSHOT builds
JGit v5.12.0.202105051250-m2
Update jetty to 9.4.40.v20210413
[releng] Update eclipse-jarsigner-plugin to 1.3.1
Implement ours/theirs content conflict resolution
ssh: ensure list is modifiable before using Iterator.remove().
Update orbit to S20210406213021 and add 4.20-staging target
Fix typo in test method name
Allow file mode conflicts in virtual base commit on recursive merge.
sshd: don't lock the known_hosts files on reading
Allow info messages in UsernamePasswordCredentialsProvider
ssh config: do environment variable replacement
sshd: implement server-sig-algs SSH extension (client side)
Upgrade ecj to 3.25.0
...
Change-Id: Ibc39a9c4e431d15b67ab4a307241f47a7f3740a9
* stable-5.12:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I1c936183e1fa17ea95ada7849a75bc76af275fa3
* stable-5.11:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I6184772bdeca1b9ccecf6e400ae15604ab4f5a69
* stable-5.10:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I0f1511be5375716d41565e72b271cb956c3e847b
* stable-5.9:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: Ifa135077d8d07d2317df3b479822e30d87eca950
* stable-5.8:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I9ca7a0237f87d1d4bcaba81e709eaa67902f27e5
* stable-5.7:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I7ed3f47cb46e6c1bf483702c8925a24e88658e47
* stable-5.6:
Remove texts which were added by mistake in 00386272
Fix formatting which was broken in 00386272
Change-Id: I45d444b360485564744bf3dfad2c2f5a5e7fcdf6
* stable-5.9:
LockFile: create OutputStream only when needed
Remove ReftableNumbersNotIncreasingException
Fix stamping to produce stable file timestamps
Change-Id: I056382d1d93f3e0a95838bdd1f0be89711c8a722
Don't create the stream eagerly in lock(); that may cause JGit to
exceed OS or JVM limits on open file descriptors if many locks need
to be created, for instance when creating many refs. Instead create
the output stream only when one really needs to write something.
Bug: 573328
Change-Id: If9441ed40494d46f594a896d34a5c4f56f91ebf4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Don't create the stream eagerly in lock(); that may cause JGit to
exceed OS or JVM limits on open file descriptors if many locks need
to be created, for instance when creating many refs. Instead create
the output stream only when one really needs to write something.
Bug: 573328
Change-Id: If9441ed40494d46f594a896d34a5c4f56f91ebf4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a constant in ConfigConstants, and a ConflictStyle enum in
MergeCommand.
Change-Id: Idf8e036b6b6953bec06d6923a39e5ff30c2da562
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Git has different conflict resolution strategies:
* There is a tree merge strategy "ours" which just ignores any changes
from theirs ("-s ours"). JGit also has the mirror strategy "theirs"
ignoring any changes from "ours". (This doesn't exist in C git.)
Adapt StashApplyCommand and CherrypickCommand to be able to use those
tree merge strategies.
* For the resolve/recursive tree merge strategies, there are content
conflict resolution strategies "ours" and "theirs", which resolve
any conflict hunks by taking the "ours" or "theirs" hunk. In C git
those correspond to "-Xours" or -Xtheirs". Implement that in
MergeAlgorithm, and add API to set and pass through such a strategy
for resolving content conflicts.
* The "ours/theirs" content conflict resolution strategies also apply
for binary files. Handle these cases in ResolveMerger.
Note that the content conflict resolution strategies ("-X ours/theirs")
do _not_ apply to modify/delete or delete/modify conflicts. Such
conflicts are always reported as conflicts by C git. They do apply,
however, if one side completely clears a file's content.
Bug: 501111
Change-Id: I2c9c170c61c440a2ab9c387991e7a0c3ab960e07
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Similar to https://git.eclipse.org/r/c/jgit/jgit/+/175166, ignore
path that have conflicts on attributes, so that the virtual base could
be used by RecursiveMerger.
Change-Id: I99c95445a305558d55bbb9c9e97446caaf61c154
Signed-off-by: Marija Savtchouk <mariasavtchouk@google.com>
o.e.j.ssh.apache produces passphrase prompts containing
InformationalMessage items to show the fingerprint of the key
the passphrase is being asked for. Allow this so that the credentials
provider can be used with o.e.j.ssh.apache.
Change-Id: Ibc2ffd3a987d3118952726091b9b80442972dfd8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
OpenSSH 8.4 has introduced simple environment variable substitution
for some keys. Implement that feature in our ssh config file parser,
too.
Bug: 572103
Change-Id: I360f2c5510eea4ec3329aeedf3d29dfefc9163f0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.11:
Refactor CommitCommand to improve readability
CommitCommand: fix formatting
CommitCommand: remove unncessary comment
Ensure post-commit hook is called after index lock was released
sshd: try all configured signature algorithms for a key
sshd: modernize ssh config file parsing
sshd: implement ssh config PubkeyAcceptedAlgorithms
Change-Id: Ic3235ffd84c9d7537a1fe5ff4f216578e6e26724
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
OpenSSH has changed some things in ssh config files. Update our parser
to implement some of these changes:
* ignore trailing comments on a line
* rename PubkeyAcceptedKeyTypes to PubkeyAcceptedAlgorithms
Note that for the rename, openSSH still accepts both names. We do the
same, translating names whenever we get or set values.
Change-Id: Icccca060e6a4350a7acf05ff9e260f2c8c60ee1a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Apache MINA sshd 2.6.0 appears to use only the first appropriate
public key signature algorithm for a particular key. See [1]. For
RSA keys, that is rsa-sha2-512. This breaks authentication at servers
that only know the older (and deprecated) ssh-rsa algorithm.
With PubkeyAcceptedAlgorithms, users can re-order algorithms in
the ssh config file per host, if needed. Setting
PubkeyAcceptedAlgorithms ^ssh-rsa
will put "ssh-rsa" at the front of the list of algorithms, and then
authentication at such servers with RSA keys works again.
[1] https://issues.apache.org/jira/browse/SSHD-1105
Bug: 572056
Change-Id: I86c3b93f05960c68936e80642965815926bb2532
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
By returning `this` at the end of the `setCredentialsProvider()` the API
can be used as a fluent style.
This change is source compatible but not binary compatible with existing
clients, and so adding this will require a major version change.
Bug: 553116
Change-Id: I72eaefee72825fa2246319a94a0df3c0cb7061fc
Signed-off-by: Alex Blewitt <alex.blewitt@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In [1], improved RevWalk.getMergedInto() is introduced to avoid repeated
work while performing RevWalk.isMergedInto() on many refs. Modify
findBranchesReachableFrom() to use it.
[1] I65de9873dce67af9c415d1d236bf52d31b67e8fe
Change-Id: I81d615241638d4093df64b449637af601843a5ed
Signed-off-by: Adithya Chakilam <quic_achakila@quicinc.com>
In cases where we need to determine if a given commit is merged
into many refs, using isMergedInto(base, tip) for each ref would
cause multiple unwanted walks.
getMergedInto() marks the unreachable commits as uninteresting
which would then avoid walking that same path again.
Using the same api, also introduce isMergedIntoAny() and
isMergedIntoAll()
Change-Id: I65de9873dce67af9c415d1d236bf52d31b67e8fe
Signed-off-by: Adithya Chakilam <quic_achakila@quicinc.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There are two code paths for detecting renames: one on tree diffs
(using DiffFormatter#scan) and the other on single file diffs (using
DiffFormatter#format). The latter skips binary and large files
for rename detection - check [1], but the former doesn't.
This change skips content rename detection for the tree diffs case for
large files. This is essential to avoid expensive computations while
reading the file, especially for callers who don't want to pay that
cost. Content renames are those which involve files with slightly
modified content. Exact renames will still be identified.
The default threshold for file sizes is reused from
PackConfig.DEFAULT_BIG_FILE_THRESHOLD: 50 MB.
[1] 232876421d/org.eclipse.jgit/src/org/eclipse/jgit/diff/RawText.java (386)
Change-Id: Idbc2c29bd381c6e387185204638f76fda47df41e
Signed-off-by: Youssef Elghareeb <ghareeb@google.com>
Git config http.cookieFile must have ~ expansion, compare [1].
It also should be an absolute path. While a relative path is allowed,
C git just passes the value on to libcurl, so it'll be relative to the
current working directory and thus not work in all directories.
Log a warning if the path is relative.
(Alternatives would be to throw an exception, or to resolve the path
relative to the .git directory, or relative to the working tree root,
or relative to the config file it occurs in. But C git does not seem
to do either.)
[1] https://github.com/git/git/commit/e5a39ad8e
Bug: 571798
Change-Id: I5cdab6061d0613ac7d8cb7977e5b97f5b88f562d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add new constructors to PackFile to improve a common use case where
callers know the directory, id, and extension, but previously needed to
construct a valid file name (with prefix, '.', etc) to create a
PackFile. Most callers can use the variant that has id as an ObjectId,
but provide an id as String variant too.
Change-Id: I39e4466abe8c9509f5916d5bfe675066570b8585
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
It's easier to follow the logic here when we can use our own objects
instead of Strings.
Change-Id: I6a166edcc67903fc1ca3544f458634c4cef8fde7
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
This class already looked very much like an Enum, but wasn't one.
As an Enum, we can use PackExt in EnumMaps and EnumSets. Convert the
Map key usage in PackDirectory to an EnumMap.
Change-Id: Ice097fd468a05805f914e6862fbd1d96ec8c45d1
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
Provide a recovery path for objects being referenced during the pack
pruning race. Due to the pack pruning race, it is possible for objects
to become referenced after a pack has been deemed safe to prune, but
before it actually gets pruned. If this happened previously, the newly
referenced objects would be missing and potentially result in a
corrupted ref.
Add the ability to recover from this situation when an object is missing
but happens to still be available in a pack in the "preserved"
directory. This is likely only useful when used in conjunction with the
--preserve-old-packs GC option, which prunes packs by hard-linking to
the preserved directory. If an object is missing and found in a pack in
the preserved directory, immediately recover that pack and its
associated files (idx, bitmaps...) by moving them back to the original
pack directory, and then retry the operation that would have failed due
to the missing object. This retry can now succeed and the repository
may avoid corruption. This approach should drastically reduce the
chance of a corrupt repository during pack pruning at very little extra
cost. This extra cost should only be incurred when objects are missing
and a failure would normally occur.
Change-Id: I2a704e3276b88cc892159d9bfe2455c6eec64252
Signed-off-by: Martin Fick <quic_mfick@quicinc.com>
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
The only extension that was ever consulted from the bitmap was the
bitmap index. We can simplify the Pack code as well as the code of
all the callers if we focus on just that usage.
Change-Id: I799ddfdee93142af67ce5081d14a430d36aa4c15
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
Update scanPacksImpl and listPackDirectory (renamed to
getPackFilesByExtById) to use the new PackFile functionality to
validate file names and complete pack file sets (.pack, .idx, etc).
Most importantly, this allows a later change to rely on scanPacks() to
complete a packList that contains packs with the 'old-' prefix in their
extension.
This also eliminates duplication of logic for how to identify and
construct pack files.
Change-Id: I7175e5fefb187a29e0a7cf53c392aee922314f31
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
GC has several places where it tries to build files names for packs that
we can use the PackFile class for instead.
Change-Id: I99e5ceff9050f8583368fca35279251955e4644d
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
The PackFile class is intended to be a central place to do all
common pack filename manipulation and parsing to help reduce repeated
code and bugs. Use the PackFile class in the Pack class and in many
tests to ensure it works well in a variety of situations. Later changes
will expand use of PackFiles to even more areas.
Change-Id: I921b30f865759162bae46ddd2c6d669de06add4a
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A cookie file stores the expiration in seconds since the Linux Epoch,
not in milliseconds. Correct reading and writing cookie files; with
a backwards-compatibility hack to read files that contain a millisecond
timestamp.
Add a test, and fix tests not to rely on the actual current time so
that they will also run successfully after 2030-01-01 noon.
Bug: 571574
Change-Id: If3ba68391e574520701cdee119544eedc42a1ff2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In a distributed setting, one can have multiple datacenters use
reftables for serving, while the ground truth for the Ref database is
administered centrally. In this setting, replication delays combined
with compaction can cause update-index ranges to overlap.
Such a setting is used at Google, and the JGit code already handles
this correctly (modulo a bugfix that applied in change I8f8215b99a).
Remove the restriction that was applied at FileReftableDatabase.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I6f9ed0fbd7fbc5220083ab808b22a909215f13a9
Include the full file path of the .gitignore file and the line number
of the invalid pattern. Also include the pattern itself.
.gitignore files inside the repository are reported with their
repository-relative path; files outside (from git config
core.excludesFile or .git/info/exclude) are reported with their
full absolute path.
Bug: 571143
Change-Id: Ibe5969679bc22cff923c62e3ab9801d90d6d06d1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When a .gitignore pattern cannot be parsed include the pattern in the
log message. Just reporting "not closed bracket" isn't helpful if the
user doesn't know in which pattern the problem occurred.
Even better would be to include the full path of the .gitignore file
that contained the offending pattern. This is not implemented in this
change; it may need new API and needs more thought.
Bug: 571143
Change-Id: Id5b16d9cf550544ba3ad409a02041946fa8516ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We introduced the option --initial-branch=<branch-name> to allow
initializing a new repository with a different initial branch.
To allow users to override the initial branch name more permanently
(i.e. without having to specify the name manually for each 'git init'),
introduce the 'init.defaultBranch' option.
This option was added to git in 2.28.0.
See https://git-scm.com/docs/git-config#Documentation/git-config.txt-initdefaultBranch
Bug: 564794
Change-Id: I679b14057a54cd3d19e44460c4a5bd3a368ec848
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add option --initial-branch/-b to InitCommand and the CLI init command.
This is the first step to implement support for the new option
init.defaultBranch. Both were added to git in release 2.28.
See https://git-scm.com/docs/git-init#Documentation/git-init.txt--bltbranch-namegt
Bug: 564794
Change-Id: Ia383b3f90b5549db80f99b2310450a7faf6bce4c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
jgit clone --branch foo <url>
did not fail if the remote branch "foo" didn't exist in the remote
repository being cloned.
Bug: 546580
Change-Id: I55648ad3a39da4a5711dfa8e6d6682bb8190a6d6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
`copy` is documented as possibly returning a smaller number of bytes
than requested. In practice, this can occur if a block is cached and the
reader never pulls in the file to check its size.
Bug: 565874
Change-Id: I1e53b3d2f4ab09334178934dc0ef74ea99045cd3
Signed-off-by: wh <wh9692@protonmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add a GpgSignatureVerifier interface, plus a factory to create
instances thereof that is provided via the ServiceLoader mechanism.
Implement the new interface for BouncyCastle. A verifier maintains
an internal LRU cache of previously found public keys to speed up
verifying multiple objects (tag or commits). Mergetags are not handled.
Provide a new VerifySignatureCommand in org.eclipse.jgit.api together
with a factory method Git.verifySignature(). The command can verify
signatures on tags or commits, and can be limited to accept only tags
or commits. Provide a new public WrongObjectTypeException thrown when
the command is limited to either tags or commits and a name resolves
to some other object kind.
In jgit.pgm, implement "git tag -v", "git log --show-signature", and
"git show --show-signature". The output is similar to command-line
gpg invoked via git, but not identical. In particular, lines are not
prefixed by "gpg:" but by "bc:".
Trust levels for public keys are read from the keys' trust packets,
not from GPG's internal trust database. A trust packet may or may
not be set. Command-line GPG produces more warning lines depending
on the trust level, warning about keys with a trust level below
"full".
There are no unit tests because JGit still doesn't have any setup to
do signing unit tests; this would require at least a faked .gpg
directory with pre-created key rings and keys, and a way to make the
BouncyCastle classes use that directory instead of the default. See
bug 547538 and also bug 544847.
Tested manually with a small test repository containing signed and
unsigned commits and tags, with signatures made with different keys
and made by command-line git using GPG 2.2.25 and by JGit using
BouncyCastle 1.65.
Bug: 547751
Change-Id: If7e34aeed6ca6636a92bf774d893d98f6d459181
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
As the post commit hook is run after a commit is finished, it can not
abort the commit and the exit code of this hook should not have any
effect.
This can be achieved by not throwing a AbortedByHookException exception.
The stderr output is not lost thanks to contributions for bug 553471.
Bug: 553428
Change-Id: I451a76e04103e632ff44e045561c5a41f7b7d558
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
Signed-off-by: Fabian Pfaff <fabian.pfaff@vogella.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
EGit wants to add gitflow specific hooks in org.eclipse.egit.gitflow.
Make GitHook public to allow sub-classing outside of the
org.eclipse.jgit.hooks package.
Change-Id: I439575ec901e3610b5cf9d66f7641c8324faa865
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
External scripts most probably expect the default charset.
Change-Id: I318a5e1d9f536a95e70c06ffb5b6f408cd40f73a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Pack better represents the purpose of the object and paves the way to
add a PackFile object that extends File.
Change-Id: I39b4f697902d395e9b6df5e8ce53078ce72fcea3
Signed-off-by: Nasser Grainawi <quic_nasserg@quicinc.com>
If RecursiveMerger finds multiple base commits, it tries to compute
the virtual ancestor to use as a base for the three way merge.
Currently, the content conflicts between ancestors are ignored (file
staged with the conflict markers). If the path is a file in one ancestor
and a dir in the other, it results in NoMergeBaseException
(CONFLICTS_DURING_MERGE_BASE_CALCULATION).
Allow these conflicts by ignoring this unmerged path in the virtual
base. The merger will compute diff in the children instead and it
can be further fixed manually if needed.
Change-Id: Id59648ae1d6bdf300b26fff513c3204317b755ab
Signed-off-by: Marija Savtchouk <mariasavtchouk@google.com>
Subclasses can use the corresponding getter methods.
Change-Id: Iaa9ab01f5a9731a264b28608d2418a9405b601d7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add it to the GpgConfig. Change GpgConfig to load the values once only.
Add a parameter to the GpgObjectSigner interface's operations to pass
in a GpgConfig. Update CommitCommand and TagCommand to pass the value
to the signer. Let the signer decide whether it can actually produce
the wanted signature type (openpgp or x509).
No behavior change. But this makes it possible to implement different
signers that might support x509 signatures, or use gpg.program and
shell out to an external GPG executable for signing.
Change-Id: I427f83eb1ece81c310e1cddd85315f6f88cc99ea
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
DateRevQueue is expected to give out the commits that have higher
commit time. But in case of tie(same commit time), it should give
the commit that is inserted first. This is inferred from the
testInsertTie test case written for DateRevQueue. Also that test
case, right now uses just two commits which caused it not to fail
with the current implementation, so added another commit to make
the test more robust.
By fixing the DateRevQueue, we would also match the behaviour of
LogCommand.addRange(c1,c2) with git log c1..c2. A test case for
the same is added to show that current behaviour is not the
expected one.
By fixing addRange(), the order in which commits are applied during
a rebase is altered. Rebase logic should have never depended upon
LogCommand.addRange() since the intended order of addRange() is not
the order a rebase should use. So, modify the RebaseCommand to use
RevWalk directly with TopoNonIntermixSortGenerator.
Add a new LogCommandTest.addRangeWithMerge() test case which creates
commits in the following order:
A - B - C - M
\ /
-D-
Using git 2.30.0, git log B..M outputs: M C D
LogCommand.addRange(B, M) without this fix outputs: M D C
LogCommand.addRange(B, M) with this fix outputs: M C D
Change-Id: I30cc3ba6c97f0960f64e9e021df96ff276f63db7
Signed-off-by: Adithya Chakilam <achakila@codeaurora.org>
Keeping the field updateDate is unecessary, as it is set and used only
in the doRename method.
Change-Id: I1cdd1adf759b75c103480db7a74cec8c2d78b794
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
Deleting non-existing files when converting to reftable without backup
caused convertToReftable to fail. Observed this on a mirrored repository
which had no reflogs. Fix this by skipping missing files during
deletion.
Change-Id: I3bb913d5bfddccc6813677b873006efb849a6ebc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
before connecting.
A socket gets bound on connect in the next line.
Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I69a423c592e2fdd582b3c40099137b4ef3d05b39
This would run into an endless loop if the offset given was not zero.
Fix the logic to exit the read loop when the buffer is full.
Luckily all existing uses of this method call it only with offset zero.
Change-Id: I0ec2a4fb43efe4a605d06ac2e88cf155d50e2f1e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reachability checkers are retrieved from RevWalk and ObjectWalk objects:
* RevWalk.createReachabilityChecker()
* ObjectWalk.createObjectReachabilityChecker()
Since RevWalks and ObjectWalks are themselves directly instantiated
in hundreds of places (e.g. UploadPack...) overriding them in a
consistent way requires overloading 100s of methods, which isn't
feasible. Moving reachability checker generation to a more central
place solves that problem.
The ObjectReader object seems a good place from which to get
reachability checkers, because reachability checkers return
information about relationships between objects. ObjectDatabases
delegate many operations to ObjectReaders, and reachability bitmaps
are attached to ObjectReaders.
The Bitmapped and Pedestrian reachability checker objects were
package private in the org.eclipse.jgit.revwalk package. This change
makes them public and moves them to the
org.eclipse.jgit.internal.revwalk package. Corresponding tests are
also moved.
Motivation:
1) Reachability checking algorithms need to scale. One of the
internal Android repositories has ~2.4 million refs/changes/*
references, causing bad long tail performance in reachability
checks.
2) Reachability check performance is impacted by repository
topography: number of refs, number of objects, amounts of
related vs. unrelated history.
3) Reachability check performance is also affected by per-branch
access (Gerrit branch permissions) since different users can
see different branches.
4) Reachability check performance isn't affected by any state in a
RevWalk or ObjectWalk.
I don't yet know if a single algorithm will work for all cases in #2
and #3. We may need to evolve the ReachabilityChecker interfaces
over time to solve the Gerrit branch permissions case, or use
Gerrit-specific identity information to solve that in an efficient
way.
This change takes the existing public API and moves it to the
ObjectReader/whole repository level, which is where we can do
consistent customizations for #2 and #3. We intend to upstream the
best of whatever works, but anticipate the need for multiple rounds
of experimentation.
Change-Id: I9185feff43551fb387957c436112d5250486833d
Signed-off-by: Terry Parker <tparker@google.com>
* changes:
Compare getting all refs except specific refs with seek and with filter
Add getsRefsByPrefixWithSkips (excluding prefixes) to ReftableDatabase
Add seekPastPrefix method to RefCursor
We sometimes want to get all the refs except specific prefixes,
similarly to getRefsByPrefix that gets all the refs of a specific
prefix.
We now create a new method that gets all refs matching a prefix except a
set of specific prefixes.
One use-case is for Gerrit to be able to get all the refs except
refs/changes; in Gerrit we often have lots of refs/changes, but very
little other refs. Currently, to get all the refs except refs/changes we
need to get all the refs and then filter the refs/changes, which is very
inefficient. With this method, we can simply skip the unneeded prefix so
that we don't have to go over all the elements.
RefDirectory still uses the inefficient implementation, since there
isn't a simple way to use Refcursor to achieve the efficient
implementation (as done in ReftableDatabase).
Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I8c5db581acdeb6698e3d3a2abde8da32f70c854c
Adds a new FileUtils.hasFiles(Path) helper method to correctly handle
the Files.list returned Stream.
These errors were found by compiling the code using JDK11's
javac compiler.
Change-Id: Ie8017fa54eb56afc2e939a2988d8b2c5032cd00f
Signed-off-by: Terry Parker <tparker@google.com>
This method will be used by the follow-up change. This useful if we want
to go over all the changes after a specific ref.
For example, the new method allows us to create a follow-up that would
go over all the refs until we reach a specific ref (e.g refs/changes/),
and then we use seekPastPrefix(refs/changes/) to read the rest of the refs,
thus basically we return all refs except a specific prefix.
When seeking past a prefix, the previous condition that created the
RefCursor still applies. E.g, if the cursor was created by
seekRefsWithPrefix, we can skip some refs but we will not return refs
that are not starting with this prefix.
Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I2c02e89c877fe90da8619cb8a4a9a0c865f238ef
In some circumstances (eg. compacting a stack that has deletions), the
result may have a {min, max} range that already exists. In these
cases, we would rename onto an already existing file, which does not
work on Windows. By adding a random suffix, we disambiguate the files,
and avoid this failure scenario.
Change-Id: I0273f99bb845cfbdbd8cdd582b55d3c310505d29
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Heap always copied whole blocks, which leads to AIOOBEs. LocalFile
didn't overwrite the method and thus caused NPEs.
Change-Id: Ia37d4a875df9f25d4825e6bc95fed7f0dff42afb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.
Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.
Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.
No preemptive authentication will be done if the connection is
redirected to a different host.
Add tests.
Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
TransportHttp makes several HTTP requests. The SSLContext and socket
factory must be shared over these requests, otherwise authentication
information may not be propagated correctly from one request to the
next. This is important for authentication mechanisms that rely on
client-side state, like NEGOTIATE (either NTLM, if the underlying HTTP
library supports it, or Kerberos). In particular, SPNEGO cannot
authenticate on a POST request; the authentication must come from the
initial GET request, which implies that the POST request must use the
same SSLContext and socket factory that was used for the GET.
Change the way HTTPS connections are configured. Introduce the concept
of a GitSession, which is a client-side HTTP session over several HTTPS
requests. TransportHttp creates such a session and uses it to configure
all HTTP requests during that session (fetch or push). This gives a way
to abstract away the differences between JDK and Apache HTTP connections
and to configure SSL setup outside.
A GitSession can maintain state and thus give all HTTP requests in a
session the same socket factory.
Introduce an extension interface HttpConnectionFactory2 that adds a
method to obtain a new GitSession. Implement this for both existing
HTTP connection factories. Change TransportHttp to use the new
GitSession to configure HTTP connections.
The old methods for disabling SSL verification still exist to support
possibly external connection and connection factory implementations
that do not make use of the new GitSession yet.
Bug: 535850
Change-Id: Iedf67464e4e353c1883447c13c86b5a838e678f1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.
Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.
Make the global static factory reference volatile.
Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
UploadPack may log ACKs in protocol V2 that it doesn't send (if it
got a "done" from the client), or may log ACKs twice. That makes
packet log analysis difficult.
Add a new constructor to PacketLineOut to omit all logging from an
instance, and use it in UploadPack.
Change-Id: Ic29ef5f9a05cbcf5f4858a4e1b206ef0e6421c65
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This fixes errorprone error [SynchronizeOnNonFinalField]: Synchronizing
on non-final fields is not safe: if the field is ever updated, different
threads may end up locking on different objects.
Change-Id: I42fe5bde825151693e2da2d5b6cd6e1d34038dbc
File#getParent can return null which caused this spotbugs warning.
FS.FileStoreAttributes#get already gets the parent directory if the
passed File is not a directory and checks for null. Hence there is no
need to get the parent directory in the FileSnapshot constructor.
Change-Id: I77f71503cffb05970ab8d9ba55b69c96c53098b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If file doesn't exist set state to MISSING_FILE immediately. Doing that
by calling File#lastModified and File#length effectively does the same
since they set the value to 0 if the file doesn't exist.
Log an error if a different exception than NoSuchFileException is
caught.
Change-Id: I0d4396b9f80446692a088d17522d64f735ce6708
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.
Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
On Android, the co-variant override of ConcurrentHashMap.keySet()
introduced in Java 8 was undone. [1] If compiled Java code calls that
co-variant override directly, one gets a NoSuchMethodError exception
at run-time on Android.
Making the code call that method via Map.keySet() side-steps this
problem.
This is similar to bug 496262, where the same problem cropped up when
compiling with Java 8 against a Java 7 target, but here we cannot use
bootclasspath. We build against Java 8, not against the Android version
of it.
Recent Android versions should have some bytecode "magic" that adds the
co-variant override in bytecode (see the commit referenced in [1]), but
on older Android version this problem may still occur. (Or perhaps the
"magic" is ineffective...) There are two pull requests on Github for
this problem, both from 2020, [2][3] while the Android commit [1] is
from March 2018. Apparently people still occasionally run into this
problem in the wild.
[1] 0e8b937ded/ojluni/src/main/java/java/util/concurrent/ConcurrentHashMap.java (1244)
[2] https://github.com/eclipse/jgit/pull/104
[3] https://github.com/eclipse/jgit/pull/100
Change-Id: I7c07e0cc59871cb7fe60795e22867827fa9c2458
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If a file exists in head, merge, and the working tree, but not in
the index, and we're doing a force checkout, the checkout must be
an "update", not a "keep".
This is a follow-up on If3a9b9e60064459d187c7db04eb4471a72c6cece.
Bug: 569962
Change-Id: I59a7ac41898ddc1dd90e86b09b621a41fdf45667
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This fixes two warnings of type LI_LAZY_INIT_STATIC.
Change-Id: I26a7a48aed9d0a0547e908a56b7014a7620fadd8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use a dedicated Lock object to lock the scheduler in
RepositoryCache#configureEviction to fix spotbugs warning
JLM_JSR166_UTILCONCURRENT_MONITORENTER.
Change-Id: I003dcf0ed1a0a3f4eea5d8a2f51a07473d28a928
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Path#getParent can return null, return fallback filestore attributes in
that case.
Change-Id: Ic09484d527bc87b27964b625e07373b82412f2da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This fixes spotbugs warning NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT.
This implementation violated the contract defined by
java.lang.Object.equals() because it did not check for null being passed
as the argument. All equals() methods should return false if passed a
null value.
Change-Id: I607f6979613d390aae2f3546b587f63133d6d73c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#compactFully.
Change-Id: I370578ad9a027c5c9709d60a1dfafdac0cfca908
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#DfsReftableDatabase and #clearCache.
Change-Id: Ifd3189288d2a8e64139c02cd105eb335fa2f68cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Also mark the return value @Nullable to enable null analysis in Eclipse.
Change-Id: I5b286d657d432f4b32afd4dd370f76892b115422
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
File#list can return null. Fix the potential NPE by using Files#list
which is also faster since it retrieves directory entries lazily while
File#list retrieves them eagerly.
Change-Id: Idf4bda398861c647587e357326b8bc8b587a2584
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
File#listFiles can return null. Use Files#list which does not return
null and should be faster since it's returning directory entries lazily
while File#listFiles fetches them eagerly.
Change-Id: I3bfe2a52278244fc469143692c06b05d9af0d0d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If TransportException is thrown in the finally block of execute()
ensure that the exception handled in the previous catch block isn't
suppressed.
Change-Id: I670acdfb4d36e7a419a9a79ae9faab2e085a43ee
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If TransportException is thrown in the finally block of
downloadPackedObject() ensure that the exception handled in the previous
catch block isn't suppressed.
Change-Id: I23982a2b215e38f681cc1719788985e60232699a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This was experimental code and never used in production.
Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The ObjectDirectory class manages the interactions for the entire object
database, this includes loose objects, packfiles, alternates, and
shallow commits. To help reduce the complexity of this class, abstract
some of the loose object specific details into a class which understands
just this, leaving the ObjectDirectory to focus more on the interactions
between the different mechanisms.
Change-Id: I39f3a74d6308f042a2a2baa57769f4acde5ba5e0
Signed-off-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The ObjectDirectory class manages the interactions for the entire object
database, this includes loose objects, packfiles, alternates, and
shallow commits. To help reduce the complexity of this class, abstract
some of the packfile specific details into a class which understands
just this, leaving the ObjectDirectory to focus more on the interactions
between the different mechanisms.
Change-Id: I5cc87b964434b0afa860b3fe23867a77b3c3a4f2
Signed-off-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Some clients may wish to allow NO_CHANGE lightweight tag updates
without setting the force flag. (For instance EGit does so.)
Command-line git does not allow this.
Propagate the RefUpdate result via the RefAlreadyExistsException.
That way a client has the possibility to catch it and check the
failure reason without having to parse the exception message, and
take appropriate action, like ignoring the exception on NO_CHANGE.
Change-Id: I60e7a15a3c309db4106cab87847a19b6d24866f6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
JGit treated a NO_CHANGE RefUpdate as an error in all cases. But when
updating a lightweight tag, this is a successful result if -f was
specified.
Change-Id: Iddfa6d6a6dc8bf8fed81138a008ebc32d5f960bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add the two config constants from C git that can switch on signing
of annotated tags. Add them to the GpgConfig, and implement actually
signing a tag in TagCommand.
The interactions between command line options for "git tag" and config
options is a bit murky in C git. There are two config settings for it:
* tag.gpgSign is the main option, if set to true, it kicks in if
neither -s nor -u are given on the command line.
* tag.forceSignAnnotated signs only tags created via "git tag -m",
but only if command-line option "-a" is not present. It applies
even if tag.gpgSign is set explicitly to false.
Giving -s or -u on the command line also forces an annotated tag
since lightweight tags cannot be signed.
Bug: 386908
Change-Id: Ic8a1a44b5f12f47d5cdf3aae2456c1f6ca9ef057
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Factor out a common ObjectBuilder as super class of CommitBuilder
and TagBuilder, and make the GpgSigner work on ObjectBuilder.
In order not to break API, add the new method for signing an
ObjectBuilder in a new interface GpgObjectSigner.
The signature for a tag is just tacked onto the end of the tag
message. The message of a signed tag must end in LF.
Bug: 386908
Change-Id: I5e021e3c927f4051825cd7355b129113b949455e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Observed the error when trying to force checkout from a branch
that had no changes on it. When the 'keep()' method from 'DirCacheCheckout'
method was called the 'DirCacheEntry e' was null and was passed like
this to the 'checkoutEntry()' method where the 'getObjectId()' is
being called on the 'e' object
Change-Id: If3a9b9e60064459d187c7db04eb4471a72c6cece
We are running several servers with jGit. We need to run repack from
time to time to keep the repos performant. I.e. after push we test how
many small packs are in the repo and when a threshold is reached we run
the repack.
After upgrading jGit version we've found that if someone does the clone
at the time repack is running the clone sometimes (not always) fails
because the repack removes .pack file used by the clone. Server
exception and client error attached.
I've tracked down the cause and it seems to be introduced between jGit
5.2 (which we upgraded from) and 5.3 and being caused by this commit:
Move throw of PackInvalidException outside the catch -
afef866a44
The problem is that when the throw was inside of the try block the last
catch block catched the exception and called openFailed(false) method.
It is true that it called it with invalidate = false, which is wrong.
The real problem though is that with the throw outside of the try block
the openFail is not called at all and the fields activeWindows and
activeCopyRawData are not set to 0. Which affects the later called tests
like: if (++activeCopyRawData == 1 && activeWindows == 0).
The fix for this is relatively simple keeping the throw outside of the
try block and still having the invalid field set to true. I did
exhaustive testing of the change running concurrent clones and pushes
indefinitely and with the patch applied it never fails while without the
patch it takes relatively short to get the error.
See: https://www.eclipse.org/lists/jgit-dev/msg04014.html
Bug: 569349
Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This would allow other JGit users to access and reuse the constants.
Change-Id: I1608802f45586af5f8582afa592e26679e9cebe3
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A SymbolicRef is added to the advertised refs for any symref in
capabilities whose target is an advertised ref; this may replace an
existing entry, such as HEAD.
When cloning, if any advertised HEAD is symbolic then use the target
rather than looking for an advertised ref with a matching objectId.
Add --symref option to LsRemote command.
Bug: 514052
Change-Id: Idfb48e6f6e8dcfe57a6896883fe6d84d533aa9d0
Signed-off-by: Lee Worrall <worrall.la@gmail.com>
- log an error
- either there is no list or it is incomplete hence return immediately
Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.
The problem was caused by the following race.
All mentioned files are located in "objects/pack/".
File endings relevant in "pack" dir:
.pack
.keep
.idx
.bitmap
When ReceivePack receives a pack file it executes the following steps:
ReceivePack.service():
receivePackAndCheckConnectivity():
receivePack():
receive the pack
parse the pack, returns packLock (.keep file)
PackInserter.flush():
write tmpPck file: "insert_<random>.pack"
write tmpIdx file: "insert_<random>.idx"
real pack name: "pack-<SHA1>.pack"
real index name: "pack-<SHA1>.idx"
atomic rename tmpPack to realPack
atomic rename tmpIdx to tmpIdx
execute commands
unlock pack by removing .keep file
trigger auto gc if enabled
When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.
Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.
Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.
Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.
Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
DirCacheEditor unconditionally applied a PathEdit to all stages in the
index. This gives wrong results if one wants to check out a file from
some commit to resolve a conflict: JGit would update the working tree
file multiple times (once per stage), and set all stages to point to
the checked-out blob.
C git replaces the stages by the entry for the checked-out file.
To support this, add a DirCacheEntry.setStage() method so that
CheckoutCommand can force the stage to zero. In DirCacheEditor, keep
only the zero stage if the PathEdit re-set the stage.
Bug: 568038
Change-Id: Ic7c635bb5aaa06ffaaeed50bc5e45702c56fc6d1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Due to Java bug JDK-8220793 [1] Java cannot set timestamps of a symlink
but only of the symlink target. This bug was fixed in Java 13. Since we
don't have a use case to set the timestamp of the symlink itself simply
document the current behavior of setLastModified methods.
[1] https://bugs.openjdk.java.net/browse/JDK-8220793
Change-Id: Ibc28c1702a1b9845602bd257606fbd44803a43fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
PerformanceLogContext threw NullPointerException when multiple threads
tried to add an event to the PerformanceLogContext. The cause for this
is that the ThreadLocal was initialized only in the class constructor
for the first thread; for subsequent threads it was null.
To fix this initialize eventRecords for each thread.
Change-Id: I18ef67dff8f0488e3ad28c9bbc18ce73d5168cf9
Signed-off-by: Alexa Panfil <alexapizza@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fix IOException occurring when calling
GC on a repository with absent objects/pack folder.
Change-Id: I5be1333a0726f4d7491afd25ddac85451686c30a
Signed-off-by: Nail Samatov <sanail@yandex.ru>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This reverts commit f802f06e7f.
I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.
Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does
if protocol V2
doFetchV2()
else
doFetchV0()
with doFetchV0 the old code, and doFetchV2 completely new.
Plus there would need to be a HTTP test case requiring several
negotiation rounds.
This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.
Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.
Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This problem occurred when calling SubmoduleWalk#getModuleName if the
first submodule processed has a name and a path which do not match
SubmoduleWalk#getModuleName returned the module path instead of the
module name. In order to fix this SubmoduleWalk#getModuleName needs to
ensure that the modules config is loaded.
Bug: 565776
Change-Id: I36ce1fbc64c4849f9d8e39864b825c6e28d344f8
Signed-off-by: John Dallaway <john@dallaway.org.uk>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add new performance logging to register events of type duration.
The proposed logging is similar to the performance logging
in OS Gerrit https://gerrit-review.googlesource.com/c/gerrit/+/225628:
a global Singleton (LoggingContext in Gerrit) is
collecting the performance logs in a thread-safe events list,
and at the end of the monitored command the list of events is
retrieved and written to a log, after which it is cleared.
What this patch does:
The main component is the Singleton (PerformanceLogContext), which
is used to collect the records (PerformanceLogRecord) in one place
(ThreadLocal eventsList) from anywhere using
PerformanceLogContext.getInstance().addEvent().
Reason why this change is needed:
The current monitoring in JGit has several issues:
1. git fetch and git push events are handled separately
(PackStatistics and ReceivedPackStatistics), with no unified way
of writing or reading the statistics.
2. PostUploadHook is only invoked on the event of sending the
pack, which means that the PackStatistics is not available for
the fetch requests that did not end with sending the pack
(negotiation requests).
3. The way the logs are created is different from the performance
log approach, so the long-running operations need to be collected
from both performance log (for JGit DFS overridden operations and
Gerrit operations) and gitlog (for JGit ones).
The proposed performance logging is going to solve the above
mentioned issues: it collects all of the performance logs in one
place, thus accounting for the commands that do not result in
sending a pack. The logs are compatible with the ones on Gerrit.
Moreover, the Singleton is accessible anywhere in the call stack,
which proved to be successful in other projects like Dapper
(https://research.google/pubs/pub36356/).
Signed-off-by: Alexa Panfil <alexapizza@google.com>
Change-Id: Iabfe667a3412d8a9db94aabb0f39b57f43469c41
This enables jgit to use any refs in the refs/ namespace when describing
commits.
Signed-off-by: Jason Yeo <jasonyeo88@gmail.com>
Change-Id: I1fa22d1c39c0e2f5e4c2938c9751d8556494ac26
Reason why this change is needed:
Currently the durations of fetch events are computed by
registering time instants with System.currentTimeMillis()
and calculating the differences with simple minus operation,
but multiple sources suggest that the best practice is to use
the Java 8 Duration and Instant objects.
What this patch does:
Get time measurements with Instant.now() instead of
System.currentTimeMillis() and calculate the duration of fetch
events (Reachability checks and Negotiation) using
Duration.between().toMillis().
Signed-off-by: Alexa Panfil <alexapizza@google.com>
Change-Id: I573a0a0562070083cf5a5a196d9710f69a7fa587
Building with Bazel is failing with this error message:
HttpSupport.java:480: error: [OperatorPrecedence] Use grouping
parenthesis to make the operator precedence explicit
if (c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z') {
^
(see https://errorprone.info/bugpattern/OperatorPrecedence)
Did you mean 'if ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')) {'?
Change-Id: I96089d3158d06ba981cfdf2b03865261b328de23
Validate the extra headers and log but otherwise ignore invalid
headers. An empty http.extraHeader starts the list afresh.
The http.userAgent is restricted to printable 7-bit ASCII, other
characters are replaced by '.'.
Moves a support method from the ssh.apache bundle to HttpSupport in
the main JGit bundle.
Bug:541500
Change-Id: Id2d8df12914e2cdbd936ff00dc824d8f871bd580
Signed-off-by: James Wynn <james@jameswynn.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This enables EGit to override with a lenient variant that logs the
problem and continues with the default value. EGit needs this because
otherwise a corrupt core.excludesFile entry can render the whole UI
unusable (until the git config is fixed) because any use of the
WorkingTreeIterator will throw an InvalidPathException.
This is not a problem on OS X, where all characters are allowed in
file names. But on Windows some characters are forbidden... see bug
567296. The message of the InvalidPathException is not helpful since
it doesn't point to the origin of the problem. EGit can log a much
better message indicating the offending config file and entry in the
Eclipse error log, where the user can see it.
Bug: 567309
Change-Id: I4e57afa715ff3aaa52cd04b5733f69e53af5b1e0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Reason why this change is needed:
Getting this metric will help estimate how much time will be saved once
the reachability checks get optimized
What this patch does:
Measure time spent by requestValidator.checkWants() in parseWants() and save
it in an instance of PackStatistics.Accumulator.
Signed-off-by: Alexa Panfil <alexapizza@google.com>
Change-Id: Id7fe4016f96549d9511a2c24052dad93cfbb31a4
Reason why this change is needed:
Getting this metric will help estimate how much time is spent
on negotiation in fetch V2.
What this patch does:
Measure time spent on negotiation rounds in UploadPack.fetchV2()
and save it in an instance of PackStatistics.Accumulator.
This is the same way the statistics are already gathered for
protocol V0/V1.
Change-Id: I14e55dd6ff743cb0b21b4953b533269ef069abb1
When comparing git directory paths to check whether one is a prefix
of another, one must add a slash to avoid false prefix matches when
one directory name is a prefix of another. The path "audio" is not
a prefix of the path "audio-new", but would be a prefix of a path
"audio/new".
Bug: 566799
Change-Id: I6f671ca043c7c2c6044eb05a71dc8cca8d0ee040
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This is useful to access git repositories behind a bastion server
(jump host).
Add a constant for the config; rewrite the whole connection initiation
to parse the value and (recursively) set up the chain of hops. Add
tests for a single hop and two different ways to configure a two-hop
chain.
The connection timeout applies to each hop in the chain individually.
Change-Id: Idd25af95aa2ec5367404587e4e530b0663c03665
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Since there is no negotiation for a push, the client is probably sending
redundant objects and bytes which already exist in the server.
Add more metrics in the stats to quantify it. Duplicated size and number
to measure the size and the number of duplicated objects which should
not be pushed.
Change-Id: Iaacd4761ee9366a0a7ec4e26c508eff45c8744de
Signed-off-by: Yunjie Li <yunjieli@google.com>
DfsBundleWriter writes out the entire repository to a Git bundle file.
It packs all objects included in the packfile by concatenating all pack
files. This makes the bundle creation fast and cheap. Useful for backing
up a repository as-is.
Change-Id: Iee20e4b1ab45b2a178dde8c72093c0dd83f04805
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
* changes:
ResolveMerger: do not content-merge gitlinks on del/mod conflicts
ResolveMerger: Adding test cases for GITLINK deletion
ResolveMerger: choose OURS on gitlink when ignoreConflicts
ResolveMerger: improving content merge readability
ResolveMerger: extracting createGitLinksMergeResult method
ResolveMerger: Adding test cases for GITLINK merge
Previously ResolveMerger tried to make a fulltext merge entry in case
one of sides got deleted regardless of file mode. This is not
applicable for GITLINK type of entry. After this change it is
rendering appropriate merge result.
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Ibdb4557bf8781bdb48bcee6529e37dc80582ed7e
Option ignoreConflicts is used when a caller want to create a virtual
commit and use it in a future merge (recursive merge) or show it on
UI (e.g. Gerrit). According to contract in case of ignoreConflicts
ResolveMerger should populate only stage 0 for entries with merge
conflicts as if there is no conflict. Current implementation breaks
this contract for cases when gitlink revision is ambiguous.
Therefore, always select 'ours' when we merge in ignoreConflicts mode.
This will satisfy the contract contract, so recursive merge can
succeed, however it is an arbitrary decision, so it is not guaranteed
to select best GITLINK in all cases.
GITLINK merging is a special case of recursive merge because of
limitations of GITLINK type of entry. It can't contain more than 1 sha-1
so jgit can't write merge conflicts in place like it can with a blob.
Ideally we could signal the conflict with a special value (like
'0000...'), but that must be supported by all tooling (git fsck, c-git)."
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Id4e9bebc8e828f7a1ef9f83259159137df477d89
Due to an integer overflow bug, the current "Index file is too large
for jgit" check did not work properly and subsequently a
NegativeArraySizeException was raised.
Change-Id: I2736efb28987c29e56bc946563b7fa781898a94a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Previous code used a minimum granularity of 1 microsecond and would
iterate 233 times on a system where the resolution is 1 second (for
instance, Java 8 on Mac APFS).
New code uses a binary search between the maximum we care about (2
seconds) and zero, with a minimum granularity of also 1 microsecond.
This takes at most 19 iterations (guaranteed). For a file system with 1
second resolution, it takes 4 iterations (1s, 0.5s, 0.8s, 0.9s). With
an up-front check at 1 microsecond and at 1 millisecond this performs
equally well as the old code on file systems with a fine resolution.
(For instance, Java 11 on Mac APFS.)
Also handle obscure cases where the file timestamp implementation may
yield bogus values (as observed on HP NonStop). If such an error case
occurs, log a warning and abort the measurement at the last good value.
Bug: 565707
Change-Id: I82a96729b50c284be7c23fbdf3d0df1bddf60e41
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
LockFile.lock() will create it anyway when the config file is created.
Bug: 565637
Change-Id: I078b89a695193fd76f130f6de7ac1cf26d2f8f0f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Otherwise locking problems may ensue if the JGit config itself is
on a different file system. Since the internal is already updated,
it is not really important when exactly the value gets persisted.
By queueing up separate Runnables executed by a single thread we
avoid concurrent write access to the JGit config, and nested calls
to getFileStoreAttributes(Path) result in serialized attempts to
write.
The thread for writing the config must not be a daemon thread. If
it were, JVM shutdown might kill it anytime, which may lead to
the config not being written, or worse, a config.lock file being
left behind.
Bug: 566170
Change-Id: I07e3d4c5e029d3cec9ab5895002fc4e0c7948c40
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the background job is a little late, the true result might
arrive and be cached later. So make sure we don't cache the large
fallback resolution in the per-directory cache. Otherwise we'd work
with the large fallback until the next restart.
Bug: 566170
Change-Id: I7354a6cfddfc0c05144bb0aa41c23029bd4f6af0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Git never converts line endings if the version in the repository is a
text file with CR/LF and text=auto. See [1]: "When the file has been
committed with CRLF, no conversion is done."
Because the sentence just before is about converting line endings on
check-in, I had understood that in commit 60cf85a [2] to mean that no
conversion on check-in was to be done. However, as bug 565048 and a
code inspection of the C git code showed it really means no conversion
is done on check-in *or check-out*.
If the text attribute is not set but core.autocrlf = true, this is
the same as text=auto eol=crlf. C git does not convert on check-out
even on text=auto eol=lf if the index version is a text file with
CR/LF.
For check-in, one has to look at the intended target, which is done
in WorkingTreeIterator since commit 60cf85a. For check-out, it can
be done by looking at the source and can thus be done in the
AutoLFOutputStream.
Additionally, provide a constructor for AutoLFInputStream to do
the same; for cases where the equivalent of a check-out is done via
an input stream obtained from a blob. (EGit does that in its
GitBlobStorage for the Eclipse compare framework; it's more efficient
than using a TemporaryBuffer and DirCacheCheckout.getContent(), and
it avoids the need for a temporary file.)
Adapt existing tests, and add new checkout and merge tests to verify
the resulting files have the correct line endings.
EGit's GitBlobStorage will need to call the new version of
EolStreamTypeUtil.wrapInputStream().
[1] https://git-scm.com/docs/gitattributes#Documentation/gitattributes.txt-Settostringvalueauto
[2] https://git.eclipse.org/r/c/jgit/jgit/+/127324
Bug: 565048
Change-Id: If1282ef43e2abd00263541bd10a01fe1f5c619fc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The WindowCache is configured statically with a default
WindowCacheConfig. The default config says (for backwards
compatibility reasons) to publish the MBean. As a result,
the MBean always gets published.
By delaying the MBean registration until the first call to
getInstance() or get(PackFile, long) we can avoid the forced
registration and do it only if not re-configured in the meantime
not to publish the bean. (As is done by Egit, to avoid a very
early costly access to the user and system config during plug-in
activation.)
Bug: 563740
Change-Id: I8a941342c0833acee2107515e64299aada7e0520
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Index format version 4 was introduced in C git in 2012. It's about
time that JGit can deal with it.
Version 4 added prefix path compression. Instead of writing the full
path for each index entry to disk, only the difference to the previous
entry's path is written: a variable-encoded int telling how many bytes
to remove from the previous entry's path to get the common prefix,
followed by the new suffix.
Also, cache entries in a version 4 index are not padded anymore.
Internally, version 3 and version 4 index entries are identical; it's
only the stored format that changes.
Implement this path compression, and make sure we write an index file
that we read previously in the same format. (Only changing from version
2 to version 3 if there are extended flags.)
Add support for the "feature.manyFiles" and the "index.version" git
configs, and honor them when writing a new index file.
Add tests, including a compatibility test that verifies that JGit can
read a version 4 index generated by C git and write an identical
version 4 index.
Bug: 565774
Change-Id: Id83241cf009e50f950eb42f8d56b834fb47da1ed
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The timeout on RemoteSession.exec() cannot be a timeout for the
whole command. It can only be a timeout for setting up the process;
after that it's the application's responsibility to implement some
timeout for the execution of the command, for instance by calling
Process.waitFor(int, TimeUnit) or through other means.
Sessions returned by an SshSessionFactory are already connected and
authenticated -- they must be, because RemoteSession offers no
operations for connecting or authenticating a session.
Change the implementation of SshdExecProcess.waitFor() to wait
indefinitely. The original implementation used the timeout from
RemoteSession.exec() because of that erroneous javadoc.
Change-Id: I3c7ede24ab66d4c81f72d178ce5012d383cd826e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
SshSupport.runSshCommand() had a comment that wait with time-out
could not be used because JSchProcess.exitValue() threw the wrong
unchecked exception when the process was still running.
Fix this and make JSchProcess.exitValue() throw the right exception,
then wait with a time-out in SshSupport.
The Apache sshd client's SshdExecProcess has always used the correct
IllegalThreadStateException.
Add tests for SshSupport.runCommand().
Change-Id: Id30893174ae8be3b9a16119674049337b0cf4381
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If I create a repository containing an empty file and clone it
with
git clone --no-checkout --filter=blob:none \
https://url/of/repository
then I would expect no blobs to be transferred over the wire. Alas,
JGit rewrites filter=blob:none to filter=blob:limit=0, so if the
repository contains an empty file then the empty blob gets
transferred.
Fix it by teaching JGit about filters based on object type to
complement the existing filters based on object size. This prepares
us for other future filters such as object:none.
In particular, this means we do not need to look up the size of the
filtered blobs, which should speed up clones. Noticed by Anna
Pologova and Terry Parker.
Change-Id: Id4b234921a190c108d8be2c87f54dcbfa811602a
Signed-off-by: Jonathan Nieder <jrn@google.com>
Teach the FilterSpec serialization code about tree filters so they can
be communicated over the wire and understood by the server.
While we're here, harden the FilterSpec serialization code to throw
IllegalStateException if we encounter a FilterSpec that cannot be
expressed as a "filter" line. The only public API for creating a
Filterspec is to pass in a "filter" line to be parsed, so these should
not appear in practice.
Change-Id: I9664844059ffbc9c36eb829e2d860f198b9403a0
Signed-off-by: Jonathan Nieder <jrn@google.com>
OS X ships with a default /usr/bin/git that is just a wrapper that
at run-time delegates to the selected XCode toolchain, and that
prompts the user to install the XCode command line tools if not
already installed.
This is annoying for people who don't want to do so, since they'll
be prompted on each Eclipse start. Also, since on OS X the $PATH for
applications started via the GUI is not the same as the $PATH as set
via the shell profile, just using /usr/bin/git (which will normally
be found when JGit runs inside Eclipse) may give slightly surprising
results if the user has installed a non-Apple git and changed his
$PATH in the shell such that the non-Apple git is used in the shell.
(For instance by placing /usr/local/bin earlier on the path.) Eclipse
and the shell will use different git executables, and thus different
git system configs.
Therefore, try to find git via bash --login -c 'which git' not only
if we couldn't find it on $PATH but also if we found the default git
/usr/bin/git. If that finds some other git, use that. If the bash
approach also finds /usr/bin/git, double check via xcode-select -p
that an XCode git is present. If not, assume there is no git installed,
and work without any system config.
Bug: 564372
Change-Id: Ie9d010ebd9437a491ba5d92b4ffd1860c203f8ca
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Using a fixed thread pool with unbounded LinkedBlockingQueue fixes the
RejectedExecutionException thrown if too many threads try to
concurrently determine filesystem attributes.
Comparing that to an alternative implementation using an unbounded
thread pool instead showed similar performance with the reproducer (in
range of 100-1000 threads in reproducer) on my mac:
threads time
fixed threadpool up to 5 threads with LinkedBlockingQueue of unlimited
queue size
100 1103 ms
200 1602 ms
300 2369 ms
500 4002 ms
1000 11071 ms
unbounded cached threadpool
100 1108 ms
200 1591 ms
300 2299 ms
500 4577 ms
1000 11196 ms
Bug: 564202
Change-Id: I773da7414a1dca8e548349442dca9b56643be946
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In JGit 5.0, the FileTreeIterator was changed to skip ignored folders
by default. To catch tracked files inside ignored folders, the tree
walk needs to have a DirCacheIterator, and the FileTreeIterator has
to know about that DirCacheIterator via setDirCacheIterator(). (Or
the optimization has to be switched off explicitly via
setWalkIgnoredDirectories(true).)
Skipping ignored directories is an important optimization in some
cases, for instance in node.js/npm projects, where we'd otherwise
traverse the whole huge and deep hierarchy of the typically ignored
node_modules folder.
While all uses of WorkingTreeIterator in JGit had been adapted,
DiffFormatter was forgotten. To make it work correctly (again) also
for such cases, make it set up a WorkingTreeeIterator automatically,
and make sure the WorkingTreeSource can find such files, too. Also
pass the repository to the TreeWalks used inside the DiffFormatter
to pick up the correct attributes, filters, and line-ending settings.
Bug: 565081
Change-Id: Ie88ac81166dc396ba28b83313964c1712b6ca199
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make sure we don't produce a spurious empty line at the end.
Bug: 564428
Change-Id: Ib991d93fbd052baca65d32a7842f07f9ddeb8130
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
MergedReftable ignores the last reftable in the stack while calculating the
minUpdateIndex.
Update the loop indices to include all reftables in the minUpdateIndex
calculation, while skipping position 0 as it is read outside the loop.
Change-Id: I12d3e714581e93d178be79c02408a67ab2bd838e
Signed-off-by: Minh Thai <mthai@google.com>
Currently we're buffering the inflated bitmap entry in
BasePackBitmapIndex to optimize running time. However, this will use
lots of memory during the creation of the pack bitmap index file.
And change 161456, which rewrote the entire getBitmap method, increased
the fetch latency significantly.
This commit introduces getBitmapWithoutCaching method which is used in
the pack bitmap index file creation only and aims to save memory during
garbage collection and not increase fetch latency.
Change-Id: I7b982c9d4e38f5f6193eaa03894e894ba992b33b
Signed-off-by: Yunjie Li <yunjieli@google.com>
The current mechanism for updating the unpack error handler requires
that the error handler is replaced entirely, including communicating
the error to the user. Adding a getter means that delegating
implementations can be constructed so that the error can be processed
before sending to the user, for example for logging.
Change-Id: I4b6f78a041d0f6f5b4076a9a5781565ca3857817
Signed-off-by: Jack Wickham <jwickham@palantir.com>
* stable-5.7:
ObjectDirectoryInserter: Open FileOutputStream in try-with-resource
ObjectDirectoryInserter: Remove redundant 'throws' declarations
ObjectDirectory: Further clean up insertUnpackedObject
ObjectDirectory: Explicitly handle NoSuchFileException
ObjectDirectory: Fail immediately when atomic move is not supported
Change-Id: I05186baa517388680fcc6825c940c4c772f26d32
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
ObjectWritingException and FileNotFoundException are subclasses
of IOException, which is already declared. Error does not need
to be explicitly declared.
Change-Id: I879820a33e10ec3a7ef676adc9c9148d2b3c4b27
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
- The code to move the file is repeated. Split it out into a
utility method.
- Remove the catch block for AtomicMoveNotSupportedException which
is redundant because it's handled in exactly the same way as the
IOException further down. The only exception we need to explicitly
handle differently in this block is NoSuchFileException.
- Improve the comments.
Change-Id: Ifc5490953ffb25ecd1c48a06289eccb3f19910c6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The shutdown method releases
- ThreadLocal held by NLS
- GlobalBundleCache used by NLS
- Executor held by WorkQueue
Bug: 437855
Bug: 550529
Change-Id: Icfdccd63668ca90c730ee47a52a17dbd58695ada
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a hunk does not apply at the position stated in the hunk header
try to determine its position using the old lines (context and
deleted lines).
This is still a far cry from a full git apply: it doesn't do binary
patches, it doesn't handle git's whitespace options, and it's perhaps
not the fastest on big patches. C git hashes the lines and uses these
hashes to speed up matching hunks (and to do its whitespace magic).
Bug: 562348
Change-Id: Id0796bba059d84e648769d5896f497fde0b787dd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Running recent error prone version complaining on that code:
CharacterHead.java:22: error: [ProtectedMembersInFinalClass] Make
members of final classes package-private: <init>
protected CharacterHead(char expectedCharacter) {
^
(see https://errorprone.info/bugpattern/ProtectedMembersInFinalClass)
Did you mean 'CharacterHead(char expectedCharacter) {'
Bug: 562756
Change-Id: Ic46a0b07e46235592f6e63db631f583303420b73
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
On the first attempt to move the temp file, NoSuchFileException can
be raised if the destination folder does not exist. Instead of handling
this implicitly in the catch of IOException and then continuing to
create the destination folder and try again, explicitly catch it and
create the destination folder. If any other IOException occurs, treat
it as an unexpected error and return FAILURE.
Subsequently, on the second attempt to move the temp file, if ANY kind
of IOException occurs, also consider this an unexpected error and
return FAILURE.
In both catch blocks for IOException, add logging at ERROR level.
Change-Id: I9de9ee3d2b368be36e02ee1c0daf8e844f7e46c8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
If atomic move is not supported, AtomicMoveNotSupportedException will
be thrown on the first attempt to move the temp file. There is no
point attempting the move operation a second time because it will only
fail for the same reason.
Add an immediate return of FAILURE on the first occasion. Remove the
unnecessary handling of the exception in the second block.
Change-Id: I4658a8b37cfec2d7ef0217c8346e512968d0964c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Running recent error prone version complaining on that code:
RefDatabase.java:444: error: [InvalidInlineTag] Tag name `linkObjectId`
is unknown.
* Includes peeled {@linkObjectId}s. This is the inverse lookup of
^
(see https://errorprone.info/bugpattern/InvalidInlineTag)
Bug: 562756
Change-Id: If91da51d5138fb753c0550eeeb9e3883a394123d
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Motivation: JSch serves as 'default' implementations of the SSH
transport. If a client application does not use it then there is no need
to pull in this dependency.
Move the classes depending on JSch to an OSGi fragment extending the
org.eclipse.jgit bundle and keep them in the same package as before
since moving them to another package would break API. Defer moving them
to a separate package to the next major release.
Add a new feature org.eclipse.jgit.ssh.jsch feature to enable
installation. With that users can now decide which of the ssh client
integrations (JCraft JSch or Apache Mina SSHD) they want to install.
We will remove the JCraft JSch integration in a later step due to the
reasons discussed in bug 520927.
Bug: 553625
Change-Id: I5979c8a9dbbe878a2e8ac0fbfde7230059d74dc2
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Motivation: BouncyCastle serves as 'default' implementation of
the GPG Signer. If a client application does not use it there is no need
to pull in this dependency, especially since BouncyCastle is a large
library.
Move the classes depending on BouncyCastle to an OSGi fragment extending
the org.eclipse.jgit bundle. They are moved to a distinct internal
package in order to avoid split packages. This doesn't break public API
since these classes were already in an internal package before this
change.
Add a new feature org.eclipse.jgit.gpg.bc to enable installation. With
that users can now decide if they want to install it.
Attempts to sign a commit if org.eclipse.jgit.gpg.bc isn't available
will result in ServiceUnavailableException being thrown.
Bug: 559106
Change-Id: I42fd6c00002e17aa9a7be96ae434b538ea86ccf8
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
If the determination of the user home directory produces a Java File
object with an invalid path, spurious exceptions may occur at the
most inopportune moments anytime later. In the case in the linked bug
report, start-up of EGit failed, leading to numerous user-visible
problems in Eclipse.
So validate the return value of FS.userHomeImpl(). If converting that
File to a Path throws an exception, log the problem and fall back to
Java system property user.home. If that also is not valid, use null.
(A null user home directory is allowed by FS, and calling in Java
new File(null, "some_string") is fine and produces a File relative
to the current working directory.)
Bug: 563739
Change-Id: If9eec0f9a31a45bd815231706285c71b09f8cf56
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make it possible to programmatically suppress the JMX bean
registration. In EGit it is not needed but can be rather costly
because it occurs during plug-in activation and accesses the
git user config.
Bug: 563740
Change-Id: I07ef7ae2f0208d177d2a03862846a8efe0191956
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Only the presence or absence of whitespace is significant; but not the
actual whitespace characters. Don't compare whitespace bytes.
Compare the C git implementation at [1].
[1] https://github.com/git/git/blob/0d0e1e8/xdiff/xutils.c#L173
Bug: 563570
Change-Id: I2d0522b637ba6b5c8b911b3376a9df5daa9d4c27
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This reverts commit 3aee92478c, which
increased fetch latency significantly.
Change-Id: Id31a94dff83bf7ab2121718ead819bd08306a0b6
Signed-off-by: Yunjie Li <yunjieli@google.com>
A builder API provides a more convenient way to define a customized
SshdSessionFactory by hiding the subclassing.
Also provide a new interface SshConfigStore to abstract away the
specifics of reading a ssh config file, and provide a way to customize
the concrete ssh config implementation to be used. This facilitates
using an alternate ssh config implementation that may or may not be
based on files.
Change-Id: Ib9038e8ff2a4eb3a9ce7b3554d1450befec8e1e1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Avoid trying other authentication methods on SocketException or on
InterruptedIOException. SocketException is rather fatal, such as
nothing listening on the peer's port, connection reset, or it could
be a connection time-out.
Time-outs enforced by Timeout{Input,Output}Stream may result in
InterruptedIOException being thrown.
In both cases, it makes no sense to try other authentication methods,
and doing so may wrongly report "authentication not supported" or
"cannot open git-upload-pack" or some such instead of reporting a
time-out.
Bug: 563138
Change-Id: I0191b1e784c2471035e550205abd06ec9934fd00
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Config core.eol is to be ignored if core.autocrlf is true or input.[1]
JGit didn't do so when core.autocrlf=input was set.
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-coreeol
Bug: 561877
Change-Id: I5e62e0510d160b5113c1090319af09c2bc1bcb59
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In Git 2.10.0 the interpretation of gitattributes changed or was fixed
such that "* text=auto eol=crlf" would indeed still do auto-detection
of text vs. binary content.[1] Previously this was identical to
"* text eol=crlf", i.e., treating all files as text.
JGit still did the latter, which caused surprises because it changed
binary files.
[1] https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248
Bug: 561341
Change-Id: I5b6fb97b5e86fd950a98537b6b8574f768ae30e5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Update dependency to Bouncy Castle to 1.65.
Add the IssuerFingerprint as a hashed sub-packet in the signature. If
added unhashed, GPG ignores it.
Bug: 553206
Change-Id: I6807e8e2385e6ec5790f388e4753a44aa9474ebb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* changes:
PackBitmapIndex: Set distance threshold
PackBitmapIndex: Not buffer inflated bitmap in BasePackBitmapIndex
PackBitmapIndex: Remove convertedBitmaps in the Remapper
PackBitmapIndex: Reduce memory usage in GC
PackBitmapIndex: Add AddToBitmapWithCacheFilter class
PackBitmapIndex: Add util methods and builder to BitmapCommit
PackBitmapIndex: Move BitmapCommit to a top-level class
Refactor: Make retriveCompressed an method of the Bitmap class
Setting the distance threshold to 2000 in PackWriterBitmapPreparer to
reduce memory usage in garbage collection. When the threshold is 0, GC
for the msm repository would use about 37 GB memory to complete. After
setting it to 2000, GC can finish in 75 min with about 10 GB memory.
Change-Id: I39783eeecbae58261c883735499e61ee1cac75fe
Signed-off-by: Yunjie Li <yunjieli@google.com>
Currently we're buffering the inflated bitmap entry in BasePackBitmapIndex
to optimize running time. However, this will use lots of memory during
the construction of the pack bitmap index file which may cause failure of
garbage collection.
The running time didn't increase significantly, if there's any increase,
after removing the buffering here. The report about usage of time/memory
will come in the next commit.
Change-Id: I874503ecc85714acab7ca62a6a7968c2dc0b56b3
Signed-off-by: Yunjie Li <yunjieli@google.com>
The convertedBitmaps serves for time-optimization purpose. But it's
actually not saving time much but using lots of memory. So remove the
field here to save memory.
Currently the remapper class is only used in the construction of the
bitmap index file. And during the preparation of the file, we're only
getting bitmaps from the remapper when finding objects accessible from
a commit, so bitmap associated with each commit will only be fetched once
and thus the convertedBitmaps would hardly be read, which means that it's
not saving time.
Change-Id: Ic942a8e485135fb177ec21d09282d08ca6646fdb
Signed-off-by: Yunjie Li <yunjieli@google.com>
Currently, the garbage collection is consistently failing for some large
repositories in the building bitmap phase, e.g.Linux-MSM project:
https://source.codeaurora.org/quic/la/kernel/msm-3.18
Historically, bitmap index creation happened in 3 phases:
1. Select the commits to which bitmaps should be attached.
2. Create all bitmaps for these commits, stored in uncompressed format
in the PackBitmapIndexBuilder.
3. Deltify the bitmaps and write them to disk.
We investigated the process. For phase 2 it's most efficient to create
bitmaps starting with oldest commit and moving to the newest commit,
because the newer commits are able to reuse the work for the old ones.
But for bitmap deltification in phase 3, it's better when a newer
commit's bitmap is the base, and the current disk format writes bitmaps
out for the newest commits first.
This change introduces a new collection to hold the deltified and
compressed representations of the bitmaps, keeping a smaller subset of
commits in the PackBitmapIndexBuilder to help make the bitmap index
creation more memory efficient.
And in this commit, we're setting DISTANCE_THRESHOLD to 0 in the
PackWriterBitmapPreparer, which means the garbage collection will not
have much behavoir change and will still use as much memory as before.
Change-Id: I6ec2c3e8dde11805af47874d67d33cf1ef83660e
Signed-off-by: Yunjie Li <yunjieli@google.com>
Add a new revwalk filter, AddToBitmapWithCachedFilter. This filter updates
a client-provided {@code BitmapBuilder} as a side effect of a revwalk.
Similar to {@code AddToBitmapFilter}, it short circuits the walk when it
encounters a commit which is included in the provided bitmap's BitmapIndex.
It also short circuits the walk if it encounters the client-provided
cached commit.
Change-Id: I62cb503016f4d3995d648d92b82baab7f93549a9
Signed-off-by: Yunjie Li <yunjieli@google.com>
Add some utility methods and a builder class for BitmapCommit class in
preparation for improving the memory footprint of GC's bitmap generation
phase.
Change-Id: Ice3d257fc26f3917a65a64eaf53b508b89043caa
Signed-off-by: Yunjie Li <yunjieli@google.com>
Move BitmapCommit from inside the PackWriterBitmapPreparer to a new
top-level class in preparation for improving the memory footprint of GC's
bitmap generation phase.
Change-Id: I4d404a5b3a34998b441d23105197f33d32d39670
Signed-off-by: Yunjie Li <yunjieli@google.com>
Make retrieveCompressed() a method of Bitmap interface to avoid type
casting and later reuse in improving the memory footprint of GC's bitmap
generation phase.
Change-Id: I098d85105cf17af845d43b8c71b4ca48b02fd7da
Signed-off-by: Yunjie Li <yunjieli@google.com>
When downloading LFS objects also accept response code 203 as successful
download. This response may be seen when downloading via a proxy.
Bug: 563022
Change-Id: Iee85fdb451b33369d08859872e5bfc2a67dffa6d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Introduce an IterativeConnectivityChecker which runs a connectivity
check with a filtered set of references, and falls back to using the
full set of advertised references.
It uses references during first check attempt:
- References that are ancestors of an incoming commits (e.g., pushing
a commit onto an existing branch or pushing a new branch based on
another branch)
- Additional list of references we know client can be interested in
(e.g. list of open changes for Gerrit)
We tested it inside Google and it improves connectivity for certain
topologies. For example connectivity counts for
chromium.googlesource.com/chromium/src:
percentile_50: 1923 (was: 22777)
percentile_90: 23272 (was: 353003)
percentile_99: 345522 (was: 353435)
This saved ~2 seconds on every push to this repository.
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I6543c2e10ed04622ca795b195665133e690d3b10
Moving transport related internal classes into dedicated subpackage in
o/e/j/internal package.
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I21ed029d359f5f7d8298f102efbb4b1dcdf404ad
Fix NullPointerException occurring when calling
CheckoutCommand with forced == true option when
the branch isn't changed and there is deleted
uncommitted file.
Change-Id: I99bf1fc25e6889f07092320d7bc2772ec5d341b5
Signed-off-by: Nail Samatov <sanail@yandex.ru>
When applying a patch that contains renames or copies using ApplyCommand,
also apply all hunks that apply to the renamed or copied file.
Change-Id: I9f3fa4370458bd7c14beeb2e2b49e846d70203cb
Signed-off-by: Jack Wickham <jwickham@palantir.com>
Before this change, applying a patch will fail if the destination directory
doesn't exist; after, the necessary parent directories are created.
If renaming the file fails, the directories won't be deleted, so this change
isn't atomic. However, ApplyCommand is already not atomic - if one hunk fails
to apply, other hunks still get applied - so I don't think that is a blocker.
Change-Id: Iea36138b806d4e7012176615bcc673756a82f365
Signed-off-by: Jack Wickham <jwickham@palantir.com>
ObjectReachabilityChecker interface is the only public API. The
implementation is instantiated by ObjectWalk and doesn't need to be
visible outside the package.
Change-Id: I5b97bb98990cded637686bdc15c9655330b7780f
Signed-off-by: Ivan Frade <ifrade@google.com>
The bitmap-bassed object reachability checker, tries to find the objects
in the first starter, then adding the second starter... and so on. This
rewards passing the most popular refs first.
Order the refs with heads first, then tags, then others (e.g. changes)
for the object reachability checker. Using streams, delay also the
resolution of the ref to RevObject until necessary.
Change-Id: I9414b76754d7c0ffee1e2eeed6939895c8e92cbe
Signed-off-by: Ivan Frade <ifrade@google.com>
ObjectWalk#createObjectReachabilityChecker() returns the best
implementation for the repo. UploadPack can use the interface and fold
the with/without commits cases in one code path.
Change-Id: I857c11735d1d8e36c3ed8185ff11de8a62e86540
Signed-off-by: Ivan Frade <ifrade@google.com>
Extract ObjectReachabilityChecker interface from the walk-based
implementation, to add a bitmapped based implementation later.
Refactor the test case to use it for both implementations.
Change-Id: Iaac7c6b037723811956ac22625f27d3b4d742139
Signed-off-by: Ivan Frade <ifrade@google.com>
Preparing the code to optimize the bitmap-based object reachability
checker. We are mirroring first the commit reachability checker
structure (interface + 2 implementations).
Move the walk-base reachability checker to its own class.
This class is public at the moment. Later ObjectWalk will return an
interface and this implementation will be package-private.
Change-Id: Ifac70094e1af137291c3607d95e689992f814b26
Signed-off-by: Ivan Frade <ifrade@google.com>
After negotiation phase of a fetch, the advertised ref map is no longer used and
can be safely cleared. For >1GiB repos object selection and packfile writing may
take 10s of minutes. For the chromium.googlesource.com/chromium/src repo, this
advertised ref map is >400MiB. Returning this memory to the Java heap is a major
scalability win.
Change-Id: I00d453c5ef47630c21f199e333e1cfcf47b7e92a
Signed-off-by: Minh Thai <mthai@google.com>
Since reftables might have update index ranges that are overlapped.
Change-Id: I8f8215b99a0a978d4dd0155dbaf33e5e06ea8202
Signed-off-by: Minh Thai <mthai@google.com>
(cherry picked from commit 06748c205c)
Ensure files are writable before trying to delete them.
Bug: 408846
Change-Id: I930a547594bba853c33634ae54bd64d236afade3
Signed-off-by: Alexander Nittka <alex@nittka.de>
* stable-5.7:
FS.runInShell(): handle quoted filters and hooksPath containing blanks
Handle non-normalized index also for executable files
Prepare 5.7.1-SNAPSHOT builds
JGit v5.7.0.202003110725-r
Change-Id: I8a8580e44bfa05989d476cf22a029abd4fd407c6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.6:
FS.runInShell(): handle quoted filters and hooksPath containing blanks
Handle non-normalized index also for executable files
Change-Id: I240377e87c073ee7a621a88e39fc319c59fa037a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Revert commit 2323d7a. Using $0 in the shell command call results in
the command string being taken literally. That was introduced to fix
a problem with backslashes, but is actually not correct.
First, the problem with backslashes occurred only on Win32/Cygwin,
and has been properly fixed in commit 6f268f8.
Second, this is used only for hooks (which don't have backslashes in
their names) and filter commands from the git config, where the user
is responsible for properly quoting or escaping such that the commands
work.
Third, using $0 actually breaks correctly quoted filter commands
like in the bug report. The shell really takes the command literally,
and then doesn't find the command because of quotes.
So revert this change.
At the same time there's a related problem with hooks. If the path to
the hook contains blanks, runInShell() would also fail to find the
hook. In this case, the command doesn't come from user input but is
just a Java File object with an absolute path containing blanks. (Can
occur if core.hooksPath points to such a path with blanks, or if the
repository has such a path.)
The path to the hook as obtained from the file system must be quoted.
Add a test for a hook path with a blank.
This reverts commit 2323d7a1ef.
Bug: 561666
Change-Id: I4d7df13e6c9b245fe1706e191e4316685a8a9d59
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Change-Id: Ifb8227cb62370029d6774f2a22b15d6478c713ca
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
ReceiveCommand can have an error message. This is shown only for some
cases even if it's set. This change uses the error message if it's set,
and fallback to the default message if unset.
Change-Id: I8d906e71ad08cf49bcdb28caea8fcc66798c68ff
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Commit 60cf85a4 corrected the handling of check-in for files where
the index version is non-normalized, i.e., contains CR-LF line endings.
However, it did so only for regular files, not executable files.
Bug: 561438
Change-Id: I372cc990c5efeb00315460f36459c0652d5d1e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Since reftables might have update index ranges that are overlapped.
Change-Id: I8f8215b99a0a978d4dd0155dbaf33e5e06ea8202
Signed-off-by: Minh Thai <mthai@google.com>
The recursive merge strategy builds a virtual ancestor merging
recursively the common bases (when more than one) between the
want-to-merge commits. While building this virtual ancestor, content
conflicts are ignored, but current code doesn't do so when a file is
removed.
This was spotted in [1], for example. Merging two commits to build the
virtual ancestor bumped into a conflict (modified in one side, deleted
in the other) that stopped the process.
Follow the "spec" and in case of conflict leave the unmerged content in
the index and working trees.
[1] https://android-review.googlesource.com/c/kernel/common/+/1228962
Change-Id: Ife9c32ae3ac3a87d3660fa1242e07854b65169d5
Signed-off-by: Ivan Frade <ifrade@google.com>
Allow explicitly setting the tag option for the remote configuration
when cloning a repository.
Bug: 561021
Change-Id: Iac43268a2bb231ae7599c3255bf555883d34fa32
Signed-off-by: Alexander Nittka <alex@nittka.de>
The error message for an Exception thrown by StartGenerator when given
both the TOPO flag and the TOPO_KEEP_BRANCH_TOGETHER flag mentions a
non-existent flag, TOPO_NON_INTERMIX. The error message was introduced
in commit e498d43.
Replace TOPO_NON_INTERMIX with TOPO_KEEP_BRANCH_TOGETHER in the error
message of an Exception thrown by the StartGenerator when the TOPO flag
is provided together with the TOPO_KEEP_BRANCH_TOGETHER flag.
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Id24640dc08e96a196508fe38ce144aa7e035082f
The topological sort algorithm in TopoSortGenerator for RevWalk may mix
multiple lines of history, producing results that differ from C git's
git-log whose man page states: "Show no parents before all of its
children are shown, and avoid showing commits on multiple lines of
history intermixed." Lines of history are mixed because
TopoSortGenerator merely delays producing a commit until all of its
children have been produced; it does not immediately produce a commit
after its last child has been produced.
Therefore, add a new RevSort option called TOPO_KEEP_BRANCH_TOGETHER
with a new topo sort algorithm in TopoNonIntermixGenerator. In the
Generator, when the last child of a commit has been produced, unpop
that commit so that it will be returned upon the subsequent call to
next(). To avoid producing duplicates, mark commits that have not yet
been produced as TOPO_QUEUED so that when a commit is popped, it is
produced if and only if TOPO_QUEUED is set.
To support nesting with other generators that may produce the same
commit multiple times like DepthGenerator (for example, StartGenerator
does this), do not increment parent inDegree for the same child commit
more than once.
Commit b5e764abd2 modified the existing
TopoSortGenerator to avoid mixing lines of history, but it was reverted
in e40c38ab08 because the new behavior
caused problems for EGit users. This motivated adding a new Generator
for the new behavior.
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Icbb24eac98c00e45c175b01e1c8122554f617933
RFC 7538[1] added HTTP response code 308, signifying a permanent
redirect that, contrary to the older 301, does not allow changing
the request method from POST to GET.
[1] https://tools.ietf.org/html/rfc7538
Bug: 560936
Change-Id: Ib65f3a3ed75db51d74d1fe81d4abe6fe92b0ca12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Andre Bossert