Commit Graph

2912 Commits

Author SHA1 Message Date
Matthias Sohn bf6b2b9314 JGit v5.9.0.202008260805-m3
Change-Id: Ic4de8340f3ab038e38b239b725b8bd6d6dbee413
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-08-26 14:05:07 +02:00
Marc Strapetz 0220f32e5a Fix possible NegativeArraySizeException in PackIndexV1
Due to an integer overflow bug, the current "Index file is too large
for jgit" check did not work properly and subsequently a
NegativeArraySizeException was raised.

Change-Id: I2736efb28987c29e56bc946563b7fa781898a94a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2020-08-25 12:42:53 -04:00
Thomas Wolf efd1cc05af Keep line endings for text files committed with CR/LF on text=auto
Git never converts line endings if the version in the repository is a
text file with CR/LF and text=auto. See [1]: "When the file has been
committed with CRLF, no conversion is done."

Because the sentence just before is about converting line endings on
check-in, I had understood that in commit 60cf85a [2] to mean that no
conversion on check-in was to be done. However, as bug 565048 and a
code inspection of the C git code showed it really means no conversion
is done on check-in *or check-out*.

If the text attribute is not set but core.autocrlf = true, this is
the same as text=auto eol=crlf. C git does not convert on check-out
even on text=auto eol=lf if the index version is a text file with
CR/LF.

For check-in, one has to look at the intended target, which is done
in WorkingTreeIterator since commit 60cf85a. For check-out, it can
be done by looking at the source and can thus be done in the
AutoLFOutputStream.

Additionally, provide a constructor for AutoLFInputStream to do
the same; for cases where the equivalent of a check-out is done via
an input stream obtained from a blob. (EGit does that in its
GitBlobStorage for the Eclipse compare framework; it's more efficient
than using a TemporaryBuffer and DirCacheCheckout.getContent(), and
it avoids the need for a temporary file.)

Adapt existing tests, and add new checkout and merge tests to verify
the resulting files have the correct line endings.

EGit's GitBlobStorage will need to call the new version of
EolStreamTypeUtil.wrapInputStream().

[1] https://git-scm.com/docs/gitattributes#Documentation/gitattributes.txt-Settostringvalueauto
[2] https://git.eclipse.org/r/c/jgit/jgit/+/127324

Bug: 565048
Change-Id: If1282ef43e2abd00263541bd10a01fe1f5c619fc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-17 08:52:55 +02:00
Thomas Wolf e9cb0a8e47 DirCache: support index V4
Index format version 4 was introduced in C git in 2012. It's about
time that JGit can deal with it.

Version 4 added prefix path compression. Instead of writing the full
path for each index entry to disk, only the difference to the previous
entry's path is written: a variable-encoded int telling how many bytes
to remove from the previous entry's path to get the common prefix,
followed by the new suffix.

Also, cache entries in a version 4 index are not padded anymore.

Internally, version 3 and version 4 index entries are identical; it's
only the stored format that changes.

Implement this path compression, and make sure we write an index file
that we read previously in the same format. (Only changing from version
2 to version 3 if there are extended flags.)

Add support for the "feature.manyFiles" and the "index.version" git
configs, and honor them when writing a new index file.

Add tests, including a compatibility test that verifies that JGit can
read a version 4 index generated by C git and write an identical
version 4 index.

Bug: 565774
Change-Id: Id83241cf009e50f950eb42f8d56b834fb47da1ed
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-15 12:47:45 +02:00
Jonathan Nieder 3c807e0158 Do not send empty blob in response to blob:none filter
If I create a repository containing an empty file and clone it
with

	git clone --no-checkout --filter=blob:none \
		https://url/of/repository

then I would expect no blobs to be transferred over the wire.  Alas,
JGit rewrites filter=blob:none to filter=blob:limit=0, so if the
repository contains an empty file then the empty blob gets
transferred.

Fix it by teaching JGit about filters based on object type to
complement the existing filters based on object size.  This prepares
us for other future filters such as object:none.

In particular, this means we do not need to look up the size of the
filtered blobs, which should speed up clones.  Noticed by Anna
Pologova and Terry Parker.

Change-Id: Id4b234921a190c108d8be2c87f54dcbfa811602a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2020-07-29 21:04:20 -07:00
Jonathan Nieder dceedbcd6e Add support for tree filters when fetching
Teach the FilterSpec serialization code about tree filters so they can
be communicated over the wire and understood by the server.

While we're here, harden the FilterSpec serialization code to throw
IllegalStateException if we encounter a FilterSpec that cannot be
expressed as a "filter" line.  The only public API for creating a
Filterspec is to pass in a "filter" line to be parsed, so these should
not appear in practice.

Change-Id: I9664844059ffbc9c36eb829e2d860f198b9403a0
Signed-off-by: Jonathan Nieder <jrn@google.com>
2020-07-29 20:52:12 -07:00
Matthias Sohn fed5b2e6fb Remove unused imports
Change-Id: I7c44e3603df2dd368cb7c0ba0072413b887b6903
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-07-19 12:00:35 +02:00
Thomas Wolf 5332723729 DiffFormatter: correctly deal with tracked files in ignored folders
In JGit 5.0, the FileTreeIterator was changed to skip ignored folders
by default. To catch tracked files inside ignored folders, the tree
walk needs to have a DirCacheIterator, and the FileTreeIterator has
to know about that DirCacheIterator via setDirCacheIterator(). (Or
the optimization has to be switched off explicitly via
setWalkIgnoredDirectories(true).)

Skipping ignored directories is an important optimization in some
cases, for instance in node.js/npm projects, where we'd otherwise
traverse the whole huge and deep hierarchy of the typically ignored
node_modules folder.

While all uses of WorkingTreeIterator in JGit had been adapted,
DiffFormatter was forgotten. To make it work correctly (again) also
for such cases, make it set up a WorkingTreeeIterator automatically,
and make sure the WorkingTreeSource can find such files, too. Also
pass the repository to the TreeWalks used inside the DiffFormatter
to pick up the correct attributes, filters, and line-ending settings.

Bug: 565081
Change-Id: Ie88ac81166dc396ba28b83313964c1712b6ca199
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-07-17 00:50:24 +02:00
Matthias Sohn b5ba237559 Prepare 5.8.2-SNAPSHOT builds
Change-Id: I33327417cd5f5b55e860c5d9c6ee06cac7d10b44
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-07-14 20:53:51 +02:00
Matthias Sohn 246954e0d6 JGit v5.8.1.202007141445-r
Change-Id: Ib63831292eded15af18ed30a9653831dadfd0d4a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-07-14 20:43:21 +02:00
Thomas Wolf 9b033a1b6d Fix writing GPG signatures with trailing newline
Make sure we don't produce a spurious empty line at the end.

Bug: 564428
Change-Id: Ib991d93fbd052baca65d32a7842f07f9ddeb8130
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-07-08 09:28:29 +02:00
David Pursehouse 8774f54190 Improve error message when receive.maxCommandBytes is exceeded
The message "Too many commands" implies there is a hard limit on the
number of commands, which isn't the case. The limit is on the total
size of the received data, as explained in change I84317d396 which
introduced the configuration setting receive.maxCommandBytes:

  shorter reference names allow for more commands, longer reference
  names permit fewer commands per batch.

Change the message to:

  Commands size exceeds limit defined in receive.maxCommandBytes

Change-Id: I678b78f919b2fec8f8058f3403f2541c26a5d00e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-06-29 08:57:42 +09:00
David Pursehouse 14509f4cbb DiffFormatterTest: Add a test to confirm the default rename detection settings
Add a test that confirms:

- No rename detector is initialized by default

- Rename detector is initialized after calling setDetectRenames(true)

- Rename limit and rename score have the default values 400 and
  60, respectively. Note that there are no constants for these values
  so the test hard codes them.

Change-Id: I327e2b348a40ef67d8a184e5ab09f4e9ab573e1c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-06-26 11:24:48 +09:00
Minh Thai 9719ca411e MergedReftable: Include the last reftable in determining minUpdateIndex
MergedReftable ignores the last reftable in the stack while calculating the
minUpdateIndex.

Update the loop indices to include all reftables in the minUpdateIndex
calculation, while skipping position 0 as it is read outside the loop.

Change-Id: I12d3e714581e93d178be79c02408a67ab2bd838e
Signed-off-by: Minh Thai <mthai@google.com>
2020-06-22 17:14:35 -07:00
Demetr Starshov 0ec6e4b3d5 SimpleMergeTest: Clean up code style
Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Ib0149279324a8b085eec4c60d02c8f32a8cc86bb
2020-06-10 11:35:58 +09:00
Matthias Sohn 6abe695aa5 Prepare 5.8.1-SNAPSHOT builds
Change-Id: Ic654fb45abe4e94f4eee532af0f4278d372d37f5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-06-09 16:45:56 +02:00
Matthias Sohn 8e79d5a290 JGit v5.8.0.202006091008-r
Change-Id: I2020e9821c359b90b7c830031945e2fc659ea607
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-06-09 16:07:50 +02:00
Matthias Sohn 855842af19 Prepare 5.9.0-SNAPSHOT builds
Change-Id: Ia998e2772df1285a4c674b07201f15d53156eb78
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-06-09 12:51:12 +02:00
Thomas Wolf ed481f96b8 ApplyCommand: use context lines to determine hunk location
If a hunk does not apply at the position stated in the hunk header
try to determine its position using the old lines (context and
deleted lines).

This is still a far cry from a full git apply: it doesn't do binary
patches, it doesn't handle git's whitespace options, and it's perhaps
not the fastest on big patches. C git hashes the lines and uses these
hashes to speed up matching hunks (and to do its whitespace magic).

Bug: 562348
Change-Id: Id0796bba059d84e648769d5896f497fde0b787dd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-06-04 22:16:12 +02:00
Thomas Wolf 55371c5f06 Add tests for RawTextComparator.WS_IGNORE_CHANGE.hash()
Change-Id: I1ed4df789094e09c39b3c2054fe5b9bd0c1a8f9b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-06-02 00:13:37 +02:00
Matthias Sohn 8d2d683655 Decouple JSch from JGit Core
Motivation: JSch serves as 'default' implementations of the SSH
transport. If a client application does not use it then there is no need
to pull in this dependency.

Move the classes depending on JSch to an OSGi fragment extending the
org.eclipse.jgit bundle and keep them in the same package as before
since moving them to another package would break API. Defer moving them
to a separate package to the next major release.

Add a new feature org.eclipse.jgit.ssh.jsch feature to enable
installation. With that users can now decide which of the ssh client
integrations (JCraft JSch or Apache Mina SSHD) they want to install.
We will remove the JCraft JSch integration in a later step due to the
reasons discussed in bug 520927.

Bug: 553625
Change-Id: I5979c8a9dbbe878a2e8ac0fbfde7230059d74dc2
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:46:59 +02:00
Matthias Sohn 77848d635b Decouple BouncyCastle from JGit Core
Motivation: BouncyCastle serves as 'default' implementation of
the GPG Signer. If a client application does not use it there is no need
to pull in this dependency, especially since BouncyCastle is a large
library.

Move the classes depending on BouncyCastle to an OSGi fragment extending
the org.eclipse.jgit bundle. They are moved to a distinct internal
package in order to avoid split packages. This doesn't break public API
since these classes were already in an internal package before this
change.

Add a new feature org.eclipse.jgit.gpg.bc to enable installation. With
that users can now decide if they want to install it.

Attempts to sign a commit if org.eclipse.jgit.gpg.bc isn't available
will result in ServiceUnavailableException being thrown.

Bug: 559106
Change-Id: I42fd6c00002e17aa9a7be96ae434b538ea86ccf8
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:26:22 +02:00
Thomas Wolf 6f17f9ed3f RawTextComparator.WS_IGNORE_CHANGE must not compare whitespace
Only the presence or absence of whitespace is significant; but not the
actual whitespace characters. Don't compare whitespace bytes.

Compare the C git implementation at [1].

[1] https://github.com/git/git/blob/0d0e1e8/xdiff/xutils.c#L173

Bug: 563570
Change-Id: I2d0522b637ba6b5c8b911b3376a9df5daa9d4c27
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-28 12:06:57 +02:00
Thomas Wolf 3dbd1f2fe7 Ignore core.eol if core.autocrlf=input
Config core.eol is to be ignored if core.autocrlf is true or input.[1]
JGit didn't do so when core.autocrlf=input was set.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-coreeol

Bug: 561877
Change-Id: I5e62e0510d160b5113c1090319af09c2bc1bcb59
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-22 17:09:23 -04:00
Thomas Wolf 3c34e0acbf Attributes: fix handling of text=auto in combination with eol
In Git 2.10.0 the interpretation of gitattributes changed or was fixed
such that "* text=auto eol=crlf" would indeed still do auto-detection
of text vs. binary content.[1] Previously this was identical to
"* text eol=crlf", i.e., treating all files as text.

JGit still did the latter, which caused surprises because it changed
binary files.

[1] https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248

Bug: 561341
Change-Id: I5b6fb97b5e86fd950a98537b6b8574f768ae30e5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-22 17:08:52 -04:00
Terry Parker 55b0203c31 Merge changes I39783eee,I874503ec,Ic942a8e4,I6ec2c3e8,I62cb5030, ...
* changes:
  PackBitmapIndex: Set distance threshold
  PackBitmapIndex: Not buffer inflated bitmap in BasePackBitmapIndex
  PackBitmapIndex: Remove convertedBitmaps in the Remapper
  PackBitmapIndex: Reduce memory usage in GC
  PackBitmapIndex: Add AddToBitmapWithCacheFilter class
  PackBitmapIndex: Add util methods and builder to BitmapCommit
  PackBitmapIndex: Move BitmapCommit to a top-level class
  Refactor: Make retriveCompressed an method of the Bitmap class
2020-05-13 16:34:23 -04:00
Yunjie Li d23254ee57 PackBitmapIndex: Move BitmapCommit to a top-level class
Move BitmapCommit from inside the PackWriterBitmapPreparer to a new
top-level class in preparation for improving the memory footprint of GC's
bitmap generation phase.

Change-Id: I4d404a5b3a34998b441d23105197f33d32d39670
Signed-off-by: Yunjie Li <yunjieli@google.com>
2020-05-12 17:32:15 -07:00
Terry Parker 0642e49f97 Merge changes I6543c2e1,I21ed029d
* changes:
  ReceivePack: adding IterativeConnectivityChecker
  Moving transport/internal -> internal/transport
2020-05-11 17:40:20 -04:00
Demetr Starshov 9075beefb1 ReceivePack: adding IterativeConnectivityChecker
Introduce an IterativeConnectivityChecker which runs a connectivity
check with a filtered set of references, and falls back to using the
full set of advertised references.

It uses references during first check attempt:
- References that are ancestors of an incoming commits (e.g., pushing
a commit onto an existing branch or pushing a new branch based on
another branch)
- Additional list of references we know client can be interested in
(e.g. list of open changes for Gerrit)

We tested it inside Google and it improves connectivity for certain
topologies. For example connectivity counts for
chromium.googlesource.com/chromium/src:

percentile_50: 1923 (was: 22777)
percentile_90: 23272 (was: 353003)
percentile_99: 345522 (was: 353435)

This saved ~2 seconds on every push to this repository.

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I6543c2e10ed04622ca795b195665133e690d3b10
2020-05-08 17:57:20 -07:00
Nail Samatov f30e48914e Fix error occurring during checkout
Fix NullPointerException occurring when calling
CheckoutCommand with forced == true option when
the branch isn't changed and there is deleted
uncommitted file.

Change-Id: I99bf1fc25e6889f07092320d7bc2772ec5d341b5
Signed-off-by: Nail Samatov <sanail@yandex.ru>
2020-05-07 12:18:44 +03:00
Jack Wickham b6d70a66ae Apply hunks when renaming or copying from patch files
When applying a patch that contains renames or copies using ApplyCommand,
also apply all hunks that apply to the renamed or copied file.

Change-Id: I9f3fa4370458bd7c14beeb2e2b49e846d70203cb
Signed-off-by: Jack Wickham <jwickham@palantir.com>
2020-05-06 08:50:08 +02:00
Jack Wickham d69c0ef5bd Create parent directories when renaming a file in ApplyCommand
Before this change, applying a patch will fail if the destination directory
doesn't exist; after, the necessary parent directories are created.

If renaming the file fails, the directories won't be deleted, so this change
isn't atomic. However, ApplyCommand is already not atomic - if one hunk fails
to apply, other hunks still get applied - so I don't think that is a blocker.

Change-Id: Iea36138b806d4e7012176615bcc673756a82f365
Signed-off-by: Jack Wickham <jwickham@palantir.com>
2020-05-06 08:50:08 +02:00
David Ostrovsky 3a7e4eaecd Add missing test source file to the build
Change-Id: I020ff281eaa49794aeb1853711485bc8c7047dd8
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-05-03 23:20:18 +02:00
Ivan Frade 003002c1cb revwalk: Introduce bitmap-based object reachability checker
Change-Id: I0b1a2bd21f98894862aab339f8c2e4a417897b89
Signed-off-by: Ivan Frade <ifrade@google.com>
2020-04-28 22:44:33 -07:00
Ivan Frade 20bb312421 revwalk: Extract ObjectReachabilityChecker interface
Extract ObjectReachabilityChecker interface from the walk-based
implementation, to add a bitmapped based implementation later.

Refactor the test case to use it for both implementations.

Change-Id: Iaac7c6b037723811956ac22625f27d3b4d742139
Signed-off-by: Ivan Frade <ifrade@google.com>
2020-04-28 16:56:32 -07:00
Ivan Frade ae26fa19b7 UploadPack: Extract walk-based reachability check
Preparing the code to optimize the bitmap-based object reachability
checker.  We are mirroring first the commit reachability checker
structure (interface + 2 implementations).

Move the walk-base reachability checker to its own class.

This class is public at the moment. Later ObjectWalk will return an
interface and this implementation will be package-private.

Change-Id: Ifac70094e1af137291c3607d95e689992f814b26
Signed-off-by: Ivan Frade <ifrade@google.com>
2020-04-28 16:40:52 -07:00
Minh Thai d9f84b0b7c UploadPack: Clear advertised ref map after negotiation
After negotiation phase of a fetch, the advertised ref map is no longer used and
can be safely cleared. For >1GiB repos object selection and packfile writing may
take 10s of minutes. For the chromium.googlesource.com/chromium/src repo, this
advertised ref map is >400MiB. Returning this memory to the Java heap is a major
scalability win.

Change-Id: I00d453c5ef47630c21f199e333e1cfcf47b7e92a
Signed-off-by: Minh Thai <mthai@google.com>
2020-04-23 12:14:02 -07:00
Konrad Windszus 54a2d48008 Fix human name for local .bundle files
Bug: 560903
Change-Id: I15d45330398cc573940265d16a2db29ddce085aa
Signed-off-by: Konrad Windszus <konrad_w@gmx.de>
2020-04-21 07:46:50 +02:00
Matthias Sohn 7bebce42cc Merge branch 'stable-5.7'
* stable-5.7:
  Bazel: Disable SecurityManagerMissingPermissionsTest test

Change-Id: Id8665d1ba630c1359e86846548143a785b178e75
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-04-18 19:08:20 +02:00
David Ostrovsky 9b2f8ced90 Bazel: Disable SecurityManagerMissingPermissionsTest test
In Id5376f09f0d a test with dependency on log4j library was added, but
the library was missed to be added to the Bazel build tool chain.

Given that Bazel test runner doesn't suport custom security manager the
test wouldn't pass even if the missing dependency would be added. The
only solution we have for now is to exclude that test from Bazel tool
chain.

Filed a feature request for bazel to support such tests at
https://github.com/bazelbuild/bazel/issues/11146

Bug: 562274
Change-Id: I873a0e09addc583455b68122f66cd3952e485f0e
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-04-18 18:51:36 +02:00
Minh Thai 0eeed6ad2b Scan through all merged reftables for max/min update indices
Since reftables might have update index ranges that are overlapped.

Change-Id: I8f8215b99a0a978d4dd0155dbaf33e5e06ea8202
Signed-off-by: Minh Thai <mthai@google.com>
(cherry picked from commit 06748c205c)
2020-04-05 17:26:20 -04:00
Alexander Nittka bc4ed530a5 FileUtils: improve delete (Windows)
Ensure files are writable before trying to delete them.

Bug: 408846
Change-Id: I930a547594bba853c33634ae54bd64d236afade3
Signed-off-by: Alexander Nittka <alex@nittka.de>
2020-04-03 19:30:31 +02:00
Thomas Wolf 9aaa58052b Merge branch 'stable-5.7'
* stable-5.7:
  FS.runInShell(): handle quoted filters and hooksPath containing blanks
  Handle non-normalized index also for executable files
  Prepare 5.7.1-SNAPSHOT builds
  JGit v5.7.0.202003110725-r

Change-Id: I8a8580e44bfa05989d476cf22a029abd4fd407c6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-04-03 10:40:30 +02:00
Thomas Wolf 3ae0b5cfeb Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  FS.runInShell(): handle quoted filters and hooksPath containing blanks
  Handle non-normalized index also for executable files

Change-Id: I240377e87c073ee7a621a88e39fc319c59fa037a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-04-03 10:21:00 +02:00
Thomas Wolf 2640d38f14 FS.runInShell(): handle quoted filters and hooksPath containing blanks
Revert commit 2323d7a. Using $0 in the shell command call results in
the command string being taken literally. That was introduced to fix
a problem with backslashes, but is actually not correct.

First, the problem with backslashes occurred only on Win32/Cygwin,
and has been properly fixed in commit 6f268f8.

Second, this is used only for hooks (which don't have backslashes in
their names) and filter commands from the git config, where the user
is responsible for properly quoting or escaping such that the commands
work.

Third, using $0 actually breaks correctly quoted filter commands
like in the bug report. The shell really takes the command literally,
and then doesn't find the command because of quotes.

So revert this change.

At the same time there's a related problem with hooks. If the path to
the hook contains blanks, runInShell() would also fail to find the
hook. In this case, the command doesn't come from user input but is
just a Java File object with an absolute path containing blanks. (Can
occur if core.hooksPath points to such a path with blanks, or if the
repository has such a path.)

The path to the hook as obtained from the file system must be quoted.

Add a test for a hook path with a blank.

This reverts commit 2323d7a1ef.

Bug: 561666
Change-Id: I4d7df13e6c9b245fe1706e191e4316685a8a9d59
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-04-03 08:18:39 +02:00
Thomas Wolf 24f82b533a Handle non-normalized index also for executable files
Commit 60cf85a4 corrected the handling of check-in for files where
the index version is non-normalized, i.e., contains CR-LF line endings.
However, it did so only for regular files, not executable files.

Bug: 561438
Change-Id: I372cc990c5efeb00315460f36459c0652d5d1e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-03-25 09:50:12 +01:00
Minh Thai 06748c205c Scan through all merged reftables for max/min update indices
Since reftables might have update index ranges that are overlapped.

Change-Id: I8f8215b99a0a978d4dd0155dbaf33e5e06ea8202
Signed-off-by: Minh Thai <mthai@google.com>
2020-03-20 11:12:34 -07:00
Ivan Frade 7d3b6308fc ResolveMerger: Ignore merge conflicts if asked so
The recursive merge strategy builds a virtual ancestor merging
recursively the common bases (when more than one) between the
want-to-merge commits. While building this virtual ancestor, content
conflicts are ignored, but current code doesn't do so when a file is
removed.

This was spotted in [1], for example. Merging two commits to build the
virtual ancestor bumped into a conflict (modified in one side, deleted
in the other) that stopped the process.

Follow the "spec" and in case of conflict leave the unmerged content in
the index and working trees.

[1] https://android-review.googlesource.com/c/kernel/common/+/1228962

Change-Id: Ife9c32ae3ac3a87d3660fa1242e07854b65169d5
Signed-off-by: Ivan Frade <ifrade@google.com>
2020-03-19 16:36:21 -07:00
Christian Halstrick d22a8a2510 Merge "tag option for clone command" 2020-03-15 18:11:14 -04:00
Alexander Nittka 769606d059 tag option for clone command
Allow explicitly setting the tag option for the remote configuration
when cloning a repository.

Bug: 561021
Change-Id: Iac43268a2bb231ae7599c3255bf555883d34fa32
Signed-off-by: Alexander Nittka <alex@nittka.de>
2020-03-15 15:34:21 +01:00
Alex Spradlin cf8c84c6d1 RevWalk: fix bad topo flags error message
The error message for an Exception thrown by StartGenerator when given
both the TOPO flag and the TOPO_KEEP_BRANCH_TOGETHER flag mentions a
non-existent flag, TOPO_NON_INTERMIX. The error message was introduced
in commit e498d43.

Replace TOPO_NON_INTERMIX with TOPO_KEEP_BRANCH_TOGETHER in the error
message of an Exception thrown by the StartGenerator when the TOPO flag
is provided together with the TOPO_KEEP_BRANCH_TOGETHER flag.

Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Id24640dc08e96a196508fe38ce144aa7e035082f
2020-03-12 09:04:36 -07:00
Alex Spradlin e498d43186 RevWalk: new topo sort to not mix lines of history
The topological sort algorithm in TopoSortGenerator for RevWalk may mix
multiple lines of history, producing results that differ from C git's
git-log whose man page states: "Show no parents before all of its
children are shown, and avoid showing commits on multiple lines of
history intermixed." Lines of history are mixed because
TopoSortGenerator merely delays producing a commit until all of its
children have been produced; it does not immediately produce a commit
after its last child has been produced.

Therefore, add a new RevSort option called TOPO_KEEP_BRANCH_TOGETHER
with a new topo sort algorithm in TopoNonIntermixGenerator. In the
Generator, when the last child of a commit has been produced, unpop
that commit so that it will be returned upon the subsequent call to
next(). To avoid producing duplicates, mark commits that have not yet
been produced as TOPO_QUEUED so that when a commit is popped, it is
produced if and only if TOPO_QUEUED is set.

To support nesting with other generators that may produce the same
commit multiple times like DepthGenerator (for example, StartGenerator
does this), do not increment parent inDegree for the same child commit
more than once.

Commit b5e764abd2 modified the existing
TopoSortGenerator to avoid mixing lines of history, but it was reverted
in e40c38ab08 because the new behavior
caused problems for EGit users. This motivated adding a new Generator
for the new behavior.

Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Icbb24eac98c00e45c175b01e1c8122554f617933
2020-03-11 15:39:38 -07:00
Matthias Sohn 0d78edad53 Prepare 5.7.1-SNAPSHOT builds
Change-Id: I29cd1b9872603810f9c4d42424ac326fedf5569f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-11 13:35:23 +01:00
Matthias Sohn 0713f6e962 JGit v5.7.0.202003110725-r
Change-Id: Ia4a9122eeb4eea0f379f6611984cfe1143a32f8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-11 13:24:54 +01:00
Matthias Sohn d408e41e4e Prepare 5.7.1-SNAPSHOT builds
Change-Id: I23748c20b69a1cfb47c5cd0d7858182d376b3ce1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 23:37:43 +01:00
Matthias Sohn 8fab712f75 Prepare 5.8.0-SNAPSHOT builds
Change-Id: I056b45806a82eae80177932e42e3dc806015351a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 22:47:10 +01:00
Matthias Sohn f5793bea53 JGit v5.7.0.202003090808-r
Change-Id: Ia23bffa3952687f6107d496a8932d54b11c12412
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 14:09:24 +01:00
Matthias Sohn fb0858e9c9 Merge branch 'stable-5.6'
* stable-5.6:
  Silence API errors introduced by 093fbbd1
  Bump Bazel version to 2.2.0
  Expose FileStoreAttributes.setBackground()
  Update reftable storage repo layout

Change-Id: I237eaaed7991e8bbd56a7624f47bbba985330026
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-05 15:44:42 +01:00
Michael Dardis d7f304045a Add validation to hex decoder
Does not fix any issue but prevents user from shooting themselves in the
foot with improper configuration.

Suggested by Demetr Starshov at https://git.eclipse.org/r/#/c/157681/

Change-Id: I006d65022f0a7d4066970825d00080c59404fdc3
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-04 16:34:01 +01:00
Han-Wen Nienhuys 30fb4808f2 Update reftable storage repo layout
The change Ic0b974fa (c217d33, "Documentation/technical/reftable:
improve repo layout") defines a new repository layout, which was
agreed with the git-core mailing list.

It addresses the following problems:

* old git clients will not recognize reftable-based repositories, and
  look at encompassing directories.

* Poorly written tools might write directly into
  .git/refs/heads/BRANCH.

Since we consider JGit reftable as experimental (git-core doesn't
support it yet), we have no backward compatibility. If you created a
repository with reftable between mid-Nov 2019 and now, you can do the
following to convert:

  mv .git/refs .git/reftable/tables.list
  git config core.repositoryformatversion 1
  git config extensions.refStorage reftable

Change-Id: I80df35b9d22a8ab893dcbe9fbd051d924788d6a5
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-04 15:45:01 +01:00
Jonathan Nieder fb45137a7c Merge "ObjectWalk: Add null check before skip tree." 2020-03-02 13:37:59 -05:00
Jonathan Nieder 6586c171b3 Merge "Revert "RevWalk: stop mixing lines of history in topo sort"" 2020-02-26 17:48:18 -05:00
Yunjie Li 7ba261fb5f ObjectWalk: Add null check before skip tree.
currVisit could be null if a blob is marked as start point in
ObjectWalk. Add null check before skipping current tree.

Change-Id: Ic5d876fe2800f3373d136979be6c27d1bbd38dc1
Signed-off-by: Yunjie Li <yunjieli@google.com>
2020-02-26 12:03:58 -08:00
Alex Spradlin e40c38ab08 Revert "RevWalk: stop mixing lines of history in topo sort"
This reverts commit b5e764abd2.

PlotWalk uses the TopoSortGenerator, which is causing problems for EGit users
who rely on the emission of commits being somewhat based on date as in the
previous topo-sort algorithm.

Bug: 560529
Change-Id: I3dbd3598a7aeb960de3fc39352699b4f11a8c226
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
2020-02-26 14:47:20 -05:00
Matthias Sohn 25a6bd4d61 Merge branch 'stable-5.6'
* stable-5.6:
  Revert "Prepend hostname to subsection used to store file timestamp resolution"
  SimilarityRenameDetector: Fix inconsistent indentation
  Use indexOf(char) and lastIndexOf(char) rather than String versions
  Reorder modifiers to follow Java Language Specification
  GitmoduleEntry: Remove redundant import of class from same package
  Remove redundant "static" qualifier from enum declarations

Change-Id: Ibb66bef7e8373f81e3e653c9843d986243446d68
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-22 23:36:08 +01:00
Michael Dardis 67b9effc65 Remove use of org.bouncycastle.util.encoders.Hex
Change-Id: I5c1ed0397ef99eb5d4f120da331b66c2d0f1707a
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-22 00:12:20 +01:00
Michael Dardis 4055522359 Remove use of org.bouncycastle.util.io.TeeOutputStream
Bug: 559106
Change-Id: Ife06f21a00d258780a2030745738194098c631cf
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-21 23:45:43 +01:00
David Pursehouse 064834d350 Reorder modifiers to follow Java Language Specification
The Java Language Specification recommends listing modifiers in
the following order:

1. Annotations
2. public
3. protected
4. private
5. abstract
6. static
7. final
8. transient
9. volatile
10. synchronized
11. native
12. strictfp

Not following this convention has no technical impact, but will reduce
the code's readability because most developers are used to the standard
order.

This was detected using SonarLint.

Change-Id: I9cddecb4f4234dae1021b677e915be23d349a380
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:15:43 +09:00
David Pursehouse 9be93b7991 Remove redundant "static" qualifier from enum declarations
Nested enum types are implicitly static.

Change-Id: Id3d7886087494fb67bc0d080b4a3491fb4baac19
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:04:43 +09:00
Matthias Sohn 9e4781878b Merge branch 'stable-5.7'
* stable-5.7:
  Update API problem filter
  Prepare 5.6.2-SNAPSHOT builds
  JGit v5.6.1.202002131546-r
  Simplify ReftableCompactor

Change-Id: I1aa650e030fbaa7a1c898eef1836de529e0c30db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-19 01:45:18 +01:00
Matthias Sohn 7a940542ee Merge branch 'stable-5.6' into stable-5.7
# By Matthias Sohn (3) and Han-Wen Nienhuys (1)
* stable-5.6:
  Update API problem filter
  Prepare 5.6.2-SNAPSHOT builds
  JGit v5.6.1.202002131546-r
  Simplify ReftableCompactor

Change-Id: I16ed174f9fc662934c3ebaea85a60690efbed1c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-19 01:44:18 +01:00
Jonathan Tan deab5e1a43 Merge "RevWalk: stop mixing lines of history in topo sort" 2020-02-18 18:41:03 -05:00
Alex Spradlin b5e764abd2 RevWalk: stop mixing lines of history in topo sort
The topological sort algorithm in TopoSortGenerator for RevWalk may mix
multiple lines of history, producing results that differ from C git's
git log whose man page states: "Show no parents before all of its
children are shown, and avoid showing commits on multiple lines of
history intermixed." Lines of history are mixed because
TopoSortGenerator merely delays a commit until all of its children have
been produced; it does not immediately produce a commit after its last
child has been produced.

Therefore, when the last child of a commit has been produced, unpop the
commit so that it will be returned upon the subsequent call to next() in
TopoSortGenerator. To avoid producing duplicates, mark commits that
have not yet been produced as TOPO_QUEUED so that when a commit is
popped, it is produced if and only if TOPO_QUEUED is set.

To support nesting with other generators that may produce the same
commit multiple times like DepthGenerator (for example, StartGenerator
does this), do not increment parent inDegree for the same child commit
more than once.

Modify tests that assert that TopoSortGenerator mixes lines of commit
history.

Change-Id: I4ee03c7a8e5265d61230b2a01ae3858745b2432b
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
2020-02-18 15:17:23 -08:00
Andrey Loskutov 5f53de2bf1 Removed unused imports
This fixes compilation errors in 4.15 M3

Bug: 560210
Change-Id: I7f36edc7fb83c66c8889d7573b4439d5a52334e4
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2020-02-17 09:05:53 +01:00
Matthias Sohn 2899483e96 Prepare 5.6.2-SNAPSHOT builds
Change-Id: Ie04d749bf16eea6db5e5c98be54ba093ca249d05
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-14 00:15:39 +01:00
Matthias Sohn 1b44f0e4a6 JGit v5.6.1.202002131546-r
Change-Id: I2807b9a9f7e8b877b72702ebd1f70c4f9f223481
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-13 21:45:37 +01:00
Han-Wen Nienhuys 753fbaf11a Simplify ReftableCompactor
The ReftableCompactor supported a byteLimit, but this is currently
unused. The FileReftableStack has a more sophisticated strategy that
amortizes compaction costs.

Rename min/maxUpdateIndex to reflogExpire{Min,Max}UpdateIndex to
reflect their purpose  more accurately.

Since reflogs are generally pruned chronologically (oldest entries are
expired first), one can only prune entries on full compaction, so they
should not be set by default.

Rephrase the function Reader#minUpdateIndex and maxUpdateIndex. These
vars are documented to affect log entries, but semantically, they are
about ref entries. Since ref entries have their timestamps
delta-compressed, it is important for the min/maxUpdateIndex values to
be coherent between different tables.

The logical timestamps for log entries do not have to be coherent in
different tables, as the timestamps of a log entry is part of the key.
For example, a table written at update index 20 may contain a tombstone
log entry at timestamp 1.

Therefore, we set ReftableWriter's min/maxUpdateIndex from the merged
tables we are compacting, rather than from the compaction settings
(which should only control reflog expiry.)

The previous behavior could drop log entries erroneously, especially
in the presence of tombstone log entries. Unfortunately, testing this
properly requires both an API for adding log tombstones, and a more
refined API for controlling automatic compaction. Hence, no test.

Change-Id: I2f4eb7866f607fddd0629809e8e61f0b9097717f
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2020-02-12 23:08:20 +01:00
Matthias Sohn 003e4cfc23 Prepare 5.3.8-SNAPSHOT builds
Change-Id: I64ef3b4cf30e87d058001c2c6f1458f4b0c1df00
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 11:54:11 +01:00
Matthias Sohn 0c3b425099 JGit v5.3.7.202002110540-r
Change-Id: I3ba41af516b9d3b74ac580b3e170ad0e6e0663d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 11:37:34 +01:00
Matthias Sohn 425e2607fe Prepare 5.1.14-SNAPSHOT builds
Change-Id: Ic7d6f1a6c3123af1a1fa782f052b0cea3b6f28c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 11:12:20 +01:00
Matthias Sohn e0abe9e2a7 JGit v5.1.13.202002110435-r
Change-Id: Iedc8699ad1a24efe7ddb47ae919c75b9d36141d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 10:30:36 +01:00
Matthias Sohn e68e0e7f03 Merge branch 'stable-5.6'
* stable-5.6:
  reftable: don't check deadline on the first try
  reftable: clarify comment
  reftable: clear cache on full compaction
  reftable: remove outdated comment
  reftable: clarify that LogCursor may return a null ReflogEntry

Change-Id: I9458a746311984fa687b3da964805e2568ed37f3
2020-02-11 01:39:27 +01:00
Han-Wen Nienhuys 8c9f7656c3 reftable: clear cache on full compaction
The merged table contains handles to open files. A full compaction
causes those files to be closed, and so further lookups would fail
with EBADF.

Change-Id: I7bb74f7228ecc7fec9535b00e56a617a9c18e00e
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 00:03:09 +01:00
Matthias Sohn 009674e99f Merge branch 'stable-5.6'
* stable-5.6:
  Restore behavior of CloneCommand

Change-Id: I3092bf214c41436b57e0ede9d2202f8aabf15471
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-10 10:48:39 +01:00
Thomas Wolf 75a80c5d3c Restore behavior of CloneCommand
Commit 6216b0de changed the behavior of the setMirror(),
setCloneAllBranches(), and setBranchesToClone() operations. Before
that commit, these could be set and reset independently and only in
call() it would be determined what exactly to do. Since that commit,
the last of these calls would determine the operation. This means
that the sequence

  cloneCommand.setCloneAllBranches(true);
  cloneCommand.setBranchesToClone(/* some list of refs */);

would formerly do a "clone all" giving a fetch refspec with wildcards

  +refs/heads/*:refs/remotes/origin/*

which picks up new upstream branches, whereas since commit 6216b0de
individual non-wildcard fetch refspecs would be generated and new
upstream branches would not be fetched anymore.

Undo this behavioral change. Make the operations independently settable
and resettable again, and determine the exact operation only in call():
mirror=true > cloneAll=true > specific refs, where ">" means "takes
precedence over", and if none is set assume cloneAll=true.

Note that mirror=true implies setBare(true).

Bug: 559796
Change-Id: I7162b60e99de5e3e512bf27ff4113f554c94f5a6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-02-04 08:49:07 +01:00
Matthias Sohn eaff0a2eba Merge branch 'stable-5.7'
* stable-5.7:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I0538bcba259f7229790a602ac9de120464a1260d
2020-02-01 02:17:49 +01:00
Matthias Sohn 7e3e740cc7 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I79d615dff66493b60d3a4bcbdc57b9455e8d6673
2020-02-01 02:09:37 +01:00
Matthias Sohn 3d59d1b80c Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I73d16b53df02bf735c2431588143efe225a4b5b4
2020-02-01 02:06:47 +01:00
Matthias Sohn 68b0645a2e Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I76a62da98182f0c504b1ea8b7d37cecdf4eea7e0
2020-02-01 01:53:27 +01:00
Matthias Sohn 195b8fc94c Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I12002dbfed9dff14fc6d2df9787d92eab5b1fa78
2020-02-01 01:28:40 +01:00
Matthias Sohn 48dd3385ab Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
2020-02-01 01:14:01 +01:00
David Pursehouse f57a7bd230 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows

Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-01 01:09:46 +01:00
Alex Jitianu 64715a189f FS: Don't use innocuous threads for CompletableFuture
The default threads from the ForkJoinPool run without any privileges
when a SecurityManager is installed, leading to SecurityExceptions
when trying to create the probe file even if the application otherwise
has write privileges in the directory.

Use a dedicated ThreadPoolExecutor using daemon threads instead.

Bug: 551690
Change-Id: Id5376f09f0d7da5ceea367e1f0dfc70f823d62d3
Signed-off-by: Alex Jitianu <alex_jitianu@sync.ro>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-01-28 09:52:14 +01:00
David Pursehouse 4cc13297cc ErrorProne: Enable and fix UnusedException check
Enable UnusedException at ERROR level which causes the build to fail
in many places with:

  [UnusedException] This catch block catches an symbol and re-throws
  another, but swallows the caught symbol rather than setting it as a
  cause. This can make debugging harder.

Fix it by setting the caught exception as cause on the subsequently
thrown exception.

Note: The grammatically incorrect error message is copy-pasted as-is
from the version of ErrorProne currently used in Bazel; it has been
fixed by [1] in the latest version.

[1] https://github.com/google/error-prone/commit/d57a39c

Change-Id: I11ed38243091fc12f64f1b2db404ba3f1d2e98b5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-27 14:40:08 +01:00
Matthias Sohn 8ada9048c5 Replace ExpectedException which was deprecated in junit 4.13
Change-Id: I64b0c057dd0a12aef2f3d56fa0c8a10e3b23fffd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-24 15:46:08 +09:00
Matthias Sohn c49c0f9912 Replace deprecated junit assertion methods with hamcrest
Change-Id: Icebe8071eeefbc13d3bdbc231c568209cdc26195
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-22 18:16:17 +01:00
David Pursehouse c02a57f4fa Update to Orbit I20200120214610 and JUnit to 4.13
Since version 4.13 JUnit has an assertThrows method. Remove the
implementation in MoreAsserts and use the one from JUnit.

CQ: 21439
Change-Id: I086baa94aa3069cebe87c4cbf91ed1534523c6cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-22 18:16:14 +01:00
Matthias Sohn 709f83d489 WindowCache: add option to use strong refs to reference ByteWindows
Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.

Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.

If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.

Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-20 17:53:08 +01:00
Matthias Sohn 9f1dad4d4f Prepare 5.7.0-SNAPSHOT builds
Change-Id: I106098e85db05bb0142f9fb406e3aef3cf0238b4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-15 22:15:27 +01:00
Matthias Sohn 3c527a8a37 JGit v5.7.0.202001151323-m1
Change-Id: I7701bee6f05d66012c59092126eb4d52f828dc24
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-15 19:22:36 +01:00
Matthias Sohn db019c473e Merge branch 'stable-5.6'
* stable-5.6:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I4a77c602bfffed60535d0b8fc251a552b128068d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 16:52:56 +01:00
Matthias Sohn d7304eddaf Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I11f9a387ac3dc7d22a4f2e70bb8d89169b4e9afe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 13:15:12 +01:00
Matthias Sohn 5cfa74c7b1 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I78902d5feecb2c09134b64ec2f3b48b2c3bab37d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 12:32:03 +01:00
Matthias Sohn 4076e6aa3e Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I1b560b36d169cfa02cc5450ad0fa0bd85f9f42d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 11:38:28 +01:00
Matthias Sohn c5c1bbbe49 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 02:19:03 +01:00
Matthias Sohn 6f177b3f6c Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: Ic90aacf1ea40e13dc564d4d659e79535e86d0300
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 02:03:45 +01:00
Matthias Sohn b2bf9d4304 Remove duplicate copyright statements
Change-Id: I9a85136e1cc47693d1e475d00916c214c9aa55f4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-04 01:34:46 +01:00
Matthias Sohn 5c5f7c6b14 Update EDL 1.0 license headers to new short SPDX compliant format
This is the format given by the Eclipse legal doc generator [1].

[1] https://www.eclipse.org/projects/tools/documentation.php?id=technology.jgit

Bug: 548298
Change-Id: I8d8cabc998ba1b083e3f0906a8d558d391ffb6c4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-04 01:10:05 +01:00
Carsten Hammer 74bc50125d Replace chain of if statements with switch
and switch over strings where possible. Sometimes if statements are
chained and form a series of comparisons against constants. Using switch
statements improves readability.

Bug: 545856
Change-Id: Iacb78956ee5c20db4d793e6b668508ec67466606
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-14 20:42:03 +01:00
Matthias Sohn 4cb80f897f Prepare 5.6.1-SNAPSHOT builds
Change-Id: Iaa72d2ea6764ccd4fb6a124b51d89fe6492c602d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-11 02:04:21 +01:00
Matthias Sohn 9710c6292a JGit v5.6.0.201912101111-r
Change-Id: Icbb3b46f9d04e45da53936860e07e69fde12971c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-10 17:35:05 +01:00
Matthias Sohn 42f0c7c9cb Enhance WindowCache statistics
Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time

Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.

Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.

Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-08 01:09:24 +01:00
David Pursehouse 8a633bde8a Merge branch 'stable-5.6'
* stable-5.6:
  Cleanup CommitAndLogCommandTest
  CLI: Add support for excluding paths from Git log command

Change-Id: I2f48e16640ab1b13f0c32779422add3162260c50
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-06 10:20:07 +09:00
Matthias Sohn 02dc0ca688 Cleanup CommitAndLogCommandTest
- if a test can throw different exceptions declare it throws Exception
- fix code formatting

Change-Id: I55d63918f3163b31f2297d6217d5855108dd43b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 22:48:55 +01:00
John Tipper 5bf226d50c CLI: Add support for excluding paths from Git log command
Git log supports the exclude pathspec, which allows for excluding paths
from the log command. JGit only supports adding paths to the log
command. See the following StackOverflow question for details:
https://stackoverflow.com/questions/59143934/java-jgit-how-to-get-git-
commits-not-affecting-certain-directories
This commit adds an excludePath() method to the log command. It does not
yet support regex or glob wildcards.

Change-Id: I8cd59950b87850b55a15c7e2ea5470145c9aae28
Signed-off-by: John Tipper <john_tipper@hotmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 22:38:12 +01:00
David Pursehouse f350e889d6 Merge branch 'stable-5.6'
* stable-5.6:
  ReftableTest: Clean up boxing warnings on usage of String.format
  MergedReftable: Remove unnecessary semicolon

Change-Id: Id35406acfdd91c3ddf17ce4e1023395f774774f2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-05 10:46:44 +09:00
David Pursehouse b24ec8f715 ReftableTest: Clean up boxing warnings on usage of String.format
Passing int as an argument to String.format causes a warning:

  The expression of type int is boxed into Integer

Most of these are already suppressed, but there are a couple that are
not. Add suppressions for those.

For the existing ones, move the suppression scope from the method to
the actual usage. Where necessary extract the usage out to a local
variable.

Change-Id: I7a7ff6dec49467e4b5c58d27a231c74e6e1c5437
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-05 09:54:14 +09:00
Matthias Sohn f487a9eb2e Prepare 5.7.0-SNAPSHOT builds
Change-Id: I7efa0cd08dee711d6fefc3e6bb478a3cb11fe85c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:32:59 +01:00
Matthias Sohn 08d4c63b6d Merge branch 'stable-5.6'
* stable-5.6:
  Introduce RefDatabase#hasFastTipsWithSha1
  Update Orbit to R20191126223242 for 2019-12
  Prepare 5.6.0-SNAPSHOT builds
  JGit v5.6.0.201912041214-rc1
  Prepare 5.6.0-SNAPSHOT builds
  JGit v5.6.0.201911271000-m3

Change-Id: Id33b054131bbcd27a0ae7200f564775049698d2c
2019-12-05 01:31:32 +01:00
Han-Wen Nienhuys 5185d288c1 Introduce RefDatabase#hasFastTipsWithSha1
The reftable format supports fast inverse (SHA1 => ref) queries.

If the ref database does not support fast inverse queries, it may be
advantageous to build a complete SHA1 to ref map in advance for
multiple uses. To let applications decide, this function indicates
whether the inverse map is available.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idaf7e01075906972ec21332cade285289619c2b3
2019-12-04 23:35:32 +01:00
Matthias Sohn 473527e42c Prepare 5.6.0-SNAPSHOT builds
Change-Id: I5e737ff7f262fdd43fc975a0b3594c8b33919663
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 22:13:56 +01:00
Matthias Sohn a7e454bc51 JGit v5.6.0.201912041214-rc1
Change-Id: I5ed21fbc5f83096bf0b79f2aa751db415cbcc7e8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 18:14:43 +01:00
Masaya Suzuki 6bc3665764 transport: Move handling code to the caller side
This removes a raw IOException at one level. Later we'll add a custom
exception handling mechanism like UploadPack.

Change-Id: I52a7423798c97b032d848351be8b6f144776b017
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 12:25:03 -08:00
Matthias Sohn 4ea42bc816 Prepare 5.6.0-SNAPSHOT builds
Change-Id: I1e52d2dfb202b87ecb9d0273deaa2c8d8ce1864e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 21:25:38 +01:00
Matthias Sohn 6c9aa82f70 JGit v5.6.0.201911271000-m3
Change-Id: I3810892c2bed947e4dfaa615a7b4d93eeb46abb9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 16:01:20 +01:00
Thomas Wolf 59f9d206c9 Make blame work correctly on merge conflicts
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.

Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].

Implement that and add tests.

I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.

(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)

[1] https://github.com/git/git/blob/v2.22.0/blame.c#L174

Bug: 434330
Change-Id: I9d763dd6ba478b0b6ebf9456049d6301f478ef7c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-27 03:03:06 +01:00
Jonathan Nieder aab5ee18fe Merge changes I8698001d,I5c1177ac
* changes:
  Add missing license header to ReftableDatabase
  Add missing license header to UploadPackRefSortingForReachabilityTest
2019-11-25 20:56:23 -05:00
Jonathan Nieder 3e01123a53 Add missing license header to UploadPackRefSortingForReachabilityTest
Forgot to add this in commit 989a927a5f
(checkNotAdvertisedWants: Be lazy convering Ref to RevCommit,
2019-11-20).

Change-Id: I5c1177ac60eabb3a71959bbad4537e076a901b7e
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-11-25 17:37:40 -08:00
Matthias Sohn 3d4c95293c [spotbugs] Fix potential NPE in FSTest
Change-Id: I6ca37d045642e3aada7e6b42b3cb8c3bba014455
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-26 01:57:20 +01:00
Han-Wen Nienhuys 9cb8e2f310 RepositoryCache: don't require HEAD in git repositories
Reftable-enabled repositories don't have a file called HEAD. Check for
reftable/ instead.

This fixes repository creation on reftable in Gerrit.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I778c2be01d96aaf135affae4b457b5fe5b483bee
2019-11-26 01:57:19 +01:00
Han-Wen Nienhuys e0744891fe FileRepository: cleanup refs outside refs/ on reftable conversion
Change-Id: Iab7d3a08906e826e26572f534512a09d3a5876b0
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-26 01:57:19 +01:00
David Pursehouse f2ccc42132 UploadPackTest: Fix unused parameter in checkUnadvertisedIfUnallowed
Change-Id: I6042c401fff55325128d47e7e9098fc7e60d501d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-25 09:31:49 +09:00
Ivan Frade f5f5c80bf5 BitmappedReachabilityChecker: Use only one bitmap for the whole check
The checker is creating a new bitmap per branch leading to excessive
memory consumption. For the reachability check one bitmap with the
reachability of all branches aggregated is enough.

Build the reachability bitmap with a filter. The filter itself uses it
to emit only commits not reached before and the caller to check what
targets have been reached already.

BitmapCalculator is not required anymore.

Change-Id: Ic5c62f77fe0f188913215b7eaa51d849a9aae6a5
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 14:07:04 -08:00
Ivan Frade 989a927a5f checkNotAdvertisedWants: Be lazy converting Ref to RevCommit
The ref points to an ObjectId that then is translated into a RevCommit.
This translation can be costly and with the incremental reachability
check is probably not needed for most of the elements.

Delay the translation from ObjectId to RevCommit to when it is needed.
Use Streams, that have the laziness built-in, all the way from Ref to
RevCommit.

This should reduce the latency for reachability checks over big sets of
references.

Change-Id: I28693087321b2beff3eaa1f3d2e7840ab0eedc6d
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 09:30:19 -08:00
Ivan Frade a0204a4727 ReachabilityChecker: Receive a Stream instead of a Collection
Preparatory change. Converting ObjectIds to RevCommits is potentially
expensive and in the incremental reachability check, it is probably not
required for all elements in the collection.

Pass a Stream to the reachability checker. In the follow up we make
the conversion from ObjectId to RevCommit in the stream (i.e. on
demand). This should reduce the latency of reachability checks over big
sets of references.

Change-Id: I9f310e331de5b0bf8de34143bd7dcd34316d2fba
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 09:30:19 -08:00
Han-Wen Nienhuys 38586d54d0 file: implement FileReftableDatabase
Reftable is a binary, block-based storage format for the ref-database.
It provides several advantages over the traditional packed + loose
storage format:

 * O(1) write performance, even for deletions and transactions.

 * atomic updates to the ref database.

 * O(log N) lookup and prefix scans

 * free from restrictions imposed by the file system: it is
   case-sensitive even on case-insensitive file systems, and has
   no inherent limitations for directory/file conflicts

 * prefix compression reduces space usage for repetitive ref names,
   such as gerrit's refs/changes/xx/xxxxx format.

FileReftableDatabase is based on FileReftableStack, which does
compactions inline. This is simple, and has good median performance,
but every so often it will rewrite the entire ref database.

For testing, a FileReftableTest (mirroring RefUpdateTest) is added to
check for Reftable specific behavior. This must be done separately, as
reflogs have different semantics.

Add a reftable flavor of BatchRefUpdateTest.

Add a FileReftableStackTest to exercise compaction.

Add FileRepository#convertToReftable so existing testdata can be
reused.

CQ: 21007
Change-Id: I1837f268e91c6b446cb0155061727dbaccb714b8
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-16 23:16:01 -08:00
Thomas Wolf 1daf6f13aa RewriteGenerator: avoid adding null parent
Prevent adding a null parent to a commit's parent array. Doing so
can cause NPEs later on.

Bug: 552160
Change-Id: Ib24b7b9b7b08e0b6f246006b4a4cade7eeb830b9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 16:00:48 -08:00
Thomas Wolf 345e2648df DiffFormatter: support core.quotePath = false
core.quotePath = false means that "bytes higher than 0x80 are not
considered "unusal" anymore"[1], i.e., they are not escaped. In
essence this preserves non-ASCII characters in path names in output.

Note that control characters and other special characters in the
ASCII range will still be escaped.

Add a new QuotedString.GIT_PATH_MINIMAL singleton implementing this.
Change the normal GIT_PATH algorithm to use bytes instead of characters
so it can be re-used. Provide a setter in DiffFormatter for the quoting
style so that an application can override the default, which is the
setting from the git config (and by default "true"). Use the new
QuotedString.GIT_PATH_MINIMAL when core.quotePath == false.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-corequotePath

Bug: 552467
Change-Id: Ifcb233e7d10676333bf42011e32d01a4e1138059
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 15:12:23 -08:00
Thomas Wolf 7a3b93cbed IndexDiff/SubmoduleWalk: handle submodule.<name>.ignore correctly
IndexDiff would apply ignore mode ALL from .gitmodules to all remaining
submodules, and would ignore other settings from .gitignore and always
apply the setting defined on the IndexDiff instead. Correct that.

In canonical git the ignore setting from .gitmodules can also be
overridden by .git/config.[1] Implement that override in SubmoduleWalk.

[1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtignore

Bug: 521613
Change-Id: I9199fd447e41c7838924856dce40678370b66395
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 14:48:44 -08:00
Jonathan Nieder 83b9b84e4a Merge changes I583491f6,Idd44e3bb
* changes:
  Run Eclipse formatter on BatchRefUpdateTest
  Move KetchSystem.delay to FileUtils.
2019-11-14 18:21:04 -05:00
Han-Wen Nienhuys 63fba337b2 Run Eclipse formatter on BatchRefUpdateTest
Change-Id: I583491f621e54401c60150efb285fcc7243a9355
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-11-14 13:48:28 -08:00
Masaya Suzuki 3e9a5f993b transport: Merge BaseReceivePack into ReceivePack
Move the BaseReceivePack implementation back into ReceivePack. This is a
backward-incompatible change. For example, BaseReceivePack.FirstLine no
longer exists and cannot be referenced.  However, most of the code
should just work by replacing BaseReceivePack with ReceivePack.

Although this is an API change, it only affects callers using JGit as a
server, and there are very few of those in the wild.

Change-Id: I1ce92869435d5eebb7d671be44561e69c6233134
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-11-13 15:57:04 -08:00
Matthias Sohn 826317942f Merge branch 'stable-5.5'
* stable-5.5:
  BaseReceivePack: Fix the format
  Prepend hostname to subsection used to store file timestamp resolution
  Store filesystem timestamp resolution in extra jgit config
  SystemReader: extract updating config and its parents if outdated

Change-Id: Iecfddce8081303af29badcdcd3d72a0da50c964f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-11 11:07:17 -08:00
Matthias Sohn 838b5a84b5 Store filesystem timestamp resolution in extra jgit config
This avoids polluting hand-crafted user level config with
auto-configured options which might disturb in environments where
the user level config is replicated between different machines.

Add a jgit config as parent of the system level config. Persist
measured timestamp resolutions always in this jgit config and read it
via the user global config. This has the effect that auto-configured
timestamp resolution will be used by default and can be overridden in
either the system level or user level config.

Store the jgit config under the XDG_CONFIG_HOME directory following the
XDG base directory specification [1] in order to ensure that we have
write permissions to persist the file. This has the effect that each OS
user will use its jgit config since they typically use different
XDG_CONFIG_HOME directories.

If the environment variable XDG_CONFIG_HOME is defined the jgit config
file is located at $XDG_CONFIG_HOME/jgit/config otherwise the default is
~/.config/jgit/config.

If you want to avoid redundant measurement for different OS users
manually copy the values measured and auto-configured for one OS user to
the system level git config.

[1] https://wiki.archlinux.org/index.php/XDG_Base_Directory

Bug: 551850
Change-Id: I0022bd40ae62f82e5b964c2ea25822eb55d94687
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-09 18:01:19 +01:00
Thomas Wolf 64f2407f19 WorkingTreeModifiedEvent: must be fired explicitly after merge
A merge may write files to the working tree. After a successful
merge one must fire a WorkingTreeModifiedEvent explicitly if
getModifiedFiles() is not empty.

Also, any touched files must be reported by the
WorkingTreeModifiedEvent fired by DirCacheCheckout.checkout().

Bug: 552636
Change-Id: I5fab8279ed8be8a4ae34cddfa726836b9277aea6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-03 10:10:16 +01:00
David Pursehouse b29e9bd1cb Merge "HttpSupport.proxyFor(): use only scheme, host, and port" 2019-10-30 19:20:30 -04:00
Han-Wen Nienhuys 7c75a68b96 reftable: enforce ascending order in sortAndWriteRefs
MergedReftableTest#scanDuplicates tests whether we can write duplicate
keys in a merged reftable. Apparently, the first key appearing should
get precedence, and this works because the sort() algorithm on ordered
collections is stable.

This is potentially confusing behavior, because you can write data
into the table that cannot be retrieved (Merged table can only have
one entry per key), and the APIs such as exactRef() only return a
single value.

Make this consistent with behavior introduced in I04f55c481 "reftable:
enforce ordering for ref and log writes" by considering a duplicate key
in sortAndWriteRefs as a fatal runtime error.

Change-Id: I1eedd18f028180069f78c5c467169dcfe1521157
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-10-30 18:00:24 +01:00
Thomas Wolf b7d1eb4185 HttpSupport.proxyFor(): use only scheme, host, and port
Some URLs cannot be converted via URL.toURI(). So don't convert
the full URL but only the bits that are needed to find a proxy
via java.net.ProxySelector.

Bug: 549690
Change-Id: I55b5ecee70c6b52f72f9bdba9ce552fde7f33976
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-30 15:16:51 +01:00
Thomas Wolf 6a39da37fe GPG: implement more OpenPGP UserId matching formats
Instead of just looking for a substring match of user.signingKey
in a key's user ID implement the GPG matching formats[1] for:

'=' Full exact match
'<' Full exact match of the e-mail address
'@' Substring match within the e-mail address only
'*' General case-insensitive substring match (default)

When user.signingKey is not set, the committer's e-mail address is
used by default. In that case, use '<', i.e., require an exact match
on the OpenPGP e-mail address.

Also handle the optional "0x" prefix for (partial) key fingerprints.

[1] https://www.gnupg.org/documentation/manuals/gnupg/Specify-a-User-ID.html

Bug: 550335
Change-Id: I6ce482a099ff1a0dc9de45435cd4d3ec5b504f12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-23 11:17:24 +02:00
Jonathan Tan 0bf165b7b6 Merge "TreeRevFilterTest: Simplify filter used" 2019-10-22 13:36:46 -04:00