Commit Graph

5551 Commits

Author SHA1 Message Date
Matthias Sohn 8fab712f75 Prepare 5.8.0-SNAPSHOT builds
Change-Id: I056b45806a82eae80177932e42e3dc806015351a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 22:47:10 +01:00
Matthias Sohn f5793bea53 JGit v5.7.0.202003090808-r
Change-Id: Ia23bffa3952687f6107d496a8932d54b11c12412
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 14:09:24 +01:00
Matthias Sohn fb0858e9c9 Merge branch 'stable-5.6'
* stable-5.6:
  Silence API errors introduced by 093fbbd1
  Bump Bazel version to 2.2.0
  Expose FileStoreAttributes.setBackground()
  Update reftable storage repo layout

Change-Id: I237eaaed7991e8bbd56a7624f47bbba985330026
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-05 15:44:42 +01:00
Matthias Sohn 2415571370 Silence API errors introduced by 093fbbd1
Change-Id: I1c9d5a25bd06a1152e953c45b683375cb05aa254
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-05 13:53:10 +01:00
Michael Dardis d7f304045a Add validation to hex decoder
Does not fix any issue but prevents user from shooting themselves in the
foot with improper configuration.

Suggested by Demetr Starshov at https://git.eclipse.org/r/#/c/157681/

Change-Id: I006d65022f0a7d4066970825d00080c59404fdc3
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-04 16:34:01 +01:00
Alex Blewitt 093fbbd11e Expose FileStoreAttributes.setBackground()
The FS.setAsyncFileStoreAttributes() static method calls
FileStoreAttributes.setBackground() as its implementation, but there are
other public attributes on this inner class already and there isn't a
real reason why this needs to be private.

By making it public we allow callers to be able to invoke it directly.
Although it doesn't appear that it would make a difference, by calling a
static method on the FS class, all static fields and the transitive
closure of class dependencies must be loaded and initialised, which can
be non-trivial.

Callers referring to FS.setAsyncFileStoreAttributes() may be replaced
with FS.FileStoreAttributes.setBackground() with no change of behaviour
other than improved performance due to less class loading required.

Bug: 560527
Change-Id: I9538acc90da8d18f53fd60d74eb54496857f93a5
Signed-off-by: Alex Blewitt <alex.blewitt@gmail.com>
2020-03-04 15:59:53 +01:00
Han-Wen Nienhuys 30fb4808f2 Update reftable storage repo layout
The change Ic0b974fa (c217d33, "Documentation/technical/reftable:
improve repo layout") defines a new repository layout, which was
agreed with the git-core mailing list.

It addresses the following problems:

* old git clients will not recognize reftable-based repositories, and
  look at encompassing directories.

* Poorly written tools might write directly into
  .git/refs/heads/BRANCH.

Since we consider JGit reftable as experimental (git-core doesn't
support it yet), we have no backward compatibility. If you created a
repository with reftable between mid-Nov 2019 and now, you can do the
following to convert:

  mv .git/refs .git/reftable/tables.list
  git config core.repositoryformatversion 1
  git config extensions.refStorage reftable

Change-Id: I80df35b9d22a8ab893dcbe9fbd051d924788d6a5
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-04 15:45:01 +01:00
Jonathan Nieder fb45137a7c Merge "ObjectWalk: Add null check before skip tree." 2020-03-02 13:37:59 -05:00
Matthias Sohn 2161c1e5e4 Merge branch 'stable-5.6'
* stable-5.6:
  Cygwin expects forward slashes for commands to be run via sh.exe
  Make Logger instances final
  Move array designators from the variable to the type

Change-Id: I9a5dc570deb478525bf48ef526d8cba5b19418bf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-28 23:53:32 +01:00
Thomas Wolf 6f268f8ceb Cygwin expects forward slashes for commands to be run via sh.exe
FS_Win32_Cygwin replaces backslashes by / as a side-effect of
relativize(). When support for core.hooksPath was added, paths were
relativized in a different place using Path.resolve(), which doesn't
do that transformation. As a result hooks could not be run on Cygwin
in some cases.

Do the transformation in FS_Win32_Cygwin.runInShell(). In all other
places, File or Path objects are used, which give no guarantee about
the file separator (typically the system-dependent default separator),
so doing the transformation earlier still wouldn't guarantee that
sh.exe indeed gets a command string using forward slashes.

Bug: 558577
Change-Id: I3c07eb85f0ac7c5628a2e92f990e5cdb7ecf532f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-02-28 17:18:23 -05:00
Lars Vogel 512ffc9e7f Using for-each loop in jdt
Running the JDT cleanup action for using a for-each loop on jgit

Change-Id: Ie724d8bbdff786ab0167089e90a9914a8135103c
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
2020-02-28 02:39:12 -05:00
David Pursehouse 14a157dff6 Make Logger instances final
Change-Id: Ibb997952917e47bc31a8cbe3863623bc959a8100
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-27 20:32:29 +09:00
David Pursehouse 5a6b6eee35 Move array designators from the variable to the type
As reported by Sonar Lint:

Array designators should always be located on the type for better code
readability. Otherwise, developers must look both at the type and the
variable name to know whether or not a variable is an array.

Change-Id: If6b41fed3483d0992d402d8680552ab4bef89ffb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-27 20:27:31 +09:00
Jonathan Nieder 6586c171b3 Merge "Revert "RevWalk: stop mixing lines of history in topo sort"" 2020-02-26 17:48:18 -05:00
Yunjie Li 7ba261fb5f ObjectWalk: Add null check before skip tree.
currVisit could be null if a blob is marked as start point in
ObjectWalk. Add null check before skipping current tree.

Change-Id: Ic5d876fe2800f3373d136979be6c27d1bbd38dc1
Signed-off-by: Yunjie Li <yunjieli@google.com>
2020-02-26 12:03:58 -08:00
Alex Spradlin e40c38ab08 Revert "RevWalk: stop mixing lines of history in topo sort"
This reverts commit b5e764abd2.

PlotWalk uses the TopoSortGenerator, which is causing problems for EGit users
who rely on the emission of commits being somewhat based on date as in the
previous topo-sort algorithm.

Bug: 560529
Change-Id: I3dbd3598a7aeb960de3fc39352699b4f11a8c226
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
2020-02-26 14:47:20 -05:00
Lajos Olah 61ec1455c0 Do not fail if known hosts file does not contain valid host key
KnownHosts (implementing HostKeyRepository) in Jsch can return null
which could cause NullPointerException in Stream.of(...)

Change-Id: Iddcf5f34f8c8475a85ca7ae018bbe48d1b3fbbc0
Signed-off-by: Lajos Olah <lajos.olah.jr@gmail.com>
2020-02-26 03:57:59 -05:00
Matthias Sohn 25a6bd4d61 Merge branch 'stable-5.6'
* stable-5.6:
  Revert "Prepend hostname to subsection used to store file timestamp resolution"
  SimilarityRenameDetector: Fix inconsistent indentation
  Use indexOf(char) and lastIndexOf(char) rather than String versions
  Reorder modifiers to follow Java Language Specification
  GitmoduleEntry: Remove redundant import of class from same package
  Remove redundant "static" qualifier from enum declarations

Change-Id: Ibb66bef7e8373f81e3e653c9843d986243446d68
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-22 23:36:08 +01:00
Matthias Sohn abb461533f Revert "Prepend hostname to subsection used to store file timestamp resolution"
This reverts commit e102bbed99.

Resolving the hostname comes with a performance penalty. We no longer
store the timestamp resolution in the global git config which might be
copied around to other machines but in a dedicated jgit config meant for
automatically determined options like timestamp resolution. Hence there
is no strong reason anymore to have a hardware specific identifier in
the subsection name of file timestamp resolution options.

Bug: 560414
Change-Id: If8dcabe981eb1792db84643850faa6033f14b1cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-22 22:57:12 +01:00
Michael Dardis 67b9effc65 Remove use of org.bouncycastle.util.encoders.Hex
Change-Id: I5c1ed0397ef99eb5d4f120da331b66c2d0f1707a
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-22 00:12:20 +01:00
Michael Dardis 4055522359 Remove use of org.bouncycastle.util.io.TeeOutputStream
Bug: 559106
Change-Id: Ife06f21a00d258780a2030745738194098c631cf
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-21 23:45:43 +01:00
Thomas Wolf f2b67be62a Make the IMatcher public API
Provide a static factory method to create a PathMatcher.

Bug: 559526
Change-Id: Ib7a4a1bcc658ac2f2a09d365b5b891669dfd7570
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-02-20 10:40:33 +01:00
David Pursehouse fdabbe67e2 SimilarityRenameDetector: Fix inconsistent indentation
Replace space indentation with tab indentation

Change-Id: Ic130d3bde5d3a73d8f5c6225974153573722d05b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 20:34:15 +09:00
David Pursehouse 6a72f2943d Use indexOf(char) and lastIndexOf(char) rather than String versions
An indexOf or lastIndexOf call with a single letter String can be
made more performant by switching to a call with a char argument.

Found with SonarLint.

As a side-effect of this change, we no longer need to suppress the
NON-NLS warnings.

Change-Id: Id44cb996bb74ed30edd560aa91fd8525aafdc8dd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:24:01 +09:00
David Pursehouse 064834d350 Reorder modifiers to follow Java Language Specification
The Java Language Specification recommends listing modifiers in
the following order:

1. Annotations
2. public
3. protected
4. private
5. abstract
6. static
7. final
8. transient
9. volatile
10. synchronized
11. native
12. strictfp

Not following this convention has no technical impact, but will reduce
the code's readability because most developers are used to the standard
order.

This was detected using SonarLint.

Change-Id: I9cddecb4f4234dae1021b677e915be23d349a380
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:15:43 +09:00
David Pursehouse 2d1acae6d8 GitmoduleEntry: Remove redundant import of class from same package
Change-Id: I613489a937dd53780b0abfd6b1bd0ed964a5ddd8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:06:18 +09:00
David Pursehouse 9be93b7991 Remove redundant "static" qualifier from enum declarations
Nested enum types are implicitly static.

Change-Id: Id3d7886087494fb67bc0d080b4a3491fb4baac19
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-02-19 13:04:43 +09:00
Matthias Sohn 9e4781878b Merge branch 'stable-5.7'
* stable-5.7:
  Update API problem filter
  Prepare 5.6.2-SNAPSHOT builds
  JGit v5.6.1.202002131546-r
  Simplify ReftableCompactor

Change-Id: I1aa650e030fbaa7a1c898eef1836de529e0c30db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-19 01:45:18 +01:00
Matthias Sohn 7a940542ee Merge branch 'stable-5.6' into stable-5.7
# By Matthias Sohn (3) and Han-Wen Nienhuys (1)
* stable-5.6:
  Update API problem filter
  Prepare 5.6.2-SNAPSHOT builds
  JGit v5.6.1.202002131546-r
  Simplify ReftableCompactor

Change-Id: I16ed174f9fc662934c3ebaea85a60690efbed1c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-19 01:44:18 +01:00
Alex Spradlin b5e764abd2 RevWalk: stop mixing lines of history in topo sort
The topological sort algorithm in TopoSortGenerator for RevWalk may mix
multiple lines of history, producing results that differ from C git's
git log whose man page states: "Show no parents before all of its
children are shown, and avoid showing commits on multiple lines of
history intermixed." Lines of history are mixed because
TopoSortGenerator merely delays a commit until all of its children have
been produced; it does not immediately produce a commit after its last
child has been produced.

Therefore, when the last child of a commit has been produced, unpop the
commit so that it will be returned upon the subsequent call to next() in
TopoSortGenerator. To avoid producing duplicates, mark commits that
have not yet been produced as TOPO_QUEUED so that when a commit is
popped, it is produced if and only if TOPO_QUEUED is set.

To support nesting with other generators that may produce the same
commit multiple times like DepthGenerator (for example, StartGenerator
does this), do not increment parent inDegree for the same child commit
more than once.

Modify tests that assert that TopoSortGenerator mixes lines of commit
history.

Change-Id: I4ee03c7a8e5265d61230b2a01ae3858745b2432b
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
2020-02-18 15:17:23 -08:00
Matthias Sohn bc98a597ff Update API problem filter
Change-Id: Iced4e3cc68cc9c2a75f9bf5befd6db7ce2fb1374
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-14 09:17:07 +01:00
Matthias Sohn 2899483e96 Prepare 5.6.2-SNAPSHOT builds
Change-Id: Ie04d749bf16eea6db5e5c98be54ba093ca249d05
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-14 00:15:39 +01:00
Matthias Sohn 1b44f0e4a6 JGit v5.6.1.202002131546-r
Change-Id: I2807b9a9f7e8b877b72702ebd1f70c4f9f223481
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-13 21:45:37 +01:00
Han-Wen Nienhuys 753fbaf11a Simplify ReftableCompactor
The ReftableCompactor supported a byteLimit, but this is currently
unused. The FileReftableStack has a more sophisticated strategy that
amortizes compaction costs.

Rename min/maxUpdateIndex to reflogExpire{Min,Max}UpdateIndex to
reflect their purpose  more accurately.

Since reflogs are generally pruned chronologically (oldest entries are
expired first), one can only prune entries on full compaction, so they
should not be set by default.

Rephrase the function Reader#minUpdateIndex and maxUpdateIndex. These
vars are documented to affect log entries, but semantically, they are
about ref entries. Since ref entries have their timestamps
delta-compressed, it is important for the min/maxUpdateIndex values to
be coherent between different tables.

The logical timestamps for log entries do not have to be coherent in
different tables, as the timestamps of a log entry is part of the key.
For example, a table written at update index 20 may contain a tombstone
log entry at timestamp 1.

Therefore, we set ReftableWriter's min/maxUpdateIndex from the merged
tables we are compacting, rather than from the compaction settings
(which should only control reflog expiry.)

The previous behavior could drop log entries erroneously, especially
in the presence of tombstone log entries. Unfortunately, testing this
properly requires both an API for adding log tombstones, and a more
refined API for controlling automatic compaction. Hence, no test.

Change-Id: I2f4eb7866f607fddd0629809e8e61f0b9097717f
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2020-02-12 23:08:20 +01:00
Yunjie Li 34cf825ba6 Change the wildcard import to explicit ones.
Change-Id: Ia7bc1c4b6b82c127e1e4e0e657f69d249684e8e6
Signed-off-by: Yunjie Li <yunjieli@google.com>
2020-02-12 18:45:26 +09:00
Matthias Sohn e68e0e7f03 Merge branch 'stable-5.6'
* stable-5.6:
  reftable: don't check deadline on the first try
  reftable: clarify comment
  reftable: clear cache on full compaction
  reftable: remove outdated comment
  reftable: clarify that LogCursor may return a null ReflogEntry

Change-Id: I9458a746311984fa687b3da964805e2568ed37f3
2020-02-11 01:39:27 +01:00
Han-Wen Nienhuys 79266a1fe5 reftable: don't check deadline on the first try
This helps debug stepping.

Change-Id: I020dafab4ffac75e6df0dbcde0ed4805c7867f72
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2020-02-11 00:03:33 +01:00
Han-Wen Nienhuys 936a031ca3 reftable: clarify comment
Change-Id: I16e32aeb325b604eb31f84db18a214f792395941
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 00:03:12 +01:00
Han-Wen Nienhuys 8c9f7656c3 reftable: clear cache on full compaction
The merged table contains handles to open files. A full compaction
causes those files to be closed, and so further lookups would fail
with EBADF.

Change-Id: I7bb74f7228ecc7fec9535b00e56a617a9c18e00e
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-11 00:03:09 +01:00
Han-Wen Nienhuys dd203f03c2 reftable: remove outdated comment
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I41a7d9934d53553b5f81f40ff9d663378676ac5d
2020-02-10 12:43:36 +01:00
Han-Wen Nienhuys 0bf2564433 reftable: clarify that LogCursor may return a null ReflogEntry
Change-Id: I1a4d5c262cd196dca37876aec00bb974a45e9fcd
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2020-02-10 12:43:36 +01:00
Matthias Sohn 009674e99f Merge branch 'stable-5.6'
* stable-5.6:
  Restore behavior of CloneCommand

Change-Id: I3092bf214c41436b57e0ede9d2202f8aabf15471
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-10 10:48:39 +01:00
Lars Vogel 9d1e477417 Use lambdas where possible
Using the batch cleanup operation via Source -> Cleanup -> "Use lambdas
where possible" from standard JDT.

Change-Id: I5452bd94fdccc920ade071228aeed3a8b9fdbe62
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
2020-02-05 06:11:06 -05:00
Thomas Wolf 75a80c5d3c Restore behavior of CloneCommand
Commit 6216b0de changed the behavior of the setMirror(),
setCloneAllBranches(), and setBranchesToClone() operations. Before
that commit, these could be set and reset independently and only in
call() it would be determined what exactly to do. Since that commit,
the last of these calls would determine the operation. This means
that the sequence

  cloneCommand.setCloneAllBranches(true);
  cloneCommand.setBranchesToClone(/* some list of refs */);

would formerly do a "clone all" giving a fetch refspec with wildcards

  +refs/heads/*:refs/remotes/origin/*

which picks up new upstream branches, whereas since commit 6216b0de
individual non-wildcard fetch refspecs would be generated and new
upstream branches would not be fetched anymore.

Undo this behavioral change. Make the operations independently settable
and resettable again, and determine the exact operation only in call():
mirror=true > cloneAll=true > specific refs, where ">" means "takes
precedence over", and if none is set assume cloneAll=true.

Note that mirror=true implies setBare(true).

Bug: 559796
Change-Id: I7162b60e99de5e3e512bf27ff4113f554c94f5a6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-02-04 08:49:07 +01:00
Joshua Redstone 2a134e1541 AmazonS3: Speed up fetch, try recent packs first
When fetching remote objects, WalkFetchConnection searches remote
packs in the order provided by WalkRemoteObjectDatabase:getPackNames.
Previously, for TransportAmazonS3, the packs were in no particular
order. This resulted in potential many extra calls to get pack idx
files.

This change modifies TransportAmazonS3 and AmazonS3 so that
getPackNames returns a list sorted with the most recently modified
packs first.  In the case of fetching recent changes to a repo,
this dramatically reduces the number of packs searched and speeds
up fetch.

Note: WalkRemoteObjectDatabase::getPackNames does not specify
the order of the returned names.

Testing: did "mvn clean install" in root dir and all tests passed.
And manually constructed some S3 repos and using jgit.sh
confirmed that the freshest pack was checked first.

Change-Id: I3b968fee825e793be55566e28c2d69d0cbe53807
Signed-off-by: Joshua Redstone <redstone@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-02 14:44:22 +01:00
Matthias Sohn eaff0a2eba Merge branch 'stable-5.7'
* stable-5.7:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I0538bcba259f7229790a602ac9de120464a1260d
2020-02-01 02:17:49 +01:00
Matthias Sohn 7e3e740cc7 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I79d615dff66493b60d3a4bcbdc57b9455e8d6673
2020-02-01 02:09:37 +01:00
Matthias Sohn 3d59d1b80c Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I73d16b53df02bf735c2431588143efe225a4b5b4
2020-02-01 02:06:47 +01:00
Matthias Sohn 68b0645a2e Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I76a62da98182f0c504b1ea8b7d37cecdf4eea7e0
2020-02-01 01:53:27 +01:00
Matthias Sohn 195b8fc94c Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I12002dbfed9dff14fc6d2df9787d92eab5b1fa78
2020-02-01 01:28:40 +01:00
Matthias Sohn 48dd3385ab Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
2020-02-01 01:14:01 +01:00
David Pursehouse f57a7bd230 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows

Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-01 01:09:46 +01:00
David Pursehouse 7d5300023e Fix string format parameter for invalidRefAdvertisementLine
The externalized error message added in f4fc640 ("BasePackConnection:
Check for expected length of ref advertisement", Dec 18, 2019) uses a
malformed string format. Since there is only one formatting argument,
it should be referenced with '{0}' rather than '{1}'.

Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-01-31 08:38:49 +09:00
Matthias Sohn 1f9c717ff7 WindowCache: add metric for cached bytes per repository
Since ObjectDatabase and PackFile don't know their repository use the
packfile's grand-grand-parent directory as an identifier for the
repository the packfile resides in.

Remove metric for a repository if the number of cached bytes for the
repository drops to 0 in order to ensure the map of cached bytes per
repository doesn't contain repositories which have no data cached in the
WindowCache.

Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-28 11:24:06 +01:00
Thomas Wolf af05850485 FS: re-order fields and use all uppercase for true constants
Change-Id: I49fb77d6a12ae9083b5bf5779dc74a3dc49d7594
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-01-28 09:52:14 +01:00
Alex Jitianu 64715a189f FS: Don't use innocuous threads for CompletableFuture
The default threads from the ForkJoinPool run without any privileges
when a SecurityManager is installed, leading to SecurityExceptions
when trying to create the probe file even if the application otherwise
has write privileges in the directory.

Use a dedicated ThreadPoolExecutor using daemon threads instead.

Bug: 551690
Change-Id: Id5376f09f0d7da5ceea367e1f0dfc70f823d62d3
Signed-off-by: Alex Jitianu <alex_jitianu@sync.ro>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-01-28 09:52:14 +01:00
David Pursehouse 4cc13297cc ErrorProne: Enable and fix UnusedException check
Enable UnusedException at ERROR level which causes the build to fail
in many places with:

  [UnusedException] This catch block catches an symbol and re-throws
  another, but swallows the caught symbol rather than setting it as a
  cause. This can make debugging harder.

Fix it by setting the caught exception as cause on the subsequently
thrown exception.

Note: The grammatically incorrect error message is copy-pasted as-is
from the version of ErrorProne currently used in Bazel; it has been
fixed by [1] in the latest version.

[1] https://github.com/google/error-prone/commit/d57a39c

Change-Id: I11ed38243091fc12f64f1b2db404ba3f1d2e98b5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-27 14:40:08 +01:00
Matthias Sohn 7a23fc1250 Fix SshSessionFactory#setInstance to use service loader
If setInstance(SshSessionFactory) is called with parameter null
set default session factory using the newly introduced service loader
instead of hard-coding the default factory class.

Change-Id: I86b5932333cc53b706534a2822e0fd96e12e6e47
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-27 09:52:51 +01:00
Emmanuel Hugonnet 54b1c7cc6a Use ServiceLoader to define the default SSH session factory.
Use ServiceLoader and define
org.eclipse.jgit.transport.DefaultSshSessionFactory in
META-INF/services/org.eclipse.jgit.transport.SshSessionFactory so that
the legacy behavior is still the same.

Bug: 553625
Change-Id: Id1a65506140d921ed76d83699e3817f0d2ca08ed
Signed-off-by: Emmanuel Hugonnet <ehugonne@redhat.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-25 15:58:47 +01:00
Matthias Sohn fc7b003163 ReceivePack: enable overriding filterCommands and executeCommands
Change-Id: Ib923e137f0fadd876434bcc49ebd0c9a4d90b45c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-24 15:46:08 +09:00
Matthias Sohn 709f83d489 WindowCache: add option to use strong refs to reference ByteWindows
Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.

Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.

If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.

Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-20 17:53:08 +01:00
Andre Bossert eca34be43c Extract method refactoring in class DirCacheCheckout
* added new public method getContent()
* changed checkoutEntry() to use the new getContent() method
* changed runExternalFilterCommand() parameter "entry" to "path" to be
independent from DirCacheEntry type
* removed runExternalFilterCommand() parameter "fs" as it can be derived
from repo

Bug: 356832
Change-Id: If8991b74870e0d8728093e98a990f93f1871b76a
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2020-01-20 13:53:41 +01:00
David Pursehouse 0d962eaa78 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I17da3c5183eca536aa2be3972bc5df45c9d75f1b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-17 01:53:47 +01:00
Matthias Sohn 9f1dad4d4f Prepare 5.7.0-SNAPSHOT builds
Change-Id: I106098e85db05bb0142f9fb406e3aef3cf0238b4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-15 22:15:27 +01:00
Matthias Sohn 3c527a8a37 JGit v5.7.0.202001151323-m1
Change-Id: I7701bee6f05d66012c59092126eb4d52f828dc24
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-15 19:22:36 +01:00
Han-Wen Nienhuys 00252730dd Add ReftableNumbersNotIncreasingException#toString
Change-Id: I28b6a6476df185f3f98f8020408b4b120086cb0b
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-15 01:51:43 +01:00
Patrick Hiesel 6185db3d77 Replace usage of ArrayIndexOutOfBoundsException in treewalk
Using exceptions during normal operations - for example with the
desire of expanding an array in the failure case - can have a
severe performance impact. When exceptions are instantiated,
a stack trace is collected. Generating stack trace can be expensive.

Compared to that, checking an array for length - even if done many
times - is cheap since this is a check that can run in just a
handful of CPU cycles.

Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 18:38:02 +01:00
Matthias Sohn 2f3258cc28 Move ConnectivityChecker to fix API leak
Protected field ReceivePack.connectivityChecker is of type
ConnectivityChecker which was located in internal package
org.eclipse.jgit.transport and was thus leaking an internal type.

In order to fix this move ConnectivityChecker to package
org.eclipse.jgit.transport.

Change-Id: Idc0af75502faf39f13a56157bbb6964d24720eb8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 18:06:05 +01:00
Matthias Sohn db019c473e Merge branch 'stable-5.6'
* stable-5.6:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I4a77c602bfffed60535d0b8fc251a552b128068d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 16:52:56 +01:00
Matthias Sohn d7304eddaf Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I11f9a387ac3dc7d22a4f2e70bb8d89169b4e9afe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 13:15:12 +01:00
Matthias Sohn 5cfa74c7b1 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I78902d5feecb2c09134b64ec2f3b48b2c3bab37d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 12:32:03 +01:00
Matthias Sohn 4076e6aa3e Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I1b560b36d169cfa02cc5450ad0fa0bd85f9f42d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 11:38:28 +01:00
Matthias Sohn c5c1bbbe49 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix API problem filters
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 02:19:03 +01:00
Matthias Sohn 6f177b3f6c Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix unclosed resource warning in SmartOutputStream
  JschConfigSessionFactory: fix boxing warning
  SshSupport#runSshCommand: don't throw exception in finally block
  Don't override already managed maven-compiler-plugin version
  Remove unused import from CreateFileSnapshotBenchmark
  Remove duplicate ignore_optional_problems entry in .classpath
  Update maven-site-plugin used by benchmark module to 3.8.2
  Add dependency to enable site generation for benchmark module
  Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
  Fix MBean registration
  Enhance WindowCache statistics

Change-Id: Ic90aacf1ea40e13dc564d4d659e79535e86d0300
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 02:03:45 +01:00
Matthias Sohn fefddb28b2 Fix API problem filters
Change-Id: Icc78570f949ad64beb58caa192c829e536aa8dad
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 01:50:00 +01:00
Matthias Sohn 2b9dd32a82 Add config constants for WindowCache configuration options
Change-Id: Icc5265f87ae58aa1e667ed1827075c4a30f75c32
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-09 01:35:51 +01:00
Matthias Sohn 14bfe08757 JschConfigSessionFactory: fix boxing warning
Change-Id: I1735033c56b444a9a7160cb7df89292a228d5b34
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-06 11:40:05 +01:00
Matthias Sohn d9e957dc24 SshSupport#runSshCommand: don't throw exception in finally block
The CommandFailedException which was thrown in finally block is silently
discarded [1]. Refactor this method to throw the exception after the
finally block.

This fixes the warning "Null comparison always yields false: The
variable failure can only be null at this location".

[1] https://wiki.sei.cmu.edu/confluence/display/java/ERR04-J.+Do+not+complete+abruptly+from+a+finally+block
https://docs.oracle.com/javase/specs/jls/se8/html/jls-14.html#jls-14.20.2

Change-Id: Idbfc303d9c9046ab9a43e0d4c6d65d325bdaf0ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-06 11:40:04 +01:00
Matthias Sohn 8e21f0f9c7 Remove no longer used API problem filter
Change-Id: I29e7af25bad5282c6b09e4688bdcf23e22a0648f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-05 22:06:06 +01:00
Matthias Sohn b2bf9d4304 Remove duplicate copyright statements
Change-Id: I9a85136e1cc47693d1e475d00916c214c9aa55f4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-04 01:34:46 +01:00
Matthias Sohn 5c5f7c6b14 Update EDL 1.0 license headers to new short SPDX compliant format
This is the format given by the Eclipse legal doc generator [1].

[1] https://www.eclipse.org/projects/tools/documentation.php?id=technology.jgit

Bug: 548298
Change-Id: I8d8cabc998ba1b083e3f0906a8d558d391ffb6c4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-04 01:10:05 +01:00
Thomas Wolf d661b9f43a TLS support on IBM JDKs
SSLContext.getInstance("TLS") by default behaves differently on IBM
JDK than on Oracle or OpenJDK.[1] On IBM JDK one gets sockets that
have only TLSv1 enabled, which makes HTTPS connections fail since most
servers refuse this old protocol version. On Oracle JDK/OpenJDK, one
gets sockets with all available protocol versions enabled.

Explicitly enable all available TLS protocol versions to make
HTTPS connections work also on IBM JDK.

[1] https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.security.component.80.doc/security-component/jsse2Docs/matchsslcontext_tls.html#matchsslcontext_tls

Bug: 558709
Change-Id: I5ffc57a78e67a6239b9dad54840a49a8ed28930a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-03 20:48:25 +01:00
Matthias Sohn 9e6f09c700 Merge branch 'stable-5.6'
* stable-5.6:
  Hooks: avoid problems with backslashes in paths

Change-Id: Ia9e4aca11de5a465b60898a37441a17133f6355b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-26 00:19:20 +01:00
Thomas Wolf 2323d7a1ef Hooks: avoid problems with backslashes in paths
Calling sh -c with a script path containing backslashes may fail since
the shell may try to process them as escape characters.

Instead of calling

  sh.exe -c 'C:\path\script "$@"' 'C:\path\script' other args

call

  sh.exe -c '$0 "$@"' 'C:\path\script' other args

which avoids this escape processing.

Note that this is not specific to Windows. If the path or the script
name contain backslashes, this also occurs on Unix.

Bug: 558577
Change-Id: I47d63db6f8644f956c55c42b07dbcad7d7f305aa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-12-24 13:31:23 +01:00
Lars Vogel ccd68dc1dc Using StringBuilder in StringUtils#capitalize method
StringBuffer is synchronized which is slower and should be replaced with
StringBuilder according to its Javadoc.

Change-Id: If4d4a5a49da289ded34bbec97132ab7636b937cc
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
2019-12-18 00:18:41 +01:00
Matthias Sohn 83f90a274c Set config "extensions" option when converting ref storage format
When converting to reftable format the option extensions.refStorage must
be set to "reftable" [1]. When converting back to refdir format this
config option needs to be removed.

Introduce constants for refStorage config options, also for the
"reftree" format.

[1] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/master/Documentation/technical/reftable.md#Version-1

Change-Id: I190222fa5edc1ad7309daa9be17ca934ff7971e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-15 10:15:46 +01:00
Matthias Sohn 57f22d6db1 Externalize error message in FileRepository#convertRefStorage
Change-Id: I5d3acaa883ee7eb8d244b55b3e708cbe018690c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-14 22:10:12 +01:00
Carsten Hammer 74bc50125d Replace chain of if statements with switch
and switch over strings where possible. Sometimes if statements are
chained and form a series of comparisons against constants. Using switch
statements improves readability.

Bug: 545856
Change-Id: Iacb78956ee5c20db4d793e6b668508ec67466606
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-14 20:42:03 +01:00
Matthias Sohn 549c3acc5f Fix MBean registration
Change-Id: I6f6b8641f6c3e8ab9f625594085014272305656a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-13 17:32:51 +01:00
Lars Vogel de234106b4 Minor - Removes unnecessary whitespace in ReftableReader
Change-Id: Id0c3463d1b03e258d8633298828ef7cfcd44e13f
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
2019-12-11 15:36:52 -05:00
Matthias Sohn 4cb80f897f Prepare 5.6.1-SNAPSHOT builds
Change-Id: Iaa72d2ea6764ccd4fb6a124b51d89fe6492c602d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-11 02:04:21 +01:00
Matthias Sohn 9710c6292a JGit v5.6.0.201912101111-r
Change-Id: Icbb3b46f9d04e45da53936860e07e69fde12971c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-10 17:35:05 +01:00
Yunjie Li ba7f9c0612 Change the wildcard import to explicit ones.
Change-Id: Ia7bc1c4b6b82c127e1e4e0e657f69d249684e8e6
Signed-off-by: Yunjie Li <yunjieli@google.com>
2019-12-09 14:39:50 -08:00
Matthias Sohn 23c5ebff79 Merge branch 'stable-5.6'
* stable-5.6:
  Add ability to redirect stderr from git hooks
  Add possibility to get pure stderr output from AbortedByHookException

Change-Id: Ifc02675542dad6ced25fdd8b9fae80b5736db688
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-09 10:15:05 +01:00
Tim Neumann 23125abc20 Add ability to redirect stderr from git hooks
This will change the behavior in the CLI to resemble that of C-Git more
closely by printing the stderr of the hooks to the CLI stderr
independently of the exit code of the hook.

This is also useful for the corresponding EGIT-Change, which will add
the ability to show the hook output in eclipse.
With this also the stderr can be shown even if the exit code is 0.

Bug: 553471
Change-Id: Ie7bc503fe39e270e9b93dd1108b5879f02a12b4c
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
2019-12-09 00:56:40 +01:00
Tim Neumann d1bdfcd0a8 Add possibility to get pure stderr output from AbortedByHookException
This is a preparation for the corresponding EGit change to remove
redundant output in the aborted by commit hook popup.

Bug: 553469
Change-Id: Id5f39a4df659fafc9d951668e6f53fed4380de9d
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-09 00:55:09 +01:00
Matthias Sohn 42f0c7c9cb Enhance WindowCache statistics
Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time

Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.

Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.

Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-08 01:09:24 +01:00
David Pursehouse 8a633bde8a Merge branch 'stable-5.6'
* stable-5.6:
  Cleanup CommitAndLogCommandTest
  CLI: Add support for excluding paths from Git log command

Change-Id: I2f48e16640ab1b13f0c32779422add3162260c50
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-06 10:20:07 +09:00
Matthias Sohn c42f314b6d Fix API check errors introduced in ReceivePack in 945d43d5
- add missing @since tag on new field connectivityChecker
- suppress warning for this added field which may affect subclasses
which is ok

Change-Id: Id6c22258efd10fa3847290f3dcfa42893b6ae882
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 23:34:40 +01:00
John Tipper 5bf226d50c CLI: Add support for excluding paths from Git log command
Git log supports the exclude pathspec, which allows for excluding paths
from the log command. JGit only supports adding paths to the log
command. See the following StackOverflow question for details:
https://stackoverflow.com/questions/59143934/java-jgit-how-to-get-git-
commits-not-affecting-certain-directories
This commit adds an excludePath() method to the log command. It does not
yet support regex or glob wildcards.

Change-Id: I8cd59950b87850b55a15c7e2ea5470145c9aae28
Signed-off-by: John Tipper <john_tipper@hotmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 22:38:12 +01:00
Demetr Starshov 945d43d507 ReceivePack: Moves connectivity check to separate class
Move all connectivity check to separate classes. Set default to be
FullConnectivityChecker i.e. checker which will check connectivity
from all advertised refs. Add ability to set other connectivity
checker which can use different approach.

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: Ibb107dbfbdde8ad109be25b5ecf42be7660ef736
2019-12-05 11:46:36 -08:00
David Pursehouse f350e889d6 Merge branch 'stable-5.6'
* stable-5.6:
  ReftableTest: Clean up boxing warnings on usage of String.format
  MergedReftable: Remove unnecessary semicolon

Change-Id: Id35406acfdd91c3ddf17ce4e1023395f774774f2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-05 10:46:44 +09:00
Matthias Sohn a4ad83c96c Suppress API error for RefDatabase.hasFastTipsWithSha1()
This method was added in stable-5.6 after 5.6 RC1 was tagged. It's shown
as an API error when using 5.6.0 RC1 as the API baseline for master.
This suppression can be removed when 5.6 is released which will serve as
the API baseline for 5.7.

Change-Id: Ic6359fe59ed2d2a9431583b0854ec86bbac76880
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:56:31 +01:00
Matthias Sohn 1cb37e89d2 Remove API filters which became obsolete after bumping version to 5.7.0
Change-Id: I00b8d3e2e1023794f24d9447e3b0214c50353838
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:55:04 +01:00
Matthias Sohn e2c12c3910 Add missing @since tag for ReceivePack#receiveWithExceptionPropagation
Change-Id: Ie64abc3e5bcf6dbaecfa8f2cb50796703ebd1f39
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:51:29 +01:00
David Pursehouse 62d16a9146 MergedReftable: Remove unnecessary semicolon
Change-Id: I67d2b8ba3cd8f94363cea864918c8ed6026f2d18
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-12-05 09:42:52 +09:00
Matthias Sohn f487a9eb2e Prepare 5.7.0-SNAPSHOT builds
Change-Id: I7efa0cd08dee711d6fefc3e6bb478a3cb11fe85c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:32:59 +01:00
Matthias Sohn 08d4c63b6d Merge branch 'stable-5.6'
* stable-5.6:
  Introduce RefDatabase#hasFastTipsWithSha1
  Update Orbit to R20191126223242 for 2019-12
  Prepare 5.6.0-SNAPSHOT builds
  JGit v5.6.0.201912041214-rc1
  Prepare 5.6.0-SNAPSHOT builds
  JGit v5.6.0.201911271000-m3

Change-Id: Id33b054131bbcd27a0ae7200f564775049698d2c
2019-12-05 01:31:32 +01:00
Han-Wen Nienhuys 5185d288c1 Introduce RefDatabase#hasFastTipsWithSha1
The reftable format supports fast inverse (SHA1 => ref) queries.

If the ref database does not support fast inverse queries, it may be
advantageous to build a complete SHA1 to ref map in advance for
multiple uses. To let applications decide, this function indicates
whether the inverse map is available.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idaf7e01075906972ec21332cade285289619c2b3
2019-12-04 23:35:32 +01:00
Matthias Sohn 473527e42c Prepare 5.6.0-SNAPSHOT builds
Change-Id: I5e737ff7f262fdd43fc975a0b3594c8b33919663
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 22:13:56 +01:00
Matthias Sohn a7e454bc51 JGit v5.6.0.201912041214-rc1
Change-Id: I5ed21fbc5f83096bf0b79f2aa751db415cbcc7e8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 18:14:43 +01:00
Masaya Suzuki e75e95e535 transport: Add methods that do not handle exceptions
Same as UploadPack. These methods do not handle exceptions and the
caller can handle them. This makes it possible to customize the error
handling.

Change-Id: Ia5203f80133cb5b3e5d754859b4167ac920eaf14
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 14:09:55 -08:00
Masaya Suzuki 6ccff81428 transport: Let users specify an error handler for unpacking a pack file
Currently, there's no way for a user to customize the error handler for
the exception happened during unpacking an incoming pack file. Create an
error handler class and let them specify one.

Change-Id: Id07638ee58c88e1365181c3ddd17ee0266f3934d
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 14:03:46 -08:00
Masaya Suzuki 224746e0f2 ReceivePack: Use try-with-resources for PostReceiveHook
Instead of dropping a second exception, use try-with-resources so that
exception thrown by a hook is attached as a suppressed exception.

This removes a need of holding an exception that is thrown while
unpacking an incoming packfile.

Change-Id: I74d8b944931abbf53d9d1ae5db2e889639fba027
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 14:01:49 -08:00
Masaya Suzuki 4fcc43a1de transport: Consolidate status reporting code
BaseReceivePack#sendStatusReport anyway needs to know
CAPABILITY_REPORT_STATUS. By moving this flag to BaseReceivePack,
simplify the status reporting code.

Change-Id: Iaa0878b1fc13057b687a7f01d25c85fd78c0423e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 12:56:17 -08:00
Masaya Suzuki a91489f4a8 transport: Add ReceiveCommandErrorHandler
This gives a chance to handle an exception for a user. For example, when
an IOException is thrown while executing
`walk.parseAny(cmd.getNewId())`, it's always handled as
REJECTED_MISSING_OBJECT. However, IOException can mean a Git storage IO
error. By introducing an error handler class, a user can add a custom
error handler for these cases.

Change-Id: I3e03a536e1d8e137cb0f6e596d71642e72adde9e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 12:49:15 -08:00
Masaya Suzuki 19293add84 transport: Move exception handler to the caller
Move exception handling code to the caller side. This is a preparation
for adding a custom error handler that is similar to UploadPack.

Note that `catch (Throwable t)` will not affect the exception spec since
Java will do an analysis of rethrown exceptions. See
https://docs.oracle.com/javase/8/docs/technotes/guides/language/catch-multiple.html.

Change-Id: I973a63d4b079c2cd1402afb3f2956e18b7d5a00c
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 12:33:11 -08:00
Masaya Suzuki 6bc3665764 transport: Move handling code to the caller side
This removes a raw IOException at one level. Later we'll add a custom
exception handling mechanism like UploadPack.

Change-Id: I52a7423798c97b032d848351be8b6f144776b017
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-12-02 12:25:03 -08:00
Matthias Sohn 4ea42bc816 Prepare 5.6.0-SNAPSHOT builds
Change-Id: I1e52d2dfb202b87ecb9d0273deaa2c8d8ce1864e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 21:25:38 +01:00
Matthias Sohn 6c9aa82f70 JGit v5.6.0.201911271000-m3
Change-Id: I3810892c2bed947e4dfaa615a7b4d93eeb46abb9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 16:01:20 +01:00
Jonathan Nieder 853098e13e Merge "ReceivePack: Open visibility for some methods" 2019-11-26 23:56:40 -05:00
Ivan Frade 5897bc4306 ReceivePack: Open visibility for some methods
This partially reverts Ic6bb5e66. It made private some methods in
ReceivePack that were protected before. At least two of those methods
(#init and #receivePackAndCheckConnectivity) are used by out-of-tree
subclasses for tracing and testing.

Make ReceivePack#init() and ReceivePack#receivePackAndCheckConnectivity
protected again to allow tracing and testing.

Change-Id: I7ff22c091fbfc2d94009c449d58e7c5ac9f4f759
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-26 20:23:38 -08:00
Thomas Wolf 59f9d206c9 Make blame work correctly on merge conflicts
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.

Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].

Implement that and add tests.

I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.

(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)

[1] https://github.com/git/git/blob/v2.22.0/blame.c#L174

Bug: 434330
Change-Id: I9d763dd6ba478b0b6ebf9456049d6301f478ef7c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-27 03:03:06 +01:00
Jonathan Nieder aab5ee18fe Merge changes I8698001d,I5c1177ac
* changes:
  Add missing license header to ReftableDatabase
  Add missing license header to UploadPackRefSortingForReachabilityTest
2019-11-25 20:56:23 -05:00
Jonathan Nieder ffb979bef4 Add missing license header to ReftableDatabase
When this class was split from DfsReftableDatabase in commit
2b1e942729 (reftable: split off generic
code from DFS code, 2019-09-26), we forgot to carry over the license
header.

Change-Id: I8698001dfb69f58784df643ef0185ab2d55e52aa
Reported-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-11-25 17:42:59 -08:00
Han-Wen Nienhuys 9cb8e2f310 RepositoryCache: don't require HEAD in git repositories
Reftable-enabled repositories don't have a file called HEAD. Check for
reftable/ instead.

This fixes repository creation on reftable in Gerrit.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I778c2be01d96aaf135affae4b457b5fe5b483bee
2019-11-26 01:57:19 +01:00
Han-Wen Nienhuys e0744891fe FileRepository: cleanup refs outside refs/ on reftable conversion
Change-Id: Iab7d3a08906e826e26572f534512a09d3a5876b0
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-26 01:57:19 +01:00
Matthias Sohn 75038afc8e Silence API error for new method in ReachabilityChecker
Following OSGi semantic versioning which tolerates breaking implementers
in a minor release.

Change-Id: I4600c5ee9cd4ae209b69870a5d1367f83678617e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-25 23:43:22 +01:00
Ivan Frade f5f5c80bf5 BitmappedReachabilityChecker: Use only one bitmap for the whole check
The checker is creating a new bitmap per branch leading to excessive
memory consumption. For the reachability check one bitmap with the
reachability of all branches aggregated is enough.

Build the reachability bitmap with a filter. The filter itself uses it
to emit only commits not reached before and the caller to check what
targets have been reached already.

BitmapCalculator is not required anymore.

Change-Id: Ic5c62f77fe0f188913215b7eaa51d849a9aae6a5
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 14:07:04 -08:00
Ivan Frade 989a927a5f checkNotAdvertisedWants: Be lazy converting Ref to RevCommit
The ref points to an ObjectId that then is translated into a RevCommit.
This translation can be costly and with the incremental reachability
check is probably not needed for most of the elements.

Delay the translation from ObjectId to RevCommit to when it is needed.
Use Streams, that have the laziness built-in, all the way from Ref to
RevCommit.

This should reduce the latency for reachability checks over big sets of
references.

Change-Id: I28693087321b2beff3eaa1f3d2e7840ab0eedc6d
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 09:30:19 -08:00
Ivan Frade a0204a4727 ReachabilityChecker: Receive a Stream instead of a Collection
Preparatory change. Converting ObjectIds to RevCommits is potentially
expensive and in the incremental reachability check, it is probably not
required for all elements in the collection.

Pass a Stream to the reachability checker. In the follow up we make
the conversion from ObjectId to RevCommit in the stream (i.e. on
demand). This should reduce the latency of reachability checks over big
sets of references.

Change-Id: I9f310e331de5b0bf8de34143bd7dcd34316d2fba
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 09:30:19 -08:00
Ivan Frade 2ff0c0abaa UploadPack: Prioritize references for non-advertised wants checks
UploadPack needs to check if object ids that weren't advertised before
are reachable from the references visible to the user. In the
bitmap-based reachability check, this is done incrementally: checking
against one reference, if anything remaining adding a second and so on.
It is more efficient to check first more common references (e.g. refs/heads/*)

Sort the references for the reachability checker. This should solve the
connectivity earlier and require less bitmap creation and less memory.

Change-Id: I48ac10d71e29fab2d346479802401eaea4aacb5c
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-11-21 09:30:19 -08:00
David Pursehouse fa1566f40c ReceivePack: Fix name hiding of 'atomic' member
Change-Id: Ia357d49ca793ff1cb86e7130bc1a5f59590496a9
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-17 10:30:46 -08:00
David Pursehouse e9c21fca58 ReceivePack: Remove unnecessarily nested else-clause
Change-Id: I604ea76e03503300a381eca9c213f73677b5e21f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-17 10:27:11 -08:00
Michael Keppler f8e60ce1cf Simplify comparator code
Use lambda style comparators where possible. They are easier to read.

Change-Id: I5b80cfcd90909c94286742fa83af71015532809f
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-11-17 18:07:09 +01:00
Michael Keppler 8fcba48b72 Fix typo in Javadoc
No functional changes.

Change-Id: Ic7f9bb85806eca2788303c2f808d003cdfc7cc7c
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-11-17 18:05:46 +01:00
Han-Wen Nienhuys 38586d54d0 file: implement FileReftableDatabase
Reftable is a binary, block-based storage format for the ref-database.
It provides several advantages over the traditional packed + loose
storage format:

 * O(1) write performance, even for deletions and transactions.

 * atomic updates to the ref database.

 * O(log N) lookup and prefix scans

 * free from restrictions imposed by the file system: it is
   case-sensitive even on case-insensitive file systems, and has
   no inherent limitations for directory/file conflicts

 * prefix compression reduces space usage for repetitive ref names,
   such as gerrit's refs/changes/xx/xxxxx format.

FileReftableDatabase is based on FileReftableStack, which does
compactions inline. This is simple, and has good median performance,
but every so often it will rewrite the entire ref database.

For testing, a FileReftableTest (mirroring RefUpdateTest) is added to
check for Reftable specific behavior. This must be done separately, as
reflogs have different semantics.

Add a reftable flavor of BatchRefUpdateTest.

Add a FileReftableStackTest to exercise compaction.

Add FileRepository#convertToReftable so existing testdata can be
reused.

CQ: 21007
Change-Id: I1837f268e91c6b446cb0155061727dbaccb714b8
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-16 23:16:01 -08:00
David Pursehouse a6bdcace30 ReceivePack: Reduce visibility of methods
Several methods were protected when they were defined in BaseReceivePack
which has since been squashed into ReceivePack. Those methods no longer
need to be protected, and can now be private instead.

Change-Id: Ic6bb5e66f353fcbdc7919847423234af398c88b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-16 00:18:44 -08:00
Matthias Sohn 830e7bf678 Suppress API error raised for constant added in 345e2648
Change-Id: I29fb091f27f38c74843d50c2602164fc63ab8a16
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-16 00:17:32 -08:00
Thomas Wolf cb85f7be8b TopoSortGenerator: simplify first-parent handling
Change-Id: I74b40f1a2f81911c1d5ac5ae93b4a160fccf8f73
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 17:12:30 -08:00
Thomas Wolf 1daf6f13aa RewriteGenerator: avoid adding null parent
Prevent adding a null parent to a commit's parent array. Doing so
can cause NPEs later on.

Bug: 552160
Change-Id: Ib24b7b9b7b08e0b6f246006b4a4cade7eeb830b9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 16:00:48 -08:00
Thomas Wolf 345e2648df DiffFormatter: support core.quotePath = false
core.quotePath = false means that "bytes higher than 0x80 are not
considered "unusal" anymore"[1], i.e., they are not escaped. In
essence this preserves non-ASCII characters in path names in output.

Note that control characters and other special characters in the
ASCII range will still be escaped.

Add a new QuotedString.GIT_PATH_MINIMAL singleton implementing this.
Change the normal GIT_PATH algorithm to use bytes instead of characters
so it can be re-used. Provide a setter in DiffFormatter for the quoting
style so that an application can override the default, which is the
setting from the git config (and by default "true"). Use the new
QuotedString.GIT_PATH_MINIMAL when core.quotePath == false.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-corequotePath

Bug: 552467
Change-Id: Ifcb233e7d10676333bf42011e32d01a4e1138059
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 15:12:23 -08:00
Thomas Wolf a227dc3ba0 IndexDiff/SubmoduleWalk: make the RepositoryBuilder configurable
Some applications using JGit use their own repository caching. In
such applications, it may be needlessly inefficient to create new
submodule repositories from a SubmoduleWalk or in an IndexDiff. It
can be much more efficient to use an already cached repository
instance.

Provide a way to configure a SubmoduleWalk with a factory to create
BaseRepositoryBuilders to use to create repositories, and use it in
IndexDiff. Provide new IndexDiff.diff() operations that take such an
additional factory as parameter.

An application that caches Repository instances (for instance EGit)
can use a factory that provides builders that don't create a new
Repository instance but that return the already cached instance, if
one is available. Note that in such a case, the application may need
to be prepared to deal with IndexDiff.diff() also _closing_ the
obtained repository; if the application expects its cached Repository
instances to remain open while being cached, it'll have to use
Repository.incrementOpen() to prevent that the repository instance
gets closed.

Bug: 550878
Change-Id: Icc1b34dfc4cebd8ed4739dd09d37744d41adf711
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 14:48:44 -08:00
Thomas Wolf 7a3b93cbed IndexDiff/SubmoduleWalk: handle submodule.<name>.ignore correctly
IndexDiff would apply ignore mode ALL from .gitmodules to all remaining
submodules, and would ignore other settings from .gitignore and always
apply the setting defined on the IndexDiff instead. Correct that.

In canonical git the ignore setting from .gitmodules can also be
overridden by .git/config.[1] Implement that override in SubmoduleWalk.

[1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtignore

Bug: 521613
Change-Id: I9199fd447e41c7838924856dce40678370b66395
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-15 14:48:44 -08:00
Jonathan Nieder 83b9b84e4a Merge changes I583491f6,Idd44e3bb
* changes:
  Run Eclipse formatter on BatchRefUpdateTest
  Move KetchSystem.delay to FileUtils.
2019-11-14 18:21:04 -05:00
Matthias Sohn 63fc6970cc Suppress the API errors introduced by 3e9a5f99
I1ce92869435d5eebb7d671be44561e69c6233134 merged BaseReceivePack into
ReceivePack which breaks API but is only affecting the few jgit based
servers out in the wild.

Change-Id: Iad856a2afaf3cad95d01ad81a0116cebcd9de2d9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-14 10:08:24 -08:00
David Pursehouse 57bb09ade9 ReceivePack: Remove unused getLockMessageProcessName method
Change-Id: I85a0c3e3a53f87ad6f6eda41c7c6cb41b403b3bb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-13 16:33:34 -08:00
David Pursehouse b89e5ecc9f ReceivePack: Remove unnecessary ReceivePack. prefix on static call
Change-Id: I2d9a95266f5baa5a2298451adbd9ca0259b26c4b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-13 16:31:48 -08:00
Masaya Suzuki 3e9a5f993b transport: Merge BaseReceivePack into ReceivePack
Move the BaseReceivePack implementation back into ReceivePack. This is a
backward-incompatible change. For example, BaseReceivePack.FirstLine no
longer exists and cannot be referenced.  However, most of the code
should just work by replacing BaseReceivePack with ReceivePack.

Although this is an API change, it only affects callers using JGit as a
server, and there are very few of those in the wild.

Change-Id: I1ce92869435d5eebb7d671be44561e69c6233134
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-11-13 15:57:04 -08:00
Han-Wen Nienhuys 74bfec4112 Move KetchSystem.delay to FileUtils.
This will provide exponential backoff with jitter to other JGit
components too.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idd44e3bbaef6d71134ce2e3f7d405f35e7397cbd
2019-11-13 15:14:09 -08:00
David Pursehouse fd798d326e BaseReceivePack: Add hasReceivedPack method
The getPackSize method will throw IllegalStateException if it is
called when the pack size is not set. This is the case for example
when the received commands are all DELETE and there is no pack.

Add a new method hasReceivedPack that can be called prior to calling
getPackSize, to avoid causing the IllegalStateException.

See [1] for context.

[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=11918

Change-Id: I56397256a05e92c8398e65c07a859cee59b46317
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-11-13 14:56:45 -08:00
Matthias Sohn 566a46e9ec Silence API errors for new API introduced in 5.5.2
Change-Id: I9fee5138ea8f407d9d34cad4659c3a2d4f3ebf35
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-12 18:51:22 +01:00
Thomas Wolf ca8887a074 Merge branch 'stable-5.5'
* stable-5.5:
  Fix NPE in SystemReader in tests

Change-Id: I8500caeca99fe05ce9a80b54330b72e23670a46b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-12 12:23:56 +01:00
Thomas Wolf 22e153177c Fix NPE in SystemReader in tests
SystemReader.updateAll() must _not_ test whether the file exists. In
tests at least there are FileBasedConfigs with a null file. Test
configs should (and do) override isOutdated() to deal with this case.

Change-Id: I56303fe0d56afeb9f2203ee807a92c5dcf3809e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-12 09:39:45 +01:00
Matthias Sohn 826317942f Merge branch 'stable-5.5'
* stable-5.5:
  BaseReceivePack: Fix the format
  Prepend hostname to subsection used to store file timestamp resolution
  Store filesystem timestamp resolution in extra jgit config
  SystemReader: extract updating config and its parents if outdated

Change-Id: Iecfddce8081303af29badcdcd3d72a0da50c964f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-11 11:07:17 -08:00
Masaya Suzuki ca800b55c8 BaseReceivePack: Fix the format
Change-Id: I6136ef5318f81a6feb1267338ca76e60122fd15b
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-11-10 09:38:00 -08:00
Matthias Sohn e102bbed99 Prepend hostname to subsection used to store file timestamp resolution
This ensures the measured filesystem timestamp resolution will be only
used on the machine where it was measured and avoid errors in case the
~/.jgitconfig file is copied to another machine.

Bug: 551850
Change-Id: Iff2a11be62ca94c3bbe4a955182988dc50852f9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-09 18:01:33 +01:00
Matthias Sohn 838b5a84b5 Store filesystem timestamp resolution in extra jgit config
This avoids polluting hand-crafted user level config with
auto-configured options which might disturb in environments where
the user level config is replicated between different machines.

Add a jgit config as parent of the system level config. Persist
measured timestamp resolutions always in this jgit config and read it
via the user global config. This has the effect that auto-configured
timestamp resolution will be used by default and can be overridden in
either the system level or user level config.

Store the jgit config under the XDG_CONFIG_HOME directory following the
XDG base directory specification [1] in order to ensure that we have
write permissions to persist the file. This has the effect that each OS
user will use its jgit config since they typically use different
XDG_CONFIG_HOME directories.

If the environment variable XDG_CONFIG_HOME is defined the jgit config
file is located at $XDG_CONFIG_HOME/jgit/config otherwise the default is
~/.config/jgit/config.

If you want to avoid redundant measurement for different OS users
manually copy the values measured and auto-configured for one OS user to
the system level git config.

[1] https://wiki.archlinux.org/index.php/XDG_Base_Directory

Bug: 551850
Change-Id: I0022bd40ae62f82e5b964c2ea25822eb55d94687
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-09 18:01:19 +01:00
Thomas Wolf ffe74210d6 SystemReader: extract updating config and its parents if outdated
Change-Id: Ia77f442e47c5670c2d6d279ba862044016aabd86
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-08 09:27:54 +01:00
Matthias Sohn 9328236cfe Update API problem filters
- filter errors for new APIs added in service release
- remove unused filters

Change-Id: Ifbf532b8a3c46d4ed78a38f6c75073a072b7f669
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-04 16:24:55 +01:00
Matthias Sohn 5b7be296d2 Remove unused API problem filters
Change-Id: I0ae0fd4919b7ccc5218c6eb0fab22350bb9c9714
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-04 13:28:53 +01:00
Matthias Sohn 6d407fe727 Silence API errors for new API added since 5.1.0
Change-Id: I92a7c1b0319d61aeb25f58048bd07c144068c3db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-04 13:25:37 +01:00
Thomas Wolf e8c3619063 DirCacheCheckout: don't use deprecated method
Pass the CheckoutMetadata to checkoutEntry(), otherwise cr-lf settings
and smudge filters would be ignored.

Change-Id: Ifb1c4cb098a0b47c9752d0231d530db6a92b25a4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-03 10:14:40 +01:00
Thomas Wolf 64f2407f19 WorkingTreeModifiedEvent: must be fired explicitly after merge
A merge may write files to the working tree. After a successful
merge one must fire a WorkingTreeModifiedEvent explicitly if
getModifiedFiles() is not empty.

Also, any touched files must be reported by the
WorkingTreeModifiedEvent fired by DirCacheCheckout.checkout().

Bug: 552636
Change-Id: I5fab8279ed8be8a4ae34cddfa726836b9277aea6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-03 10:10:16 +01:00
David Pursehouse b29e9bd1cb Merge "HttpSupport.proxyFor(): use only scheme, host, and port" 2019-10-30 19:20:30 -04:00
Han-Wen Nienhuys 7c75a68b96 reftable: enforce ascending order in sortAndWriteRefs
MergedReftableTest#scanDuplicates tests whether we can write duplicate
keys in a merged reftable. Apparently, the first key appearing should
get precedence, and this works because the sort() algorithm on ordered
collections is stable.

This is potentially confusing behavior, because you can write data
into the table that cannot be retrieved (Merged table can only have
one entry per key), and the APIs such as exactRef() only return a
single value.

Make this consistent with behavior introduced in I04f55c481 "reftable:
enforce ordering for ref and log writes" by considering a duplicate key
in sortAndWriteRefs as a fatal runtime error.

Change-Id: I1eedd18f028180069f78c5c467169dcfe1521157
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-10-30 18:00:24 +01:00
Thomas Wolf b7d1eb4185 HttpSupport.proxyFor(): use only scheme, host, and port
Some URLs cannot be converted via URL.toURI(). So don't convert
the full URL but only the bits that are needed to find a proxy
via java.net.ProxySelector.

Bug: 549690
Change-Id: I55b5ecee70c6b52f72f9bdba9ce552fde7f33976
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-30 15:16:51 +01:00
David Pursehouse 078500ef1c Merge "[errorprone] Don't throw exception in finally" 2019-10-25 21:56:06 -04:00
David Pursehouse 66aa2cc30c Merge "[errorprone] Add parenthesis to clarify operator precedence" 2019-10-25 21:55:33 -04:00
Matthias Sohn 271051ef06 Replace "Rebase interactive" with "Interactive rebase"
The RepositoryState's description is shown in EGit's interactive rebase
view. Use the grammatically correct form.

Bug: 552359
Change-Id: I860ef730a67f2b34d7fabbfbda5ea65ead9270d2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-25 17:46:58 +02:00
Matthias Sohn 5625f70d25 [errorprone] Don't throw exception in finally
Instead log the problem. There is no reason to panic when we fail to
delete a single temporary file in java.io.tmpdir.

Change-Id: Idb867b3f07b090c7453ccd3688e94097df3b2672
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-25 15:27:36 +02:00
Matthias Sohn 6ba8f37ab7 [errorprone] Add parenthesis to clarify operator precedence
This fixes OperatorPrecedence error raised by errorprone on FS#findHook.

Change-Id: Ia15f61902c7deff7328c1afa066fc53152949bbf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-25 15:27:36 +02:00
Thomas Wolf 6a39da37fe GPG: implement more OpenPGP UserId matching formats
Instead of just looking for a substring match of user.signingKey
in a key's user ID implement the GPG matching formats[1] for:

'=' Full exact match
'<' Full exact match of the e-mail address
'@' Substring match within the e-mail address only
'*' General case-insensitive substring match (default)

When user.signingKey is not set, the committer's e-mail address is
used by default. In that case, use '<', i.e., require an exact match
on the OpenPGP e-mail address.

Also handle the optional "0x" prefix for (partial) key fingerprints.

[1] https://www.gnupg.org/documentation/manuals/gnupg/Specify-a-User-ID.html

Bug: 550335
Change-Id: I6ce482a099ff1a0dc9de45435cd4d3ec5b504f12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-23 11:17:24 +02:00
Jonathan Tan d4404fb438 Merge "reftable: remove assert calls" 2019-10-22 17:07:00 -04:00
Han-Wen Nienhuys e4fd171da2 reftable: remove assert calls
The effect of assert is defined by compiler flags, so this code
introduced a potential vector for corruption. 

Change-Id: I12197432e4351a5bd4aa24d352a19937721845c3
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-10-22 11:59:24 +02:00
Gunnar Wagenknecht 4f83544e50 Merge "GpgKeyLocator: Return a signing key for a user instead of the master key" 2019-10-22 01:22:26 -04:00
Gunnar Wagenknecht 748db6a0d6 Merge "GpgKeyLocator: Return subkeys instead of the first key for subkeys" 2019-10-22 01:21:34 -04:00
Roan Hofland 0902e060f7
GpgKeyLocator: Return a signing key for a user instead of the master key
Currently when a GPG key is looked up using a user identity the first
key from the keyring that has this user identity is returned.
The code was changed to instead return the first signing [S] key in this
keyring and only return the master key if no such signing key was found.
If the master key also does not have the signing flag set null is
returned instead.

Bug: 552288
Change-Id: I194862991d13c2c7ff34a60a54a227167f88f53b
Signed-off-by: Roan Hofland <roan.hofland@hotmail.com>
2019-10-22 02:28:18 +02:00
Roan Hofland 68b8317a09
GpgKeyLocator: Return subkeys instead of the first key for subkeys
Currently when a subkey is configured for signing via the git
user.signingkey configuration option the first key from the keyring for
this subkey would be returned for use (master key). The code has been
changed to return the requested key from the keyring instead.

Bug: 552288
Change-Id: I1c1cdf64c1667316a274ff9d829fc2b563797f2a
Signed-off-by: Roan Hofland <roan.hofland@hotmail.com>
2019-10-22 02:26:23 +02:00
Thomas Wolf 385b503ae8 Support for core.hooksPath
Support the core.hooksPath git config. This can be an absolute or
relative path of a directory where to find git hooks; a relative
path is resolved relative to the directory the hook will run in.

Bug: 500266
Change-Id: I671999a6386a837e897c31718583c91d8035f3ba
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-21 18:05:31 -04:00
Matthias Sohn 6216b0de8a Implement mirror option in CloneCommand
Bug: 552173
Change-Id: If79adf578b303890314a3285d7a6d2c71f48d091
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-21 09:27:13 +02:00
David Pursehouse 98cdca9b5a Enable and fix "Statement unnecessarily nested within else clause" warnings
Since [1] the gerrit project includes jgit as a submodule, and has this
warning enabled, resulting in 100s of warnings in the console.

Also enable the warning here, and fix them.

At the same time, add missing braces around adjacent and nearby one-line
blocks.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/227897

Change-Id: I81df3fc7ed6eedf6874ce1a3bedfa727a1897e4c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-10-17 10:20:14 +09:00
Jonathan Tan 88bd901492 Merge "reftable: split off generic code from DFS code" 2019-10-16 15:13:53 -04:00
Han-Wen Nienhuys 2b1e942729 reftable: split off generic code from DFS code
This introduces ReftableBatchRefUpdate and ReftableDatabase, as
generic classes, with some code moved to DfsReftableBatchRefUpdate and
DfsReftableDatabase.

Clarify thread-safety requirements by asserting locked status in
accessors, and acquiring locks in callers. This does not fix threading
problems, because ReftableBatchRefUpdate already wraps the whole
transaction in a lock.

This also fixes a number of bugs in ReftableBatchRefUpdate:

* non-atomic updates should not bail on first failure

* isNameConflicting should also check for conflicts between names that
  are added and removed in the BatchRefUpdate.

Change-Id: I5ec91173ea9a0aa19da444c8c0b2e0f4e8f88798
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-16 19:44:30 +02:00
Jonathan Tan 3b96aeb2c9 Merge "reftable: fix lookup by ID in merged reftables" 2019-10-16 13:33:20 -04:00
Han-Wen Nienhuys 218bacdc1f reftable: fix lookup by ID in merged reftables
On changing a ref, the old SHA1 is not updated in the object => ref
mapping. This means search by object ID may still turn up a ref from
deeper within the stack. To fix this, check all refs produced by the
merged iterator against the merged reftables.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I41e9cd395b0608eedeeaead0a9fd997238d747c9
2019-10-16 15:15:55 +02:00
Christian Halstrick 101b8ea58f Merge "IndexDiff: close SubmoduleWalk and use already loaded DirCache" 2019-10-15 17:06:12 -04:00
Ivan Frade 914e320ac6 ProtocolV2Parser: Introduce advertise sideband-all option
The flag enabling sideband-all is used in two places: in UploadPack
for advertisement and in the protocol parser to read it from the
request.

This leds to problems in distributed deployments where the two requests of
a fetch can go to different servers with different configurations.

Use the existing allowsidebandall to accept the sideband-all request
(and respond to it) and introduce a new "advertisesidebandall" to toggle
the advertising of the feature.

Change-Id: I892d541bc3f321606c89bad1d333b079dce6b5fa
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-10-15 12:20:37 -07:00
Thomas Wolf 36e16435be IndexDiff: close SubmoduleWalk and use already loaded DirCache
If the SubModuleWalk isn't closed its TreeWalk's ObjectReader won't
be closed. Re-loading the DirCache during an IndexDiff is not only
inefficient but could also give strange results if an external
process had modified the index in the meantime: file diffs would
be based on a "before" state, but submodule diffs on an "after"
state.

Change-Id: Iab948c08ac342138b37263c9028d80b84101f6d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-15 08:54:20 +02:00
David Pursehouse bca00aa5f4 DfsPackFile: Remove unused parameter from loadReverseIdx method
Change-Id: I2373084c8b40ebd6f432806dddc8633a76938e80
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-10-14 15:39:15 -04:00
Matthias Sohn d1d8bc30c2 Silence API errors for new fields added to PackStatistics.Accumulator
These fields were added in a307c887.

Change-Id: I9e80abdb6f48dc7bf72de683dcf3566d2dab611f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-12 18:25:33 +02:00
Matthias Sohn e3b0ee14d9 Remove unused API warning filters
Change-Id: I78262681e12079028752d8c144720ad0256c5707
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-12 11:05:02 +02:00
Matthias Sohn 71bd70e67f Add missing @since tag to UploadPack#uploadWithExceptionPropagation
This was missed in b8d9734c

Change-Id: I4b9c1cf058eff16af725375047d743944e7f5db4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-12 11:05:02 +02:00
Ivan Frade 10b1b490a6 CachedPackUriProvider: size should be a long (not an int)
Change-Id: I5cd8b6376cdec59c5ead318edc5385da961511ec
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-10-11 16:24:45 -07:00
Ivan Frade a307c88714 PackWriter/Statistics: Report offloaded size
Add to statistics the amount and size of packfiles offloaded to HTTP
download.

Change-Id: I895a7219ecac2794368bfc4fdfae74c1238deed9
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-10-11 14:13:24 -07:00
Ivan Frade 892c804699 CachedPackUriProvider: Add size to the pack information
The object identifying packfiles to send them via packfile-uri contains
only the uri and the hash. This is the information that goes through the
wire. It would be useful to know also the size of those packfile, for
example to track how many bytes have been offloaded to HTTP.

Add size field the CachedPackUriProvider.PackInfo object.

Change-Id: If6b921b48a4764d936141c777879b148cc80bbd3
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-10-11 14:13:17 -07:00
Terry Parker 6bd9bc23b7 Merge changes Ieebbd671,I0e3e9456,Ia8d72e31
* changes:
  UploadPack: Create a method that propagates an exception as-is
  UploadPack: Consolidate the sideband handling code to one place
  UploadPack: Introduce ErrorWriter
2019-10-09 15:55:48 -04:00
Masaya Suzuki b8d9734c02 UploadPack: Create a method that propagates an exception as-is
Exception handling can be isolated from UploadPack. This makes it
possible to make the exception handler pluggable.

Change-Id: Ieebbd6711963c7f2e47a98783b4ad815793721c7
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-10-09 10:02:03 -07:00
Masaya Suzuki 1e3a7bcef7 UploadPack: Consolidate the sideband handling code to one place
This consolidates the sideband stream creation code and the error
handling code for the sideband-allowed part in the Git protocol to one
place.

Change-Id: I0e3e94564f50d1be32006f9d8bcd1ef1ce6bf07e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-10-09 10:02:01 -07:00
Masaya Suzuki 63bd24cf35 UploadPack: Introduce ErrorWriter
ErrorWriter writes an error message to the user. The implementation is
swapped once it detects that the client supports sideband. By default it
uses the protocol level ERR packet, which was introduced recently.

In total the error output is done in two different places;
UploadPack#upload and UploadPack#sendPack. These will be consolidated in
the next change.

Change-Id: Ia8d72e31170bbeafc8ffa8ddb92702196af8a587
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-10-09 10:01:06 -07:00