Commit Graph

7258 Commits

Author SHA1 Message Date
Sebastian Schuberth 0cac834668 cli: Add the --tags option to describe
Change-Id: I78924e61e2050eeaff991ee56715f36514100492
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2019-05-31 17:10:13 +02:00
Sebastian Schuberth afcb3a8c9e DescribeCommand: Consistenly omit the default value
Omit the default value of "false" for "useTags" like already done for
"longDesc".

Change-Id: I25aaacae028fc8cf27f4040ba45fe79609318aa1
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2019-05-31 17:08:39 +02:00
Matthias Sohn 22352e4751 Remove excess blank line in FileUtilsTest
Change-Id: I0d617feb7baacf9917b1da605fcb75f7d4ccd184
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-31 09:40:59 +09:00
David Pursehouse d2425e16f4 PacketLineIn: Add helper methods to check for END and DELIM
These methods will allow clients to check for END and DELIM without
doing a reference comparison on the String objects, which raises
warnings from Error Prone.

Change-Id: I9e7e59843553ed4488ee8e864033198bbb60d67c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-30 20:31:39 +09:00
David Pursehouse 9f5d271b12 UploadPackTest: Rename variable to avoid hiding class member
Change-Id: I9e02642b2f56a735681e58b8c8d6e3545ef9093b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-30 20:18:28 +09:00
David Pursehouse cd1d2d6fe3 UploadPackTest: Add missing <> operator on instantiation of ArrayList
Change-Id: I348cbcd75252d7857ada843dc496d445ee1d62fb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-30 20:17:24 +09:00
Matthias Sohn 4904a625c9 Avoid null PackConfig in GC
Initialize it using the repository's config already in the constructor.

Change-Id: I4ea620a7db72956e7109f739990f09644640206b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-29 23:41:46 +02:00
Matthias Sohn 5d0286eb7c Add FileSnapshot test testing recognition of file size changes
Change-Id: Ibcd76a5e6e4183ada0be1d4436ce957243f2094d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-29 23:41:45 +02:00
Matthias Sohn 43b06f51f9 Capture reason for result of FileSnapshot#isModified
This allows to verify the expected behavior in
FileSnapshotTest#testSimulatePackfileReplacement and enables extending
FileSnapshot for packfiles to read the packfile's checksum as another
criterion to detect modifications without reading the full content.

Also add another field capturing the result of the last check if
lastModified was racily clean.

Remove unnecessary determination of raciness in the constructor. It was
determined twice in all relevant cases.

Change-Id: I100a2f49d7949693d7b72daa89437e166f1dc107
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-29 23:41:45 +02:00
Matthias Sohn d31a0c0b5d Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows
NTFS does not support FileKey hence ignore this test on Windows.

Change-Id: I7b53a591daa5e03eb5e401b5b26d612ab68ce10d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-29 23:41:43 +02:00
Matthias Sohn 1e8b68cbc5 Tune max heap size for tests
This is an attempt to fix crashes observed on the new Jenkins
infrastructure running on Kubernetes [1].

Increase it to 512m for
- org.eclipse.jgit.ant.test
- org.eclipse.jgit.http.test
- org.eclipse.jgit.lfs.server.test
- org.eclipse.jgit.lfs.test
- org.eclipse.jgit.pgm.test

Decrease it to 768m for
- org.eclipse.jgit.test

[1] e.g. https://ci-staging.eclipse.org/jgit/job/stable/job/jgit.gerrit/16074/console

Change-Id: Id074ed0f7bcb8a13da649a547342af2a08439d9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit e19e859977)
2019-05-29 23:39:55 +02:00
Matthias Sohn 0e33b32ab7 Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc
testNewFileNoWait() was identical to testNewFileWithWait() but claims it
doesn't wait at all. Hence remove the waits.

Change-Id: I49b8ca5cb49a43c55fe61870c18c42f32fb4b74d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-27 10:49:51 +02:00
Marc Strapetz f1577909e7 ObjectDirectory: fix closing of obsolete packs
This resolves a regression introduced in fef78212.

Change-Id: Ibb4521635a87012520566efc70870c59d11be874
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2019-05-27 10:49:51 +02:00
Ivan Frade 82316fc23a BitmapCalculator: javadoc fixes
Fill out the description of when IOException is thrown.

Also fix a typo in the description for IncorrectObjectTypeException.

Change-Id: I9fafd19d68ddc4fe4e95e8516c2c38484b941a3a
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-27 10:38:37 +09:00
Matthias Sohn 92d1ca8099 Update API filters for methods added to fix bugs
Change-Id: I235c608d80a14c8af39400ae2e749488a4fa0d22
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-26 20:54:44 +02:00
David Pursehouse 245c2c09a0 Bazel: Increase severity of most error-prone checks to ERROR
Most of the checks can be increased to ERROR because there is no
code in the project that triggers them.

There are still several that are triggered, and these are left
at the WARN severity:

  https://errorprone.info/bugpattern/AmbiguousMethodReference
  https://errorprone.info/bugpattern/BadComparable
  https://errorprone.info/bugpattern/ClassNewInstance
  https://errorprone.info/bugpattern/Finally
  https://errorprone.info/bugpattern/FutureReturnValueIgnored
  https://errorprone.info/bugpattern/ImmutableEnumChecker
  https://errorprone.info/bugpattern/NarrowingCompoundAssignment
  https://errorprone.info/bugpattern/NonOverridingEquals
  https://errorprone.info/bugpattern/OperatorPrecedence
  https://errorprone.info/bugpattern/ReferenceEquality
  https://errorprone.info/bugpattern/ShortCircuitBoolean
  https://errorprone.info/bugpattern/StringEquality
  https://errorprone.info/bugpattern/TypeParameterUnusedInFormals

These can be cleaned up and increased to ERROR in follow-up
commits.

Change-Id: Icfc3b3163e129e504f10b3fc856aef262f723f99
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-26 18:03:07 +09:00
David Pursehouse 4c33f70d5a Enable error-prone checks by default
Based on recent work by Dave Borowitz on the Gerrit project [1, 2].

The warnings/error configuration is unchanged, but now the checks are
enabled by default during the build rather than having to be manually
invoked.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/225653
[2] https://gerrit-review.googlesource.com/c/gerrit/+/225714

Change-Id: I7f4849896af72de38880b13d64519cc93bb3d2a3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-26 17:47:08 +09:00
David Ostrovsky a4f5fc1572 Add bazel options to align with gerrit's
Change-Id: I6addd5fcaecb8c2508e1d5776ecfed8ae676b6ed
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2019-05-26 02:44:33 -04:00
Andrey Loskutov e57a37e767 RevWalkUtils: add progress callback to findBranchesReachableFrom
Offer a version of findBranchesReachableFrom method with progress
monitor callback. This is required to allow UI clients to cancel long
running operations and show progress.

Bug: 547642
Change-Id: I31d1de54dbaa6ffb11e03da4c447963e8defa1d0
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2019-05-25 13:52:49 +02:00
David Pursehouse 76c867d62a Merge branch 'stable-5.4'
* stable-5.4:
  SystemReader: Respect passed-in parent when no system config is present
  Prepare 5.4.0-SNAPSHOT builds
  JGit v5.4.0.201905221418-m3
  Prepare 5.4.0-SNAPSHOT builds
  JGit v5.4.0.201905081430-m2

Change-Id: If69cb0afe9915b926d977e4ad26c7d72db31667c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-24 08:34:40 +09:00
David Pursehouse 7eb222f20d Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  SystemReader: Respect passed-in parent when no system config is present

Change-Id: I758ed0c2a3527ffcf6255158a49090aba090e942
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-24 08:33:45 +09:00
Dave Borowitz 04bd5821e9 SystemReader: Respect passed-in parent when no system config is present
The default implementation of openSystemConfig has special handling for
when the FS returns null from getGitSystemConfig: it should return a
"real" FileBasedConfig instance that never actually tries to load a
file. However, this codepath was not respecting the passed-in parent
config.

Change-Id: Id0bcdc93bb42f9ebe3f5ee4c6b4be8863e0133f9
2019-05-23 16:07:49 -07:00
David Pursehouse 87c7bda914 Upgrade maven-source-plugin to 3.1.0
Change-Id: I8253e75f5038b6c0e2205d74683e0ebd9839dd33
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-23 10:13:13 +09:00
David Pursehouse d510929a25 Upgrade maven-jar-plugin to 3.1.2
Change-Id: I9a291390205ee8c44b02c4081f2d8f17635d90ec
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-23 10:13:13 +09:00
David Pursehouse 22360a4bee Upgrade jacoco-maven-plugin to 0.8.4
Change-Id: Ibab24e1f5eefdc14bcf9ab6d72eab7e0a0cfaad9
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-05-23 10:13:13 +09:00
Matthias Sohn 73ba6658cf BitmapCalculator and its test: add missing license header
Change-Id: I3c4a8b1e9159b0553aa95213bb82628370b6c036
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-23 02:12:16 +02:00
Matthias Sohn 0988276746 Prepare 5.4.0-SNAPSHOT builds
Change-Id: I9df6fccee253d4087f9afab4cb46e0a40b8a5699
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 22:52:43 +02:00
Matthias Sohn 72998dd5ee JGit v5.4.0.201905221418-m3
Change-Id: I8a7e687acfcf8c341abd726ae60b5ee173422215
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 20:18:14 +02:00
Ivan Frade 5dce8614ab RevWalk: new method createReachabilityChecker()
Every caller would need to check if bitmaps are available in the repo to
instantiate a reachability checker.

Offer a method to create the reachability checker in the walk: the
caller has already a walk over the repo, and the walk has all the
information required.

This allows us to make the implementation classes package-private.

Change-Id: I355e47486fcd9d55baa7cb5700ec08dcc230eea5
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-22 14:17:31 -04:00
Matthias Sohn bed7a45fb5 Merge branch 'master' into stable-5.4
* master:
  ReachabilityCheckers: @since 5.4
  Update Orbit to S20190521195709 for 2019-06 M3
  Tune max heap size for tests
  GPG: check that the key found is a signing key
  GPG: use key fingerprint suffix to compare id for signing key
  Recognize ReachabilityCheckerTestCase as helper
  UploadPack: restore inadvertently deleted line
  UploadPack: Use reachability checker to validate non-advertised wants
  Bazel: Simplify names of bouncy castle libraries
  Convert to lambda or member reference
  BitmappedReachabilityChecker: Reachability check using bitmaps
  BitmapCalculator: Get the reachability bitmap of a commit
  ReachabilityChecker: Default implementation with a RevWalk

Change-Id: I323c460853230abcffa94c4f34f15dbaff8c330c
2019-05-22 19:37:40 +02:00
Ivan Frade 8216929618 ReachabilityCheckers: @since 5.4
The new ReachabilityChecker interface and its implementations are marked
as @since 5.5, but they will make it to the 5.4 release.

Change-Id: I88c31b3300ccf35d18c35faddb2517f0a57bdcfd
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-22 17:37:12 +02:00
Matthias Sohn c33c0bd1b5 Update Orbit to S20190521195709 for 2019-06 M3
Update
- org.apache.ant to 1.10.6.v20190516-0412 (CQ 19829)
- org.apache.httpcomponents.httpclient to 4.5.6.v20190503-0009

CQ: 19829
Change-Id: Ia057d4f07dab21374ca66c3d87a2bab1c795562c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 17:34:58 +02:00
Matthias Sohn e19e859977 Tune max heap size for tests
This is an attempt to fix crashes observed on the new Jenkins
infrastructure running on Kubernetes [1].

Increase it to 512m for
- org.eclipse.jgit.ant.test
- org.eclipse.jgit.http.test
- org.eclipse.jgit.lfs.server.test
- org.eclipse.jgit.lfs.test
- org.eclipse.jgit.pgm.test

Decrease it to 768m for
- org.eclipse.jgit.test

[1] e.g. https://ci-staging.eclipse.org/jgit/job/stable/job/jgit.gerrit/16074/console

Change-Id: Id074ed0f7bcb8a13da649a547342af2a08439d9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 14:42:43 +02:00
Matthias Sohn 91101414ae Include filekey file attribute when comparing FileSnapshots
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).

Some filesystems expose unique file identifiers, e.g. inodes in Posix
filesystems which are named filekeys in Java's BasicFileAttributes. Use
them as another means to detect file modifications based on stat
information.

Running git gc on a repository yields a new packfile with the same id as
a packfile which existed before the gc if these packfiles contain the
same set of objects. The content of the old and the new packfile might
differ if a different PackConfig was used when writing the packfile.
Considering filekeys in FileSnapshot may help to detect such packfile
modifications.

Bug: 546891
Change-Id: I711a80328c55e1a31171d540880b8e80ec1fe095
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 08:13:56 +02:00
Matthias Sohn b513b77477 Measure file timestamp resolution used in FileSnapshot
FileSnapshot.notRacyClean() assumed a worst case filesystem timestamp
resolution of 2.5 sec (FAT has a resolution of 2 sec). Instead measure
timestamp resolution to avoid unnecessary IO caused by false positives
in detecting the racy git problem caused by finite filesystem timestamp
resolution [1].

Cache the measured resolution per FileStore since timestamp resolution
depends on the respective filesystem type. If timestamp resolution
cannot be measured or fails due to an exception fallback to the worst
case FAT timestamp resolution and avoid caching this value.

Add a 10% safety margin in FileSnapshot.notRacyClean(), though running
FsTest.testFsTimestampResolution() 1000 times which is not using a
safety margin didn't fail on Mac using APFS and Java 8, 11, 12.

Measured Java file timestamp resolution: [2]

[1] https://github.com/git/git/blob/master/Documentation/technical/racy-git.txt
[2] https://docs.google.com/spreadsheets/d/1imy0y6WmRqBf0kjCxzxj2X7M50eIVfa7oaUIzEOHmjo

Bug: 546891
Change-Id: I493f3b57b6b306285ffa7d392339d253e5966ab8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 08:13:22 +02:00
Matthias Sohn 201bbd6ead Fix FileSnapshot's consideration of file size
* fix equals() and hashCode() methods to consider size
* fix toString() to show size

Change-Id: I5485db55eda5110121efd65d86c7166b3b2e93d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 08:12:51 +02:00
Matthias Sohn ed2e515ddf Fix API problem filters
Change-Id: I566391d7c51875f30cf580d64e6784819985709f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-22 08:03:29 +02:00
Thomas Wolf efe6d2bb5b GPG: check that the key found is a signing key
Throw an exception if not.

Change-Id: I60f36b271d5f44c6dc475302b169cb5b8a1e3945
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-05-21 18:13:40 +02:00
Thomas Wolf 6536b5cbca GPG: use key fingerprint suffix to compare id for signing key
Check whether the value of the git config user.signingKey is a suffix
of the full fingerprint of the key. This was already used for finding
keys in secring.gpg, but not in pubring.kbx. This mechanism allows a
user to use any unique suffix to identify keys; to avoid needless
collisions it's recommended to use at least the last 16 characters of
the hex representation of the fingerprint, which is the key id.[1]

[1] https://tools.ietf.org/html/rfc4880#section-12.2

Bug: 545673
Change-Id: If6fb4879502b6ee4b8c26c21b2714aeac4e4670c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-05-21 18:11:28 +02:00
Jonathan Nieder 6e127c089a Merge "Recognize ReachabilityCheckerTestCase as helper" 2019-05-20 23:14:34 -04:00
Jonathan Tan bab685488d Recognize ReachabilityCheckerTestCase as helper
4e196faa1b ("ReachabilityChecker: Default implementation with a
RevWalk", 2019-05-15) added ReachabilityCheckerTestCase but did not add
it as a test helper in the corresponding BUILD file, making tests no
longer runnable with Bazel. Resolve this issue.

Change-Id: Iccc00b0d169dbaa137e130ce2ddd1b7669960b52
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-05-20 17:02:07 -07:00
Jonathan Tan f2318190bf UploadPack: restore inadvertently deleted line
In 7b96bd812e ("UploadPack: Use reachability checker to validate
non-advertised wants", 2019-05-16), a "walk.setRetainBody(false);"
statement was inadvertently deleted. (An earlier version of this commit
had this line in another part of the code and a review comment suggested
to move it back here; the line was then deleted from the other part of
the code but not readded.) Restore this line.

Change-Id: I96ff6106ba9e4eef429388c83e898b3363295f69
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-05-20 16:22:31 -07:00
Jonathan Tan 7121fa6fec Merge changes from topic 'reachability-2'
* changes:
  UploadPack: Use reachability checker to validate non-advertised wants
  BitmappedReachabilityChecker: Reachability check using bitmaps
  BitmapCalculator: Get the reachability bitmap of a commit
  ReachabilityChecker: Default implementation with a RevWalk
2019-05-20 16:04:28 -04:00
Ivan Frade 7b96bd812e UploadPack: Use reachability checker to validate non-advertised wants
In "Reachable commit" request validators, we need to check that a "want"
in the request, that hasn't been advertised, is reachable from the refs
visible to the user.

Current code has intermixed the translation of ObjectIds to RevCommits
(and its error handling) with the actual walk, with the delegation to
bitmaps in restricted circunstances.

Refactor the code to make it "flatter" and more readable. Move ObjectIds
to RevCommits translation to its own functions. Use the reachability
checker instead of a newly defined walk.

Before the non-advertised wants were validated with bitmaps only if any
"want" refered to an non-commit. Now they will be validated with bitmaps
also if the "wants" refer all to commits.

Change-Id: Ib925a48cde89672b07a88bba4e24d0457546baff
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-16 16:31:16 -07:00
David Ostrovsky 5e4be72b5a Bazel: Simplify names of bouncy castle libraries
Name bouncy castle libraries consistently. This also allows to use
local jgit tree as dependency in gerrit.

Change-Id: Ie0a995a70ecee907d25b3cdda9034a25c0290efe
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2019-05-16 19:04:40 +02:00
Carsten Hammer 84fc5c908d Convert to lambda or member reference
Convert anonymous inner classes to lambda expressions or member
references

Bug: 545856
CQ: 19537
Change-Id: I621431c178e8b99316314602f7c66c9a36f9ae98
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-16 02:58:43 -04:00
Ivan Frade 601b0ae577 BitmappedReachabilityChecker: Reachability check using bitmaps
The "basic" reachability check walks the graph starting from the tips
marking things as "uninteresting". If the target commit is marked as
"uninteresting" it was reached; it is reachable from those tips.

This requires a lot of walking and can be solved directly with bitmaps.
Most of the time the bitmaps are already calculated or a short walk
away.

This should improve the performance of reachability checks, for example
in Gitiles.

Change-Id: I83d33271f58d95d2dc9ed151967b3eda513c99f7
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-15 16:36:20 -07:00
Ivan Frade b6da4b34cc BitmapCalculator: Get the reachability bitmap of a commit
To make reachability checks with bitmaps, we need to get the
reachability bitmap of a commit, which is not always precalculated.

There is already a class returning such bitmap (BitmapWalker) but it
does too much unnecessary work: it calculates ALL reachable objects from
a commit (i.e. including trees and blobs), when for reachability the
commits are just enough.

Introduce BitmapCalculator to get the bitmap of a commit: either because
it is precalculated or generating it with a walk only over commits.

Change-Id: Ibb6c78affe9eeaf1fa362a06daf4fd2d91c1caea
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-15 15:04:00 -07:00
Ivan Frade 4e196faa1b ReachabilityChecker: Default implementation with a RevWalk
It is common to check if a certain commit is reachable from some
starting points. For example gitiles does it to check if a commit
is visible to a user based on its permissions.

Offer this functionality in JGit.

Split the interface as the next commit will introduce an implementation
using bitmap indices.

Change-Id: I0933b305c8d734f7a64502910ff4d9ef4fc92ae1
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-05-15 14:55:54 -07:00
Matthias Sohn d31716f883 Prepare 5.4.0-SNAPSHOT builds
Change-Id: Ieb4b049972509631f06b5268a90f432a1b9ee207
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-05-15 09:16:58 +02:00