Commit Graph

4013 Commits

Author SHA1 Message Date
Matthias Sohn c452c007ce Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:18:01 +01:00
Matthias Sohn 2bf3cc56db Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:16:28 +01:00
Matthias Sohn 317d747c5c Prepare 4.7.9-SNAPSHOT builds
Change-Id: I1602a6b811fd9f95e284153b560d8d9f855fae9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 23:26:43 +01:00
Matthias Sohn a2951a996f JGit v4.7.8.201903121755-r
Change-Id: Ia6dbde615671ed09789326aacad1655e7a7a59ad
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:55:42 +01:00
Matthias Sohn 92525b584a Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:39:53 +01:00
Matthias Sohn d96eb8b3e6 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:26:58 +01:00
Matthias Sohn a47367e5fb Prepare 4.5.7-SNAPSHOT builds
Change-Id: I5c275c542e12746c3d8ecf8462791969f9e89e12
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 20:54:12 +01:00
Matthias Sohn 1cb31111f7 JGit v4.5.6.201903121547-r
Change-Id: I5a071ed10e1ac1ab28f992d45cde335c12556a80
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 20:47:04 +01:00
Luca Milanesio bf3d1ded35 Check for packfile validity and fd before reading
When reading from a packfile, make sure that is valid
and has a non-null file-descriptor.

Because of concurrency between a thread invalidating a packfile
and another trying to read it, the read() may result into a NPE
that won't be able to be automatically recovered.

Throwing a PackInvalidException would instead cause the packlist
to be refreshed and the read to eventually succeed.

Bug: 544199
Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 15:06:05 +01:00
Luca Milanesio afef866a44 Move throw of PackInvalidException outside the catch
When a packfile is invalid, throw an exception explicitly
outside any catch scope, so that is not accidentally caught
by the generic catch-all cause, which would set the packfile
as valid again.

Flagging an invalid packfile as valid again would have
dangerous consequences such as the corruption of the in-memory
packlist.

Bug: 544199
Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 15:06:05 +01:00
Luca Milanesio 2d116cd0ab Use FileSnapshot to get lastModified on PackFile
Do not redundantly call File.lastModified() for extracting the
timestamp of the PackFile but rather use consistently the FileSnapshot
which reads all file attributes in a single bulk call.

Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 15:06:04 +01:00
Luca Milanesio 2dc572df24 Include size when comparing FileSnapshot
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).

Read and consider file size also, so that differing file size can help
to more accurately detect file changes without reading the file content.
Use bulk read to avoid multiple stat calls to retrieve file attributes.

Change-Id: I974288fff78ac78c52245d9218b5639603f67a46
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 14:39:14 +01:00
Luca Milanesio fef782128d Do not reuse packfiles when changed on filesystem
The pack reload mechanism from the filesystem works only by name
and does not check the actual last modified date of the packfile.

This lead to concurrency issues where multiple threads were loading
and removing from each other list of packfiles when one of those
was failing the checksum.

Rely on FileSnapshot rather than directly checking lastModified
timestamp so that more checks can be performed.

Bug: 544199
Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 14:30:36 +01:00
Matthias Sohn d4078dccda Silence API warnings for new API introduced for fixes
Change-Id: I3ea7ff2efd33ca6c780afaef9010cec82780d7fa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 14:30:20 +01:00
Juergen Denner 19c4380689 Reduce contention on PackFile.idx() function.
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.

Performance numbers are listed in the bug report.

Bug: 543739
Change-Id: Iff328d347fa65ae07ecce3267d44184161248978
Signed-off-by: Juergen Denner <j.denner@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-11 19:52:26 +01:00
Matthias Sohn 7b3ee6f62e Fix error log message in ObjectDirectory.handlePackError()
Change-Id: I154f392ad025c4b642eb1123d375a0afaa853885
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 997d785418)
2019-03-06 02:36:38 +01:00
Matthias Sohn e8f7bb3608 Properly format pack checksums in PackFile.idx()
Change-Id: Id805850dbe9a3d633168f3056e06ddeafd86f961
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit a33e4dc58a)
2019-03-06 02:36:38 +01:00
Matthias Sohn e8bd9556c0 Cancel gc if thread was interrupted
see
https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ

Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 882fed0d96)
2019-03-06 02:36:38 +01:00
Luca Milanesio daefa69502 PackFile: report correct message for checksum mismatch
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.

Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce59)
2019-03-06 02:36:38 +01:00
David Pursehouse e6fd4732d0 ObjectDirectory: Clean up logging
Externalize the message and log the pack file with absolute path.

Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit 9665d86ba1)
2019-03-06 02:36:37 +01:00
Luca Milanesio baf1bb20d0 ObjectDirectory: extra logging on packfile exceptions
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.

Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
(cherry picked from commit 962babc4b2)
2019-03-06 02:36:37 +01:00
Matthias Sohn 8f1fd09bea Prepare 4.9.9-SNAPSHOT builds
Change-Id: I0f7238ce8f79063a410c85701020ffce2b1234cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:10:49 +01:00
Matthias Sohn 67af2d46dd JGit v4.9.8.201812241815-r
Change-Id: Ic75e4611cce396b186680bc962f0c73b2ee5fff1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:01:29 +01:00
Matthias Sohn 3dd3fe9ea4 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:36:29 +01:00
Matthias Sohn a65ce1bfd1 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I68a21067705b580b40840f8039001ff1e5273c15
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:27:25 +01:00
Matthias Sohn 391371cad2 Prepare 4.7.8-SNAPSHOT builds
Change-Id: Ic4ab4aae6d87392d1a5a8bbda06384b7a5b64182
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:16:19 +01:00
Matthias Sohn 4d32049bb3 JGit v4.7.7.201812240805-r
Change-Id: I57b6001b1fbd6042299ccd4dd7d70765a46ea10b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 13:50:12 +01:00
Matthias Sohn 193a164925 Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: If637694f80dbd1e774d60c672fe78a6500650bb8
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 13:25:31 +01:00
Matthias Sohn 4ad74d2f9b Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I0fd67ddd9c4966c20d82cdfe78b2f9d4898b4665
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 12:09:39 +01:00
Matthias Sohn ce76f773f8 Prepare 4.5.6-SNAPSHOT builds
Change-Id: I57c55187ada6d824b94a17f5a79a5bcff61f9ee9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 12:00:26 +01:00
Matthias Sohn 40366c1cf5 JGit v4.5.5.201812240535-r
Change-Id: I6e89e937c08757887967d91afb39cfbe8372d6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 11:21:27 +01:00
Masaya Suzuki 61f0bd54d0 Call AdvertiseRefsHook before validating wants
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.

This bug was introduced in v2.0.0.201206130900-r~123 (Modify refs in
UploadPack/ReceivePack using a hook interface, 2012-02-08).  Stateful
bidirectional transports are not affected.

Fix it by moving the AdvertiseRefsHook call to
getAdvertisedOrDefaultRefs, ensuring the hook is called in all cases.

[jn: backported to stable-4.5 by splitting out tests and the protocol v2
 specific parts]

Change-Id: I159f396216354f2eda3968d17802e166d8c8ec2d
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 10:58:43 +01:00
Matthias Sohn 7ae004509c Prepare 4.9.8-SNAPSHOT builds
Change-Id: Iab7b0f2a21a76f801a83fbeb54f09d1525878f4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-20 00:10:02 +02:00
Matthias Sohn 97534056fc JGit v4.9.7.201810191756-r
Change-Id: Ib980731eb9ec7f4c92deac9b4cf2f7a994d670c2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-19 23:58:47 +02:00
Matthias Sohn 97b62cf9dd Update API problem filter
Change-Id: Iba8650900d3936edccbe30de9d3eafd4f9e0dba6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-19 23:54:38 +02:00
Matthias Sohn d6c587faef Prepare 4.7.7-SNAPSHOT builds
Change-Id: I131e40cbbf32bc75d2ed24ca31ba3613a3bbca25
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-19 22:27:47 +02:00
Matthias Sohn b811e4fc51 JGit v4.7.6.201810191618-r
Change-Id: Iaee27d5e2de016240f1a0940653fef82c6044a1d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-19 22:18:38 +02:00
Matthias Sohn 2674bec32b Update API problem filter
Change-Id: I66c0c490bb66adafac5d670d2d9571f1674a281e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-10-19 22:15:39 +02:00
David Pursehouse 33744a2dfe Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Replace Findbugs with Spotbugs in org.eclipse.jgit/pom.xml
  Replace FindBugs with SpotBugs

Change-Id: I24417e4ebbba31f7ff6896d585ef807327411392
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-13 09:18:03 +09:00
David Pursehouse 27f7a590d1 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Replace Findbugs with Spotbugs in org.eclipse.jgit/pom.xml
  Replace FindBugs with SpotBugs

Change-Id: I1c077e8f3530ac717b1603d3307fd15d4335b8fe
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-13 09:16:55 +09:00
David Pursehouse 2db49bc679 Replace Findbugs with Spotbugs in org.eclipse.jgit/pom.xml
Change-Id: If9cb0de7a0e7bd95eac7daeee140a18385192a48
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-13 09:14:57 +09:00
Jonathan Nieder 9d91bf189e Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  SubmoduleValidator: Remove unused import of ConfigConstants

Change-Id: Id01fd9945e23a9343f96c42db89d5a1b302dad33
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-08 23:39:18 +00:00
Jonathan Nieder f282aaa137 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  SubmoduleValidator: Remove unused import of ConfigConstants

Change-Id: I6a7d19602d6d9099928ee52de727743b9717f262
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-08 23:38:59 +00:00
David Pursehouse 26e7a74601 SubmoduleValidator: Remove unused import of ConfigConstants
Change-Id: I6afe5690bf9d1f1f4d414aa618daefc8b48d217e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-09 08:29:26 +09:00
Jonathan Nieder 8ef8b71db9 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  SubmoduleValidator: Permit missing path or url

Change-Id: I989ac2c0c124c82385fc7cac7e0544ec225f7589
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-08 21:49:24 +00:00
Jonathan Nieder 830e0d6b8c Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  SubmoduleValidator: Permit missing path or url

Change-Id: I94fdaf45abbf7665f9eddc14b1a7f7144aafeadf
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-08 21:49:09 +00:00
Jonathan Nieder d3eaf1007b SubmoduleValidator: Permit missing path or url
A .gitmodules file can include a submodule without a path to configure
the URL for a submodule that is only present on other branches.

A .gitmodules file can include a submodule with no URL and no path to
reserve the name for a submodule that existed in earlier history but
is not available from any URL any more.

"git fsck" permits both of these cases.  Permit them in JGit as well
(instead of throwing NullPointerException).

Change-Id: I3b442639ad79ea7a59227f96406a12e62d3573ae
Reported-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-08 21:40:56 +00:00
Jonathan Nieder acd35d5384 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  ObjectChecker#getGitsubmodules: Fix malformed javadoc

Change-Id: Ic73df5571e1ae86e5e026d0bf9d8e9e330cbfa97
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-10-07 03:38:26 +00:00
David Pursehouse f6eb785551 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  ObjectChecker#getGitsubmodules: Fix malformed javadoc

Change-Id: I51ddd8fa1c2fbcabfb010bc4662d2c925563b55a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-07 11:04:24 +09:00
David Pursehouse df8bd762a1 ObjectChecker#getGitsubmodules: Fix malformed javadoc
The text "<tree, blob>" with angle brackets should not be used in javadoc
since it is interpreted as an HTML tag and then rejected since it's not a
valid HTML tag. Wrap the text in a @literal tag.

Also add a missing space.

Change-Id: Ide045e8c04a39a916f5b2e964e58c151e4555830
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-10-07 09:20:31 +09:00