Commit Graph

4974 Commits

Author SHA1 Message Date
Matthias Sohn 534ede2d81 Fix GC to delete empty fanout directories after repacking
The prune method did not delete empty fanout directories when loose
objects moved to a new pack file but only when loose unreferenced
objects were pruned.

Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-20 14:10:30 +01:00
Sebastian Schuberth d9e1653825 DescribeCommand: Add a missing dots at the end of docs
Ensure the sentences to document fields consistenly end with a dot.

Change-Id: I084899cafe4dea6f2424132dce9f3adca2e20985
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2019-03-18 09:51:37 +01:00
Thomas Wolf d9d9862402 Replace "Checkout" by "Check out" when used as a verb
One occurrence in core JGit, several in jgit.pgm. One unused
occurrence in jgit.pgm; remove it.

Change-Id: I04c3dd9d9f542f1e1ac6df4cbf03bcefb6bfdf78
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-03-15 21:08:46 +01:00
Ivan Frade 23c3702686 BaseReceive/UploadPack: Stop using deprecated RefAdvertiser.send(Map)
RefAdvertiser.send(Map<String, Ref>) is deprecated in favour of
RefAdvertiser.send(Collection<Ref>). Subclasses that need to override
the "send" method need to override also the deprecated version, because
it is still invoked by BaseReceivePack and UploadPack.

Remove the last usages of the deprecated method.

Change-Id: I7eba426970251f78801ddf96b87a65d1baaebdcf
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-03-14 11:27:27 -07:00
Matthias Sohn e53b98ba2f Merge branch 'stable-5.3'
* stable-5.3:
  Prepare 5.3.1-SNAPSHOT builds
  JGit v5.3.0.201903130848-r
  Prepare 5.1.7-SNAPSHOT builds
  JGit v5.1.6.201903130242-r
  Prepare 4.11.8-SNAPSHOT builds
  JGit v4.11.7.201903122105-r
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: Ief9f6a9e89c4a1ef122daedbfd933676393fab06
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 17:49:42 +01:00
Matthias Sohn 4ade2de34f Prepare 5.3.1-SNAPSHOT builds
Change-Id: I8dc5e47bd19809d684579610edf922abca3cb3cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 15:10:33 +01:00
Matthias Sohn de4a633201 JGit v5.3.0.201903130848-r
Change-Id: I52256a6d07011030698299d9a97dd01ea825fb7f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 13:47:54 +01:00
Matthias Sohn 9e74c49808 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.7-SNAPSHOT builds
  JGit v5.1.6.201903130242-r
  Prepare 4.11.8-SNAPSHOT builds
  JGit v4.11.7.201903122105-r
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: Ia866587a910e620cb297bcb2a81fe0786f4e3db2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 10:26:22 +01:00
Matthias Sohn 6a7a265b4d Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.7-SNAPSHOT builds
  JGit v5.1.6.201903130242-r
  Prepare 4.11.8-SNAPSHOT builds
  JGit v4.11.7.201903122105-r
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I8da2761ed27ff2817a4136f843b9b293e3c0f4e6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 07:56:47 +01:00
Matthias Sohn 639d53721c Prepare 5.1.7-SNAPSHOT builds
Change-Id: Ica716b2ad18f8390e75466430ef81c86ea06f004
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 07:49:33 +01:00
Matthias Sohn a31756fbfc JGit v5.1.6.201903130242-r
Change-Id: Iaf5e451f72dc8512afee99a9f3fb91aa348e3cf5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 07:41:27 +01:00
Matthias Sohn b0d27cdae9 Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  Prepare 4.11.8-SNAPSHOT builds
  JGit v4.11.7.201903122105-r
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I71771dcc086a9a0097bad74d65ec71ad97106a38
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 07:38:38 +01:00
Matthias Sohn 7480737aac Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Prepare 4.11.8-SNAPSHOT builds
  JGit v4.11.7.201903122105-r
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I04c70f66c6f1e53bdc199fb5b24c9136c9a8e4f7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 02:21:54 +01:00
Matthias Sohn 6a30ffd033 Prepare 4.11.8-SNAPSHOT builds
Change-Id: I92789b0c02e83786961b81556bf2faec5d26b0c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 02:13:36 +01:00
Matthias Sohn a888f80856 JGit v4.11.7.201903122105-r
Change-Id: I4fc62c304b5a8452ca72a3b305e5732a26120000
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 02:04:31 +01:00
Matthias Sohn 7204e35f93 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: Ic66dc00455c05f7627919f682a7221ffeef1629d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:55:24 +01:00
Matthias Sohn f53c2d11af Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Prepare 4.9.10-SNAPSHOT builds
  JGit v4.9.9.201903122025-r
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: Ifcd8ea6a2b59f6f8e9d896d8407cfcaef0a9375d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:48:06 +01:00
Matthias Sohn 2b64345f53 Prepare 4.9.10-SNAPSHOT builds
Change-Id: I2bdbe822ee623c5f15a26d092136d8602cef06ac
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:34:49 +01:00
Matthias Sohn c28c3ab9f0 JGit v4.9.9.201903122025-r
Change-Id: I807d24461b3557eee48294bfaa652b2a13d78113
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:27:02 +01:00
Matthias Sohn c452c007ce Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:18:01 +01:00
Matthias Sohn 2bf3cc56db Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.9-SNAPSHOT builds
  JGit v4.7.8.201903121755-r
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13 01:16:28 +01:00
Matthias Sohn 317d747c5c Prepare 4.7.9-SNAPSHOT builds
Change-Id: I1602a6b811fd9f95e284153b560d8d9f855fae9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 23:26:43 +01:00
Matthias Sohn a2951a996f JGit v4.7.8.201903121755-r
Change-Id: Ia6dbde615671ed09789326aacad1655e7a7a59ad
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:55:42 +01:00
Matthias Sohn 92525b584a Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:39:53 +01:00
Matthias Sohn d96eb8b3e6 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.7-SNAPSHOT builds
  JGit v4.5.6.201903121547-r
  Check for packfile validity and fd before reading
  Move throw of PackInvalidException outside the catch
  Use FileSnapshot to get lastModified on PackFile
  Include size when comparing FileSnapshot
  Do not reuse packfiles when changed on filesystem
  Silence API warnings for new API introduced for fixes

Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 22:26:58 +01:00
Matthias Sohn a47367e5fb Prepare 4.5.7-SNAPSHOT builds
Change-Id: I5c275c542e12746c3d8ecf8462791969f9e89e12
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 20:54:12 +01:00
Matthias Sohn 1cb31111f7 JGit v4.5.6.201903121547-r
Change-Id: I5a071ed10e1ac1ab28f992d45cde335c12556a80
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 20:47:04 +01:00
Luca Milanesio bf3d1ded35 Check for packfile validity and fd before reading
When reading from a packfile, make sure that is valid
and has a non-null file-descriptor.

Because of concurrency between a thread invalidating a packfile
and another trying to read it, the read() may result into a NPE
that won't be able to be automatically recovered.

Throwing a PackInvalidException would instead cause the packlist
to be refreshed and the read to eventually succeed.

Bug: 544199
Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 15:06:05 +01:00
Luca Milanesio afef866a44 Move throw of PackInvalidException outside the catch
When a packfile is invalid, throw an exception explicitly
outside any catch scope, so that is not accidentally caught
by the generic catch-all cause, which would set the packfile
as valid again.

Flagging an invalid packfile as valid again would have
dangerous consequences such as the corruption of the in-memory
packlist.

Bug: 544199
Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 15:06:05 +01:00
Luca Milanesio 2d116cd0ab Use FileSnapshot to get lastModified on PackFile
Do not redundantly call File.lastModified() for extracting the
timestamp of the PackFile but rather use consistently the FileSnapshot
which reads all file attributes in a single bulk call.

Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 15:06:04 +01:00
Luca Milanesio 2dc572df24 Include size when comparing FileSnapshot
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).

Read and consider file size also, so that differing file size can help
to more accurately detect file changes without reading the file content.
Use bulk read to avoid multiple stat calls to retrieve file attributes.

Change-Id: I974288fff78ac78c52245d9218b5639603f67a46
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 14:39:14 +01:00
Luca Milanesio fef782128d Do not reuse packfiles when changed on filesystem
The pack reload mechanism from the filesystem works only by name
and does not check the actual last modified date of the packfile.

This lead to concurrency issues where multiple threads were loading
and removing from each other list of packfiles when one of those
was failing the checksum.

Rely on FileSnapshot rather than directly checking lastModified
timestamp so that more checks can be performed.

Bug: 544199
Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12 14:30:36 +01:00
Matthias Sohn d4078dccda Silence API warnings for new API introduced for fixes
Change-Id: I3ea7ff2efd33ca6c780afaef9010cec82780d7fa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 14:30:20 +01:00
Matthias Sohn 51f2979f13 Merge branch 'stable-5.3'
* stable-5.3:
  Reduce contention on PackFile.idx() function.
  Use SystemReader in JSchConfigSessionFactoryTest
  Avoid NPE in ObjectId.isId()

Change-Id: I1d13f6fb705258ae6d6e5fa5e733bfacd4f3d0e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:14:14 +01:00
Matthias Sohn da8e47ddc1 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Reduce contention on PackFile.idx() function.

Change-Id: I6bf4c1db695b8fa134ea425bbd488d2dc5438152
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:13:06 +01:00
Matthias Sohn 17e9ec4544 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Reduce contention on PackFile.idx() function.

Change-Id: I3f981dd923209e4d2d23f3b10db9fda1e9e68104
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:12:04 +01:00
Matthias Sohn 57f3ee4c16 Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  Reduce contention on PackFile.idx() function.

Change-Id: Ic50f375faa757076e2dfd6c25e9e0025482aa3d9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:10:59 +01:00
Matthias Sohn 35e96348d3 Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Reduce contention on PackFile.idx() function.

Change-Id: Ib1be8c04c9587c595f7d95df26f7be9b237bda40
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:09:45 +01:00
Matthias Sohn 495ecda567 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Reduce contention on PackFile.idx() function.

Change-Id: I5dd7576018ab2e85d77d336f97c8e77ad71520c9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:08:34 +01:00
Matthias Sohn 47d7f7aabb Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Reduce contention on PackFile.idx() function.

Change-Id: I277e53aa752c8ffb8560de710d27ecb58871ec02
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12 00:06:17 +01:00
Juergen Denner 19c4380689 Reduce contention on PackFile.idx() function.
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.

Performance numbers are listed in the bug report.

Bug: 543739
Change-Id: Iff328d347fa65ae07ecce3267d44184161248978
Signed-off-by: Juergen Denner <j.denner@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-11 19:52:26 +01:00
Michael Keppler 391c7a25fa Avoid NPE in ObjectId.isId()
That method can easily be invoked with a null argument (e.g.
isId(repo.getFullBranch()), therefore it should handle null arguments.

Change was suggested in https://git.eclipse.org/r/#/c/137918/, which
tried to fix the same in egit only.

Bug:544982
Change-Id: I32d1df6e9b2946ab324eda7008721159019316b3
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-03-09 19:54:29 -05:00
Michael Keppler 90e614a41d Fix Javadoc typo
Change-Id: I3f169eaf649843e954762bb4ac8f72fa5f88a67a
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-03-08 14:43:30 +01:00
Jonathan Nieder ee5f449bca UploadPack: Do not retain commit body when checking wants
The commit body contains the commit message, which is not needed for
reachability checks.

Change-Id: Ie209c3b3f022579942f05b8b5d0625ce26400a5d
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-03-07 19:58:35 -08:00
Ivan Frade a33847b5d9 FetchV2Request.getWantedRefs: Bump @since to 5.4
The visibility change was merged after 5.3 release. The @since line
is wrong.

Fixing it in a new commit looks like the easiest way forward.

Change-Id: I6f72a9e684e99a4440cda1eb532c22b4b7bbdea5
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-03-07 17:38:39 -08:00
Jonathan Nieder 94c8fd55e3 Merge "FetchV2Request: make getWantedRefs public" 2019-03-07 17:28:05 -05:00
Matthias Sohn f8a632657e Merge "Prepare 5.4.0-SNAPSHOT builds" 2019-03-07 11:16:46 -05:00
Matthias Sohn 90107d53eb Prepare 5.4.0-SNAPSHOT builds
Change-Id: I90a4791f63d0eba23da744c720e869f1830b86e7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-07 00:34:14 +01:00
Minh Thai cc2a223fea Do not retain commit body in RevWalk for reachability checks
Commit body contains the message that is not needed for reachability checks, and
takes up memory unnecessarily.

Change-Id: I0c7f6da249bf9c4fda9dc9e62e809322c68effce
Signed-off-by: Minh Thai <mthai@google.com>
2019-03-06 15:29:59 -08:00
Matthias Sohn 0295ee1164 Prepare 5.3.0-SNAPSHOT builds
Change-Id: I9b934bcfad3d2091f4da163170348b0c8ef5e732
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-06 22:26:03 +01:00
Matthias Sohn aca889f8cc JGit v5.3.0.201903061415-rc1
Change-Id: I85866af97fe1c0c0a0da83a7605484082fa56b52
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-06 20:03:00 +01:00
Terry Parker e48410ae9a Track object inflation time in DfsReaderIoStats
This can help track down poor long tail performance that isn't accounted
for in the readIdxMicros or readBlockMicros metrics.

Change-Id: I701b9cfcc124f4ddb860d1766a11ea3557e604ce
Signed-off-by: Terry Parker <tparker@google.com>
2019-03-06 10:54:52 -08:00
David Pursehouse 01cf73c7ab Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Fix error log message in ObjectDirectory.handlePackError()
  Properly format pack checksums in PackFile.idx()
  Cancel gc if thread was interrupted
  PackFile: report correct message for checksum mismatch
  ObjectDirectory: Clean up logging
  Bazel: Stop using native.git_repository
  ObjectDirectory: extra logging on packfile exceptions

Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-06 14:15:36 +09:00
David Pursehouse 3fc9d43339 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Fix error log message in ObjectDirectory.handlePackError()
  Properly format pack checksums in PackFile.idx()
  Cancel gc if thread was interrupted
  PackFile: report correct message for checksum mismatch
  ObjectDirectory: Clean up logging
  Bazel: Stop using native.git_repository
  ObjectDirectory: extra logging on packfile exceptions

Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-06 14:14:40 +09:00
David Pursehouse ae5ea80363 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Fix error log message in ObjectDirectory.handlePackError()
  Properly format pack checksums in PackFile.idx()
  Cancel gc if thread was interrupted
  PackFile: report correct message for checksum mismatch
  ObjectDirectory: Clean up logging
  Bazel: Stop using native.git_repository
  ObjectDirectory: extra logging on packfile exceptions

Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-06 14:13:48 +09:00
Matthias Sohn 7b3ee6f62e Fix error log message in ObjectDirectory.handlePackError()
Change-Id: I154f392ad025c4b642eb1123d375a0afaa853885
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 997d785418)
2019-03-06 02:36:38 +01:00
Matthias Sohn e8f7bb3608 Properly format pack checksums in PackFile.idx()
Change-Id: Id805850dbe9a3d633168f3056e06ddeafd86f961
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit a33e4dc58a)
2019-03-06 02:36:38 +01:00
Matthias Sohn e8bd9556c0 Cancel gc if thread was interrupted
see
https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ

Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 882fed0d96)
2019-03-06 02:36:38 +01:00
Luca Milanesio daefa69502 PackFile: report correct message for checksum mismatch
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.

Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce59)
2019-03-06 02:36:38 +01:00
David Pursehouse e6fd4732d0 ObjectDirectory: Clean up logging
Externalize the message and log the pack file with absolute path.

Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit 9665d86ba1)
2019-03-06 02:36:37 +01:00
Luca Milanesio baf1bb20d0 ObjectDirectory: extra logging on packfile exceptions
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.

Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
(cherry picked from commit 962babc4b2)
2019-03-06 02:36:37 +01:00
David Pursehouse 52179ee247 Merge branch 'stable-5.2'
* stable-5.2:
  Fix error log message in ObjectDirectory.handlePackError()
  Properly format pack checksums in PackFile.idx()
  Cancel gc if thread was interrupted
  PackFile: report correct message for checksum mismatch
  ObjectDirectory: Clean up logging
  Bazel: Stop using native.git_repository

Change-Id: Ib972641105cef8089791fc65389c4f43e218620a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-05 10:24:34 +09:00
David Pursehouse c2dbdabfb9 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix error log message in ObjectDirectory.handlePackError()
  Properly format pack checksums in PackFile.idx()
  Cancel gc if thread was interrupted
  PackFile: report correct message for checksum mismatch
  ObjectDirectory: Clean up logging
  Bazel: Stop using native.git_repository

Change-Id: I42b72a3bba3b4c9389d91a1a35a8004836567e7c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-05 10:23:38 +09:00
Matthias Sohn 997d785418 Fix error log message in ObjectDirectory.handlePackError()
Change-Id: I154f392ad025c4b642eb1123d375a0afaa853885
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-04 13:51:12 +01:00
Matthias Sohn a33e4dc58a Properly format pack checksums in PackFile.idx()
Change-Id: Id805850dbe9a3d633168f3056e06ddeafd86f961
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-04 13:28:40 +01:00
Matthias Sohn 882fed0d96 Cancel gc if thread was interrupted
see
https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ

Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-03 18:35:04 +01:00
Luca Milanesio 436c99ce59 PackFile: report correct message for checksum mismatch
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.

Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-03 17:52:33 +01:00
David Pursehouse 9665d86ba1 ObjectDirectory: Clean up logging
Externalize the message and log the pack file with absolute path.

Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-03 13:15:19 +09:00
Dave Borowitz ea54b6a5a7 ResolveMerger: Destroy TemporaryBuffer on unchecked exceptions
Previously, we called destroy() to delete the temp file on failure only
when catching an IOException, not a RuntimeException. Use a slightly
different construction with a finally block to ensure it's always
deleted on error (assuming the JVM is still healthy enough).

Change-Id: Ie201f3cfc81099ee1cafed066632da76223cef1f
2019-03-01 12:35:38 -08:00
Ivan Frade e1b1e01bec FetchV2Request: make getWantedRefs public
There are valid cases where a hook, invoked by ProtocolV2Hook
and probably implemented in a different package, is interested
in knowing the wanted refs in the request.

Increase visibility to public in the wanted-ref method.

Change-Id: I5da085ac7af4c396c1cb85e630f40a57fc70b33e
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-02-28 23:49:19 -08:00
Terry Parker 875dccf33c Expose the filter blob limit in UploadPack
Similar to UploadPack.getDepth() to know the shallow clone depth, expose
the user-specified filter blob limit for partial clones.

Change-Id: I04bde06862a1cf8a9862d950c15023c49d16a2a6
Signed-off-by: Terry Parker <tparker@google.com>
2019-02-28 16:00:24 -08:00
Thomas Wolf f491af08ea Merge changes from topic 'rebase_compatibility'
* changes:
  RebaseCommand: tighten check for --preserve-merges on --continue
  RebaseCommand: use orig-head to abort
  RebaseCommand: use orig-head in addition to head
2019-02-26 16:22:22 -05:00
Jonathan Nieder 2fe1f0d638 Merge "Strongly reference indices in DfsPackFile" 2019-02-26 15:08:39 -05:00
Gunnar Wagenknecht 98df7ec801 Merge "On Windows use %APPDATA%\gnupg as GPG directory if it exists" 2019-02-26 14:46:51 -05:00
Christian Halstrick 20b633d95d Merge "Remove duplicate externalized message" 2019-02-26 07:35:52 -05:00
Thomas Wolf ca011107b8 On Windows use %APPDATA%\gnupg as GPG directory if it exists
Hard-coding ~/.gnupg for the GPG directory doesn't work on Windows,
where GnuPG uses %APPDATA%\gnupg by default. Make the determination
of the directory platform-dependent.

Bug: 544797
Change-Id: Id4bfd39a981ef7c5b39fbde46fce9a7524418709
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 12:49:59 +01:00
Thomas Wolf 4169a95a65 Remove duplicate externalized message
Instead of a new "unexpectedNlinkValue" message use the already
existing "failedAtomicFileCreation". Remove a stray double quote
from the latter.

Change-Id: I1ba5e9ea48d3f7615354b2ace2575883070b3206
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 11:53:49 +01:00
Thomas Wolf 38e5a75ce4 RebaseCommand: tighten check for --preserve-merges on --continue
With native git, .git/rebase-merge/rewritten exists actually in two
different cases:

* as a file in git rebase --merge recording OIDs for copying notes
* as a directory in git rebase --preserve-merges

Add a comment, and check for isDirectory() instead of exists().

Bug: 511487
Change-Id: I6a3317b4234d4f41c41b3004cdc7ea0abf2c6223
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 16:18:27 +09:00
Thomas Wolf a64fa0bd7f RebaseCommand: fix ONTO_NAME, and --preserve-merges is interactive
ONTO_NAME must be "onto_name", not "onto-name".

For native git, --preserve-merges is an interactive mode. Create the
INTERACTIVE marker file, otherwise a native git rebase --continue
will fall back into rebase --merge mode before git 2.19.0 since it
started looking for the REWRITTEN directory to make the distinction
only then.[1]

This allows a JGit interactive rebase to be continued via native git
rebase --continue.

[1] https://github.com/git/git/commit/6d98d0c0

Bug: 511487
Change-Id: I13850e0fd96ac77d03fbb581c8790d76648dbbc6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 16:18:27 +09:00
Thomas Wolf 4ad53c0cbf RebaseCommand: use orig-head to abort
Aborting a rebase used ORIG_HEAD to reset. Strictly speaking this is
not correct, since other commands run during the rebase (for instance,
when the rebase stopped on a conflict) might have changed ORIG_HEAD.

Prefer the OID recorded in the orig-head file, falling back to the
older "head" file if "orig-head" doesn't exist, and use ORIG_HEAD only
if neither exists.

Bug: 511487
Change-Id: Ifa99221bb33e4e4754377f9b8f46e76c8936e072
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 16:18:27 +09:00
Thomas Wolf 60cf85a4a3 Fix core.autocrlf for non-normalized index
With text=auto or core.autocrlf=true, git does not normalize upon
check-in if the file in the index contains already CR/LFs. The
documentation says: "When text is set to "auto", the path is
marked for automatic end-of-line conversion. If Git decides that
the content is text, its line endings are converted to LF on
checkin. When the file has been committed with CRLF, no conversion
is done."[1]

Implement the last bit as in canonical git: check the blob in the
index for CR/LFs. For very large files, we check only the first 8000
bytes, like RawText.isBinary() and AutoLFInputStream do.

In Auto(CR)LFInputStream, ensure that the buffer is filled as much as
possible for the isBinary() check.

Regarding these content checks, there are a number of inconsistencies:

* Canonical git considers files containing lone CRs as binary.
* RawText checks the first 8000 bytes.
* Auto(CR)LFInputStream checks the first 8096 (not 8192!) bytes.

None of these are changed with this commit. It appears that canonical
git will check the whole blob, not just the first 8k bytes. Also
note: the check for CR/LF text won't work with LFS (neither in JGit
nor in git) since the blob data is not run through the smudge filter.
C.f. [2].

Two tests in AddCommandTest actually tested that normalization was
done even if the file was already committed with CR/LF.These tests
had to be adapted. I find the git documentation unclear about the
case where core.autocrlf=input, but from [3] it looks as if this
non-normalization also applies in this case.

Add new tests in CommitCommandTest testing this for the case where
the index entry is for a merge conflict. In this case, canonical git
uses the "ours" version.[4] Do the same.

[1] https://git-scm.com/docs/gitattributes
[2] https://github.com/git/git/blob/3434569fc/convert.c#L225
[3] https://github.com/git/git/blob/3434569fc/convert.c#L529
[4] https://github.com/git/git/blob/f2b6aa98b/read-cache.c#L3281

Bug: 470643
Change-Id: Ie7310539fbe6c737d78b1dcc29e34735d4616b88
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 16:18:27 +09:00
Thomas Wolf b4c212309b RebaseCommand: use orig-head in addition to head
Since 2011-02-10 (i.e., git 1.7.6)[1] native git uses "orig-head" for
REBASE_HEAD. JGit was still using "head". Currently native git has a
legacy fall-back for reading this, but for how long? Let's write to
both. Note that JGit never reads this file.

[1] https://github.com/git/git/commit/84df4560

Bug: 511487
Change-Id: Id3742bf9bbc0001d850e801b26cc8880e646abfc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-26 16:18:27 +09:00
David Pursehouse e54243278f SHA1: Use externalized message in log
The non-externalized warning message says there is a "possible SHA-1
collision" but then the Sha1CollisionException is always thrown.

Replace the message with the existing externalised string that does
not say "possible".

Change-Id: I9773ec76b416c356e234a658fb119f98d33eac83
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-02-25 11:05:44 +09:00
David Pursehouse a5a62a4cdb JGitText: Remove unnecessary suffix from externalized message identifier
Change-Id: I7a3ade56c219826b81d33d1d52566a86ad2e853a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-02-25 11:05:44 +09:00
David Pursehouse 6305869bf1 FS_POSIX: Externalize log message
Change-Id: Ida538e05c4595fdce99f1fa87469a3be60160adb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-02-25 11:05:42 +09:00
Minh Thai 01f084f5d9 Strongly reference indices in DfsPackFile
DfsBlockCache.Ref might get cleared out if the JVM is running out of
memory. As a result, the index is not persisted for the request and
will be reloaded unnecessarily.

Change-Id: I3b57ad5e6673f77f2dc66177a649ac412a97fe20
Signed-off-by: Minh Thai <mthai@google.com>
2019-02-21 16:48:43 -08:00
David Pursehouse 2902255526 Merge branch 'stable-5.3'
* stable-5.3:
  ObjectDirectory: extra logging on packfile exceptions
  Prepare 5.3.0-SNAPSHOT builds
  JGit v5.3.0.201901162155-m1
  Fix tools/version.sh
  JGit v5.3.0.201901161700-m1

Change-Id: I3c511fbc08f21eec5bb49d443de33d43094da4a4
2019-02-18 11:22:31 +09:00
David Pursehouse 9d02e52406 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  ObjectDirectory: extra logging on packfile exceptions

Change-Id: I4bbeb1d28ab042e3c6c74f7284f5d882ed034305
2019-02-18 11:21:38 +09:00
David Pursehouse 90876890bc Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  ObjectDirectory: extra logging on packfile exceptions

Change-Id: Id2fea30664d53b31ad026ee5ae06a5a24d731a52
2019-02-18 11:20:56 +09:00
Matthias Sohn 3ae22bdf66 Add missing @since tag for new API method getUpdateIndex
Change-Id: Idd4a580e241a9105a55bc6f0457ad02a35e55faf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-02-17 20:58:25 -05:00
Matthias Sohn ae79969715 Fix NPE in PlotCommitList
Bug: 544513
Change-Id: I2fca766bbdc441a2dca947fba5dde532de7a10fd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-02-18 01:25:12 +01:00
David Pursehouse 0368906353 CommitBuilder: Deprecate setEncoding(String)
Deprecate the method in favor of setEncoding(Charset).

Update the only caller in the code base that was still using
the deprecated variant.

Change-Id: I6357f2d0c727007013c72e9d5b7c72a3f5f3f2b1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-02-18 00:14:35 +01:00
David Pursehouse e5c4057467 CommitBuilder: Add missing periods on methods' Javadoc
Change-Id: Ibcf64b23de8534f7831d6e40ee4b0ad2bade74ce
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-02-18 00:14:34 +01:00
Luca Milanesio 962babc4b2 ObjectDirectory: extra logging on packfile exceptions
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.

Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-02-14 10:44:00 +09:00
Minh Thai 2b7fa04dad Fix bug in copyPackBypassCache's skip 'PACK' header logic
Bug caused the pack to be 12 bytes short when cold cache.  Also added
test for copyPackAsIs method.

Change-Id: Idf8fb0e50d1215245d4b032e2e00df4b218c115f
Signed-off-by: Minh Thai <mthai@google.com>
2019-02-13 12:48:05 -08:00
Thomas Wolf a29d11775c Atomic file creation: hard-linking may not be allowed
Android for instance forbids hard linking via a SELinux
policy. If we can't hard link, the NFS work-around for
atomic file creation cannot work at all. In this case,
fall back to not using the hard-linking mechanism.

Android throws an AccessDeniedException, so we catch that.
The javadoc on Files.createLink() indicates that another
possibility might be a SecurityException, so catch that,
too.

Bug: 543956
Change-Id: I551b7a45f7b2fbbd8cf94f0b7233dbd8a200520e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-02-02 23:23:46 +01:00
Matthias Sohn 8cad84ccfb Fix GC.deleteEmptyRefsFolders
This method tried to iterate spurious files which may exist in the
.git/refs folder, e.g. on Mac a .DS_Store may have been created there by
inspecting the folder using the finder application. This led to a
NotDirectoryException when deleteEmptyRefsFolders tried to create an
iterator for such a file entry. Skip files contained in the refs folder
to ensure the method only tries to iterate contained folders but not
files.

Change-Id: I5f31e733072a35db1e93908a9c69a8891ae5c206
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-28 02:04:45 -05:00
Thomas Wolf e7b4d108e1 Enable cloning only specific tags
Single-branch-clone should be able to clone a single tag. Enhance
CloneCommand to accept also full refs of tags in setBranchesToClone().
Make sure we also include fetch ref specs for the fetch command for
tags. This mimics the behavior of native git's single-branch clone:
git clone --branch <tag> --single-branch <URI>

Bug: 542611
Change-Id: I285cf043751d9b0ba71258ee8214c0e5d1191428
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-01-27 22:56:06 +01:00
David Pursehouse afb6339314 Bazel: Format BUILD file with buildifier
$ buildifier -lint=fix BUILD

Change-Id: Ie2e069db3936ce6a85c4c29e6a0137cda5e21f09
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-01-24 11:43:37 +09:00
Thomas Wolf 447e107069 RenameBranchCommand: more consistent handling of short ref names
Several problems:

* The command didn't specify whether it expected short or full names.
* For the new name, it expected a short name, but then got confused
  if tags or both local and remote branches with the same name existed.
* For the old name, it accepted either a short or a full name, but
  again got confused if a short name was given and a tag with the
  same name existed.

With such an interface, one cannot use Repository.findRef() to
reliably find the branch to rename. Use exactRef() for the new
name as by the time the Ref is needed its full name is known.
For determining the old Ref from the name, do the resolution
explicitly: first try exactRef (assuming the old name is a full
name); if that doesn't find anything, try "refs/heads/<old>" and
"refs/remotes/<old>" explicitly. Throw an exception if the name
is ambiguous, or if exactRef returned something that is not a
branch (refs/tags/... or also refs/notes/...).

Document in the javadoc what kind of names are valid, and add tests.

A user can still shoot himself in the foot if he chooses exceptionally
stupid branch names. For instance, it is still possible to rename a
branch to "refs/heads/foo" (full name "refs/heads/refs/heads/foo"),
but it cannot be renamed further using the new short name if a branch
with the full name "refs/heads/foo" exists. Similar edge cases exist
for other dumb branch names, like a branch with the short name
"refs/tags/foo". Renaming using the full name is always possible.

Bug: 542446
Change-Id: I34ac91c80c0a00c79a384d16ce1e727c550d54e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-01-23 19:10:36 +01:00
Gunnar Wagenknecht c4420d2434 Allow to check for signing key
The new API is intended for UIs to check if signing will be possible or
would fail

Bug: 543579
Change-Id: I6ce1fd4210e46d49dcdf420c99d08c93e022136c
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
2019-01-22 03:38:36 -05:00
Thomas Wolf 55966dc592 Handle premature EOF in BundleFetchConnection
BundleFetchConnection.readLine() must abort on EOF, otherwise
it gets stuck in an endless loop.

Bug: 543390
Change-Id: I4cb3428560277888af114b928950d620bb6564f9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-01-22 01:49:03 -05:00
David Pursehouse a0f69ff387 IO: Open TemporaryBuffer.Heap in try-with-resource
Change-Id: I78a947fd1263b47b3df17bcc6e9b32497e68dd4a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-01-21 14:19:29 +09:00
David Pursehouse d11a40a14c Scanner: Open RevWalk in try-with-resource
Change-Id: Ieeec292c1346edbda2b26f021e75cbc1435bb9f9
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-01-21 13:45:58 +09:00
David Pursehouse b7810bf5a8 RawParseUtils: Avoid import of java.nio.charset.StandardCharsets
The import is only needed because of a reference to it in the Javadoc,
and can be avoided by explicitly specifying the package instead, which
is how it's referenced in other cases (Constants, FileHeader).

Change-Id: I0c6254a9adf1f52fb8f2c04a858b11696ad264f5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-01-20 20:27:12 +09:00
David Pursehouse 91b1fe4687 UploadPack: Suppress false-positive resource leak warning
Change-Id: Ida09cabb11740e6752a6f079d0528d38021fa2dd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-01-20 17:41:59 +09:00
Matthias Sohn 23affa6c77 Prepare 5.3.0-SNAPSHOT builds
Change-Id: I9f0bf3ff2b26f4a8300c98206beb19c2b9d4946a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-17 04:23:42 +01:00
Matthias Sohn 05159ed656 JGit v5.3.0.201901162155-m1
Change-Id: I350925ff3b1fba2c83a448bc4e6d67368573693e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-17 04:17:18 +01:00
Matthias Sohn 8fc2aef89e JGit v5.3.0.201901161700-m1
Change-Id: I935cae5ae09ee5598ebdf6d9209048afec0d42dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-17 02:39:39 +01:00
Medha Bhargav Prabhala 137e91a465 Implement signing commits using BouncyCastle
This also includes a change to generating the jgit CLI jar. Shading is
no longer possible because it breaks the signature of BouncyCastle.
Instead, the Spring Boot Loader Maven plug-in is now used to generate an
executable jar.

Bug: 382212
Change-Id: I35ee3d4b06d9d479475ab2e51b29bed49661bbdc
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-16 10:05:10 +01:00
Gunnar Wagenknecht 2343c688b1 Allow CommitCommand to sign commits
This change introduces the concept of a GpgSigner which will sign
commits. The GpgSigner will be of a specific implementation (eg.,
Bouncycastle or OpenPgP executable). The actual implementation is not
part of this change.

Bug: 382212
Change-Id: Iea5da1e885c039e06bc8d679d46b124cbe504c8e
Also-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
2019-01-16 10:05:09 +01:00
Jonathan Nieder 1131f35cd1 dfs: Remove synchronization in BlockBasedFile#LazyChannel
As explained in 'The "Double-Checked Locking is Broken"
Declaration'[*], Java's memory model does not support double-checked
locking:

 class LazyReadableChannel {
   private ReachableChannel rc = null;

   public ReadableChannel get() {
     if (rc == null) {
       synchronized (this) {
         if (rc == null) {
           rc = new ReadableChannel();
	 }
       }
     }
     return rc;
   }
 }

With JDK 5 and newer, there is a formal memory model that ensures this
works if "rc" is volatile, but it is still not thread-safe without
that.

Fortunately, this ReadableChannelSupplier is never passed between
threads, so it does not need to be thread-safe.  Simplify by removing
the synchronization.

[*] https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

Change-Id: I0698ee6618d734fc129dd4f63fc047c1c17c94a9
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-14 10:55:28 -08:00
Jonathan Nieder 5f355ed4ad UploadPack: Read wanted refs in one shot
This allows scanning through refs once instead of once per ref, which
should make the lookup less expensive for some RefDatabase
implementations.

Change-Id: I1434f834186cc9a6b4e52659e692b1000c926995
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-11 11:58:08 -08:00
Jonathan Nieder ec94268fd4 Merge branch 'stable-5.2'
* stable-5.2:
  Prepare 5.2.2-SNAPSHOT builds
  JGit v5.2.1.201812262042-r
  Prepare 5.1.6-SNAPSHOT builds
  JGit v5.1.5.201812261915-r
  UploadPack: Filter refs used for deepen-not resolution
  UploadPack: Avoid calling AdvertiseRefsHook twice
  Prepare 5.1.5-SNAPSHOT builds
  JGit v5.1.4.201812251853-r
  UploadPack: Filter refs used for want-ref resolution
  UploadPack: Defer want-ref resolution to after parsing
  Call AdvertiseRefsHook for protocol v2
  Prepare 4.11.7-SNAPSHOT builds
  JGit v4.11.6.201812241910-r
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: Ia56348e54d62630d7c50a4747df89516fc5afad9
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-10 20:23:33 -08:00
Minh Thai 513d80db10 Lazily open ReadableChannel in BlockBasedFile.getOrLoadBlock
To avoid opening the readable channel in case of DfsBlockCache
hits. Also cleaning up typos around DfsBlockCache.

Change-Id: I615e349cb4838387c1e6743cdc384d1b81b54369
Signed-off-by: Minh Thai <mthai@google.com>
2019-01-10 14:22:54 -08:00
Matthias Sohn 23c30c6310 Fix warnings for usage of deprecated RefDatabase#getRef
Change-Id: I22a244114ef973b3801cf3b639ff782f21c02f98
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-08 18:41:11 -05:00
Minh Thai 4207b8d6ae DfsBlockCache: Consolidate where ReadableChannel is opened
Opening a readable channel can be expensive and the number of channels
can be limited in DFS. Ensure that caller of
BlockBasedFile.readOneBlock() is responsible for opening and closing
the file, and that the ReadableChannel is reused in the request. As a side
effect, this makes the code easier to read, with better use of
try-with-resources.

The downside is that this means a readable channel is always opened, even
when the entire pack is already available for copying from cache. This
should be an acceptable cost: it's rare enough not to overload the server
and from a client latency perspective, the latency cost is in the noise
relative to the data transfer cost involved in a clone. If this turns out
to be a problem in practice, we can reintroduce that optimization in a
followup change.

Change-Id: I340428ee4bacd2dce019d5616ef12339a0c85f0b
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-08 14:31:56 -05:00
Minh Thai 8bc9acf264 DfsBlockCache to lock while loading object references
We see the same index being loaded by multiple threads. Each is
hundreds of MB and takes several seconds to load, causing server to
run out of memory.  This change introduces a lock to avoid these
duplicate works.  It uses a new set of locks similar in implementation
to the loadLocks for getOrLoad of blocks.  The locks are kept separate
to prevent long-running index loading from blocking out fast block
loading. The cache instance can be configured with a consumer to
monitor the wait time of the new locks.

Change-Id: I44962fe84093456962d5981545e3f7851ecb6e43
Signed-off-by: Minh Thai <mthai@google.com>
2019-01-08 09:59:40 -08:00
Christian Halstrick e406d500de Fix "jgit checkout -f" to overwrite dirty worktree files
CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.

Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.

This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.

Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-08 10:01:09 +01:00
Mario Molina 6c8240a751 Return 'this' from setters in commands
To avoid breaking ABI, take the opportunity to give these setters
(hopefully sometimes better) names and deprecate their old names.

Change-Id: Ib45011678c3d941f8ecc1a1e0fdf4c09cdc337e3
Signed-off-by: Mario Molina <mmolimar@gmail.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-07 20:00:11 -05:00
Jonathan Nieder d83e85736e Don't swallow IOException
Swallowing intermittent errors and trying to recover from them
makes JGit's behavior hard to predict and difficult to debug.

Propagate the errors instead.  This doesn't violate JGit's usual
backward compatibility promise for clients because in these
contexts an IOException indicates either repository corruption or
a true I/O error.  Let's consider these cases one at a time.

In the case of repository corruption, falling back e.g. to an empty
set of refs or a missing ref will not serve a caller well.  The
fallback does not indicate the nature of the corruption, so they are
not in a good place to recover from the error.  This is analogous to
Git, which attempts to provide sufficient support to recover from
corruption (by ensuring commands like "git branch -D" cope with
corruption) but little else.

In the case of an I/O error, the best we can do is to propagate the
error so that the user sees a dialog and has an opportunity to try
again.  As in the corruption case, the fallback behavior does not
provide enough information for a caller to rely on the current error
handling, and callers such as EGit already need to be able to handle
runtime exceptions.

To be conservative, keep the existing behavior for the deprecated
Repository#peel method.  In this example, the fallback behavior is to
return an unpeeled ref, which is distinguishable from the ref not
existing and should thus at least be possible to debug.

Change-Id: I0eb58eb8c77519df7f50d21d1742016b978e67a3
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-07 14:00:28 -05:00
Jonathan Nieder 3f9a5ce13f Deprecate Repository#hasObject
Its implementation contains

		} catch (IOException e) {
			// Legacy API, assume error means "no"
			return false;
		}

Better to use ObjectDatabase#has, which throws IOException to report
errors.

Change-Id: I7de02f7ceb8f57b2a8ebdb16d2aa4376775ff933
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-01-04 17:21:20 -08:00
Michael Keppler d8bb1d09d2 Deprecate RawParseUtils.UTF8-CHARSET
That constant is just a redirection to a java standard constant
meanwhile. It is not referenced anymore in jgit code (and egit is just
removing it). Clients can use the redirection target directly.

Change-Id: I058d013f61da8d7b771c499d8743aafb8faa5ea8
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-04 03:05:39 +01:00
Christian Halstrick 530812d936 Fix detection of "initial checkout"
A checkout done directly after cloning (the "initial
checkout") has a different semantic as a default
checkout. That is defined in the documentation of
"git read-tree" [1]. JGit was detecting that it is
doing an initial checkout differently from native
git: jgit used to check that the index is empty
but native git required that the index file does
not exist [2]. Teach JGit to behave like native
git.

[1] https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L187
[2] https://marc.info/?t=154150811200001&r=1&w=2

Change-Id: I1dd0f1ede7cd7ea60d28607916d0165269a9f628
2019-01-02 21:18:29 -05:00
Michael Keppler 0d9e486714 Replace deprecated FirstLine by FirstCommand
and allow package org.eclipse.jgit.http.server to use package
org.eclipse.jgit.internal.transport.parser.

Change-Id: Ief330c3e75a735853d0a5a265a9ff56fb5128b99
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-31 16:31:37 +01:00
Jonathan Nieder dbf6f9f692 Move BaseReceivePack#advertisedRefs getter and setter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.

Change-Id: If861e28ce512f556e574352fa7d4a0df0984693f
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 11:59:27 +01:00
Jonathan Nieder 939723807e Move BaseReceivePack#walk getter to ReceivePack
Another step toward merging BaseReceivePack into ReceivePack.

Change-Id: I43cf2e36e2d5b0cd85bf23c81469909c14757b63
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 11:59:27 +01:00
Jonathan Nieder aaf41a51b7 Move BaseReceivePack#db getter to ReceivePack
Another step toward eliminating BaseReceivePack as a separate API.

Change-Id: If7b7d5c65a043607a2424211adb479fa33a9077b
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 11:59:27 +01:00
Jonathan Nieder 937beaa1ae Move BaseReceivePack#pushCert getter and setter to ReceivePack
This is a first step toward eliminating the BaseReceivePack API.

Inspired by a larger change by Dan Wang <dwwang@google.com>.

Change-Id: I5c876a67d8db24bf808823d9ea44d991b1ce5277
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 11:59:26 +01:00
Matthias Sohn e78776fa5b Remove unused API problem filters
Change-Id: I8ed98ac60362ba071506b0bdacd2941ab72b6bf4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 11:55:02 +01:00
Matthias Sohn bd6aa22333 Silence API error for RefDatabase.getRef declared final
RefDatabase.getRef was declared final in
c1954f6c36
which only affects implementers.

Change-Id: I4c14232a119670d263d88db2b8d725dcdd36ab2a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 01:07:24 +01:00
Matthias Sohn 7b989eed72 Suppress API errors and add missing @since tags caused by 6ea888a
Change-Id: Id042d46ba078af35b9c22a079da4ae14fc5fa231
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-29 00:17:33 +01:00
Jonathan Nieder 2f9ebeacc8 Merge changes Id3bb9443,I1be1948b
* changes:
  RefDatabase: Introduce findRef synonym for getRef
  RefDirectory: Look up several exact refs in one shot
2018-12-28 15:57:51 -05:00
Jonathan Nieder b4e415f8a9 Move first line parsing for v0 push out of BaseReceivePack
This simplifies the BaseReceivePack class and decreases its API
surface, which should make merging with ReceivePack easier.

Inspired by 6aca8899a5 (Move first line
parsing for v0/v1 pack negotiation out of UploadPack, 2018-09-17).

Change-Id: I1fc175d15aa7cb5968c26fc83a95480403af617c
2018-12-27 14:07:41 -08:00
Jonathan Nieder c1954f6c36 RefDatabase: Introduce findRef synonym for getRef
Using findRef instead of getRef makes it clearer that the caller wants
to search for the ref in the search path, instead of looking for a ref
that exactly matches the input.

This change introduces the new findRef method and deprecates getRef.
It updates Repository#findRef to use the new method, ensuring some
test coverage.  Other callers will be updated in followup changes.

A nice side effect of introducing the new findRef method is that it is
final and based on firstExactRef, so implementers can focus on
implementing the latter efficiently and do not have to carefully write
custom path search code respecting SEARCH_PATH.

Change-Id: Id3bb944344a9743705fd1f20193ab679298fa51c
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 20:49:35 -08:00
Jonathan Nieder d2bab65470 RefDirectory: Look up several exact refs in one shot
Override exactRef(String...) and firstExactRef(String...) with
implementations specific to FileRepository.

The specialized implementations are similar to the generic ones from
RefDatabase, but because these use readRef directly instead of
exactRef, they only need to call fireRefsChanged once.

This will allow replacing RefDirectory#getRef with a generic
implementation that uses firstExactRef without hurting performance.

Change-Id: I1be1948bd6121c1a1e8152e201aab97e7fb308bb
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 20:48:40 -08:00
Jonathan Nieder b2ec6405e4 RefDirectory: Do not use search path to find additional refs
Psuedorefs like FETCH_HEAD and MERGE_HEAD are supposed to be directly
under the .git directory, not in other locations in the SEARCH_PATH
like refs/ and refs/heads/.  Use exactRef to access them.

Change-Id: Iab8ac47008822fa78fc0691e239e518c34d7a98e
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 19:29:27 -08:00
Matthias Sohn 56562221d6 Prepare 5.2.2-SNAPSHOT builds
Change-Id: I9487f6a59ed684ad13bcfbff7d6c730ae4f37030
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-27 03:32:48 +01:00
Matthias Sohn 24e89950e7 JGit v5.2.1.201812262042-r
Change-Id: Idfefc54e898f364407a931b787a26db2489b031c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-27 02:44:17 +01:00
Matthias Sohn fafad4196d Prepare 5.1.6-SNAPSHOT builds
Change-Id: If3313d602aa57cfa031ba2c9ec03bd51b464dc19
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-27 01:35:30 +01:00
Matthias Sohn 510011b05c JGit v5.1.5.201812261915-r
Change-Id: I249a440d1ca1c997343da942ae55fcb27a645b92
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-27 01:16:15 +01:00
Jonathan Nieder 9895338de1 RefDatabase: Remove fallback exactRef implementation
This is simpler to implement than getRef.  Make it abstract so
implementers remember to override it.

Change-Id: I5f319be1fb1206d7a0142ea939dc4e1039f850ab
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 14:33:13 -08:00
Jonathan Nieder f46e223187 RefDirectory: Fire RefsChangedEvent on error, too
getRef and exactRef can produce recoverable exceptions --- for
example, a corrupt loose ref that cannot be parsed.  If readRef was
called and updated looseRefs in the process, RefsChangedEvent should
still be fired.

Noticed while improving the implementation of getRef.  This commit
only affects exactRef and getRef.  Other methods might be similarly
skipping firing RefsChangedEvent in their error handling code, and
this change does not fix them.

Change-Id: I0f460f6c8d9a585ad8453a4a47c1c77e24a1fb83
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 14:30:42 -08:00
Jonathan Nieder 32da5ac3c3 RefDirectory: Refactor getRef and exactRef to share code
Both getRef and exactRef look for a ref or pseudoref in the $GIT_DIR
directory, with careful error handling to handle non-refs like
.git/config.

Avoid the duplication by factoring out a helper that takes care of
this.  This should make the code easier to understand and manipulate.

Change-Id: I2ea67816d2385e84e2d3394b897e23df5826ba50
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 14:25:11 -08:00
Jonathan Nieder eb4c63fbbf UploadPack: Filter refs used for deepen-not resolution
Clients can use --shallow-exclude to obtain information about what
commits are reachable from refs they are not supposed to be able to
see.  Plug the hole by allowing the AdvertiseRefsHook and RefFilter to
take effect here, too.

Change-Id: If2b8e95344fa49e10a6a202144318b60d002490e
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 13:26:39 -08:00
Jonathan Nieder f909de5c44 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  UploadPack: Avoid calling AdvertiseRefsHook twice
  Prepare 5.1.5-SNAPSHOT builds
  JGit v5.1.4.201812251853-r
  UploadPack: Filter refs used for want-ref resolution
  UploadPack: Defer want-ref resolution to after parsing
  Call AdvertiseRefsHook for protocol v2
  Prepare 4.11.7-SNAPSHOT builds
  JGit v4.11.6.201812241910-r
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I5879df9b723a0dbf6a1eff89a34bbb269f3b773d
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 13:26:23 -08:00
Jonathan Nieder 53ab1188d5 Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  UploadPack: Avoid calling AdvertiseRefsHook twice

Change-Id: Ie9ce67921e38fff8338a148c4d9f2776be01d0b0
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 13:24:26 -08:00
Jonathan Nieder c961e2d773 UploadPack: Avoid calling AdvertiseRefsHook twice
The AdvertiseRefsHook can be called twice if the following conditions
hold:

 1. This AdvertiseRefsHook doesn't set this.refs.
 2. getAdvertisedOrDefaultRefs is called after getFilteredRefs.

For example, this can happen when fetchV2 is called after lsRefsV2
when using a stateful bidirectional transport.

The second call does not accomplish anything useful.  Guard it with
'if (!advertiseRefsHookCalled)' to avoid wasted work.

Reported-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Ib746582e4ef645b767a5b3fb969596df99ac2ab5
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-12-26 13:19:01 -08:00
Jonathan Nieder f5bdb9745f Merge changes from topic 'update-index-ref-decorator'
* changes:
  RefCursor: Remove unnecessary getUpdateIndex method
  RefDatabase/Ref: Add versioning to reference database
2018-12-26 13:12:14 -05:00
Ivan Frade 3381bf7e6a RefCursor: Remove unnecessary getUpdateIndex method
Now the reference carries its updateIndex, so the cursor doesn't need
to expose it.

Change-Id: Icbfca46f92a13f3d8215ad10b2a166a6f40b0b0f
Signed-off-by: Ivan Frade <ifrade@google.com>
2018-12-26 09:42:59 -08:00
Ivan Frade 6ea888a036 RefDatabase/Ref: Add versioning to reference database
In DFS implementations the reference table can fall out of sync, but
it is not possible to check this situation in the current API.

Add a property to the Refs indicating the order of its updates.  This
version is set only by RefDatabase implementations that support
versioning (e.g reftable based).

Caller is responsible to check if the reference db creates versioned
refs before accessing getUpdateIndex(). E.g:

   Ref ref = refdb.exactRef(...);
   if (refdb.hasVersioning()) {
       ref.getUpdateIndex();
   }

Change-Id: I0d5ec8e8df47c730301b2e12851a6bf3dac9d120
Signed-off-by: Ivan Frade <ifrade@google.com>
2018-12-26 09:42:59 -08:00
Matthias Sohn d14209e81b Prepare 5.1.5-SNAPSHOT builds
Change-Id: I3eeb24b01a4f955e7e933238defb42635ebfec7d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-26 00:57:01 +01:00
Matthias Sohn c93aec2b73 JGit v5.1.4.201812251853-r
Change-Id: I30ba167bcd8144074553d7446a45fa6be19584fe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-26 00:53:51 +01:00
Jonathan Nieder fcafdcc404 UploadPack: Filter refs used for want-ref resolution
In the longer term, we can add support for this to the
RequestValidator interface.  In the short term, this is a minimal
band-aid to ensure any refs the client requests are visible to the
client.

Change-Id: I0683c7a00e707cf97eef6c6bb782671d0a550ffe
Reported-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-26 00:40:00 +01:00
Jonathan Nieder 1638a2fce8 UploadPack: Defer want-ref resolution to after parsing
ProtocolV2Parser explains:

  // TODO(ifrade): This validation should be done after the
  // protocol parsing. It is not a protocol problem asking for an
  // unexisting ref and we wouldn't need the ref database here.

Do so.  This way all ref database accesses are in one place, in the
UploadPack class.

No user-visible change intended --- this is just to make the code
easier to manipulate.

Change-Id: I68e87dff7b9a63ccc169bd0836e8e8baaf5d1048
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-26 00:39:44 +01:00
Matthias Sohn 25deb30460 Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  Call AdvertiseRefsHook for protocol v2
  Prepare 4.11.7-SNAPSHOT builds
  JGit v4.11.6.201812241910-r
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: Icdc212bf5be2485d0f8028acf6c62fb8531d0e3c
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 23:58:53 +01:00
Masaya Suzuki 8eecb4f8b7 Call AdvertiseRefsHook for protocol v2
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible.

In protocol v2, the hook is not called, causing the server to advertise
all refs.  This bug was introduced in v5.0.0.201805221745-rc1~1^2~9
(Execute AdvertiseRefsHook only for protocol v0 and v1, 2018-05-14).

Even before then, the hook was not called in requests after the
capability advertisement, so in transports like HTTP that do not retain
state between round-trips, the server would advertise all refs in
response to an ls-refs (ls-remote) request.

Fix both cases by using getAdvertisedOrDefaultRefs to retrieve the
advertised refs in lsRefs, ensuring the hook is called in all cases that
use its result.

[jn: backported to stable-5.0; split out from a larger patch that also
 fixes protocol v0; avoided filtering this.refs by ref prefix]

Change-Id: I64bce0e72d15b90baccc235c067e57b6af21b55f
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 23:36:11 +01:00
Matthias Sohn 9caa94239a Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Prepare 4.11.7-SNAPSHOT builds
  JGit v4.11.6.201812241910-r
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: Ie81284ca6d580b0712c49eec610393d0c0c50203
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 23:27:13 +01:00
Matthias Sohn 23e4765a65 Prepare 4.11.7-SNAPSHOT builds
Change-Id: I97dc0f4aeb51c2084f40531ceac84e90bf70608d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 01:08:03 +01:00
Matthias Sohn cc76de549a JGit v4.11.6.201812241910-r
Change-Id: I21bfe7a76450184bce43cbe8ebcaa7891e4b12ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:56:54 +01:00
Matthias Sohn affbb99271 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I937e9a4547fc10e4de7c887163022d1ab0322d64
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:54:15 +01:00
Matthias Sohn 60a606a261 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Prepare 4.9.9-SNAPSHOT builds
  JGit v4.9.8.201812241815-r
  UploadPack: Test filtering by AdvertiseRefsHook in stateless transports
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I2e499f34b1c481af794fa9325b0dfebaccdf3cb0
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:41:41 +01:00
Matthias Sohn 8f1fd09bea Prepare 4.9.9-SNAPSHOT builds
Change-Id: I0f7238ce8f79063a410c85701020ffce2b1234cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:10:49 +01:00
Matthias Sohn 67af2d46dd JGit v4.9.8.201812241815-r
Change-Id: Ic75e4611cce396b186680bc962f0c73b2ee5fff1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-25 00:01:29 +01:00
Matthias Sohn 3dd3fe9ea4 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:36:29 +01:00
Matthias Sohn a65ce1bfd1 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.8-SNAPSHOT builds
  JGit v4.7.7.201812240805-r
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I68a21067705b580b40840f8039001ff1e5273c15
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:27:25 +01:00
Matthias Sohn 391371cad2 Prepare 4.7.8-SNAPSHOT builds
Change-Id: Ic4ab4aae6d87392d1a5a8bbda06384b7a5b64182
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 23:16:19 +01:00
Matthias Sohn 4d32049bb3 JGit v4.7.7.201812240805-r
Change-Id: I57b6001b1fbd6042299ccd4dd7d70765a46ea10b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 13:50:12 +01:00
Matthias Sohn 193a164925 Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Fix feature versions imported by feature org.eclipse.jgit.pgm
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: If637694f80dbd1e774d60c672fe78a6500650bb8
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 13:25:31 +01:00
Matthias Sohn 4ad74d2f9b Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.6-SNAPSHOT builds
  JGit v4.5.5.201812240535-r
  Call AdvertiseRefsHook before validating wants

Change-Id: I0fd67ddd9c4966c20d82cdfe78b2f9d4898b4665
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 12:09:39 +01:00
Matthias Sohn ce76f773f8 Prepare 4.5.6-SNAPSHOT builds
Change-Id: I57c55187ada6d824b94a17f5a79a5bcff61f9ee9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 12:00:26 +01:00
Matthias Sohn 40366c1cf5 JGit v4.5.5.201812240535-r
Change-Id: I6e89e937c08757887967d91afb39cfbe8372d6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 11:21:27 +01:00
Masaya Suzuki 61f0bd54d0 Call AdvertiseRefsHook before validating wants
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit.
If this hook is not called, then all refs are treated as visible,
causing the server to serve commits reachable from branches the client
should not be able to access, if asked to via a request naming a guessed
object id.

This bug was introduced in v2.0.0.201206130900-r~123 (Modify refs in
UploadPack/ReceivePack using a hook interface, 2012-02-08).  Stateful
bidirectional transports are not affected.

Fix it by moving the AdvertiseRefsHook call to
getAdvertisedOrDefaultRefs, ensuring the hook is called in all cases.

[jn: backported to stable-4.5 by splitting out tests and the protocol v2
 specific parts]

Change-Id: I159f396216354f2eda3968d17802e166d8c8ec2d
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24 10:58:43 +01:00
David Pursehouse ccc21c7d4a JGitInternalException: Fix typo in Javadoc
Change-Id: I46fabab3cb2d9164234ce04de346826687a3fa35
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-22 08:06:40 +09:00
Mincong HUANG d09388e156 Avoid double words
Change-Id: I0fdf595cba93f5a5cdd0496cee07ac91db304532
Signed-off-by: Mincong Huang <mincong.h@gmail.com>
2018-12-21 08:07:08 +01:00
Matthias Sohn 8f11539265 Silence API error for 3 digit version number in @since tag
Change-Id: I4943d9338ddf294e2ef67f0d459a3030f528d0c9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-21 01:30:30 +01:00
David Pursehouse e6b4ae2e14 Remove unnecessary @since tags from non-API classes
Change-Id: I6bed174c062a0785641dc8ad69151bf7e843cdcf
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-20 20:02:31 +09:00
David Pursehouse 3e51a078d9 Merge branch 'stable-5.2'
* stable-5.2:
  BasePackConnection: Check for expected length of ref advertisement
  TransferConfig: Make constructors public
  Update last JGit version

Change-Id: I4406d4f68136a2ce363701324b9a842ad468bc59
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-20 15:55:02 +09:00
David Pursehouse 3cb80a433d Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  BasePackConnection: Check for expected length of ref advertisement
  TransferConfig: Make constructors public

Change-Id: I2480a0455250ee381fae93cac2db30f8305fa6aa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-20 14:02:19 +09:00
David Pursehouse f4fc6404ba BasePackConnection: Check for expected length of ref advertisement
When a server sends a ref advertisement using protocol v2 it contains
lines other than ref names and sha1s.  Attempting to get the sha1 out
of such a line using the substring method can result in a SIOOB error
when it doesn't actually contain the sha1 and ref name.

Add a check that the line is of the expected length, and subsequently
that the extracted object id is valid, and if not throw an exception.

Change-Id: Id92fe66ff8b6deb2cf987d81929f8d0602c399f4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-20 11:41:02 +09:00
David Pursehouse 2269669fb1 TransferConfig: Make constructors public
UploadPack has a setTransferConfig method which allows to set the
transfer config, however since the constructors of TransferConfig
have the default package visibility it is not possible for any
application using UploadPack, for example Gerrit, to actually set
a transfer config.

Make the constructors public. This is consistent with the public
constructors for example on PackConfig.

Change-Id: I07080255838421871403b2b2bcc294aa8f621c57
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-20 08:37:45 +09:00
David Pursehouse 38d42d5ccd UploadPack: Rewrite setAdvertiseRefsHook to use ternary operator
This makes the implementation consistent with the other similar
methods in this class.

Change-Id: I007876aad883615d696c8eabc886818ae00b10ee
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-19 15:31:38 +09:00
David Pursehouse fc07fa26b9 UploadPack: Prevent setting null protocolV2Hook
The setProtocolV2Hook sets the protocolV2Hook to whatever value is
passed, which could be null, but the invocations of protocolV2Hook's
methods are not guarded by null-checks.

Annotate the parameter as @Nullable and set ProtocolV2Hook.DEFAULT
when null is passed. This makes the implementation consistent with
other similar methods that set a hook or filter with possible null
value.

Change-Id: I70919a3248d4c2658783941a37c47e437cff0baa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-19 15:31:31 +09:00
David Pursehouse 3d203114bd UploadPack: Add @Nullable annotations
The class has several methods where passing a null parameter is
valid. Annotate those parameters as @Nullable.

Change-Id: Ie08893ee3ab34c1ffb2db875b4ab049ad065c697
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-12-19 15:31:25 +09:00
Matthias Sohn 57b8cf4261 Prepare 5.2.1-SNAPSHOT builds
Change-Id: I68d9f5b1c8ca8e8eada9143093dc4fa82519edd8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-07 01:46:38 +01:00
Matthias Sohn c84c08107d JGit v5.2.0.201812061821-r
Change-Id: I8528c913ad1a820ee9a2fa6be1c9125b518caefe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-07 00:20:43 +01:00
Matthias Sohn 38da21d6b0 Prepare 5.3.0-SNAPSHOT builds
Change-Id: I8951c2cf650cc3e41d2baa0b330b94468cfed5c2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-03 02:32:49 +01:00
Thomas Wolf 450b196627 Include id_ed25519 in the known default identity files
Change-Id: I9fb30c8fbb65536782ee73644c8990eb6661c660
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-12-02 16:35:12 +01:00
Thomas Wolf a08ffb0444 Merge "Set git environment variables for hooks" into stable-5.2 2018-12-02 05:34:18 -05:00
Matthias Sohn 055c312bd1 Prepare 5.2.0-SNAPSHOT builds
Change-Id: I18646aaeee51047b234b758dcc1c2f89fd01b2f8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-11-30 22:57:04 +01:00
Thomas Wolf 9c755c9e7c Set git environment variables for hooks
Set GIT_DIR and GIT_WORK_TREE when calling hooks.

Bug: 541622
Change-Id: I6153d8a6a934ec37a3a5e7319c2d0e516f539ab7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-11-30 00:37:03 +01:00
Matthias Sohn 12d840dfcb JGit v5.2.0.201811281532-m3
Change-Id: I79988b473d67e565d8f6bcac40ee9a14b26c9e60
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-11-28 21:35:26 +01:00
Christian Halstrick 158294f912 Merge "Fix IndexDiffs for git links" 2018-11-28 06:11:05 -05:00
David Pursehouse c5306b84ae Merge branch 'stable-5.1'
* stable-5.1:
  Fix DescribeCommand with multiple match options
  Fix git-describe tie-breakers

Change-Id: I64d9b0099fc0d8449ec08f0b5653572b5c2f840f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-11-27 20:01:45 +09:00
David Pursehouse 9372cf496f Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  Fix DescribeCommand with multiple match options
  Fix git-describe tie-breakers

Change-Id: I63e7a56bb617b5ce8774e1dc7f5efdde25e7cd97
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-11-27 20:01:11 +09:00
David Pursehouse 28f0ef8509 Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Fix DescribeCommand with multiple match options
  Fix git-describe tie-breakers

Change-Id: Ibb98f143ee0ce7635beb30ec404b4134a73788f6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-11-27 19:51:01 +09:00
Matthias Sohn 7c798905ed Remove unused externalized text "submodulesNotSupported"
This text is unused since d72b9328.

Change-Id: I82e5652e205d7b238ff8adc21994029b828b09f4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-11-27 03:48:09 -05:00
Thomas Wolf 4678f4b735 Fix IndexDiffs for git links
After cloning a repo with a submodule, non-recursively, JGit would
encounter in its TreeWalk in IndexDiff:

* first, a missing gitlink (in index & HEAD, not in working tree)
* second, the untracked folder (not in index and head, in working tree)

As a result, it would report the submodule as missing. Canonical git
reports a clean workspace.

The root cause of this is that the path of a gitlink "x" did
not compare equal to the path of a tree "x" in JGit.

Correct Paths.compare() to account for that. If two paths are otherwise
equal, then let gitlinks match both trees and files. Matching trees
solves the bug. Matching files is necessary to handle the case where
the gitlink directory was replaced by a file; see the new test case
IndexDiffSubmoduleTest.testSubmoduleReplacedByFile(). Comparisons of
unequal paths are left untouched, so the sort order is unchanged.

After the fix, another bug(?) in WorkingTreeIterator became apparent:
with core.dirNoGitLinks = true, it was no longer possible to overwrite
a gitlink in the index. This is now fixed in WorkingTreeIterator.

Add new test cases for the bug itself and for some related cases
(submodule directory deleted or replaced by a file) in
IndexDiffSubmoduleTest. Add a test for missing files in IndexDiffTest,
and adapt the PathsTest to test matching gitlinks.

Bug: 467631
Change-Id: I0549d10d46b1858e5eec3cc15095aa9f1d5f5280
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-11-26 20:53:42 +01:00
Matthias Sohn df6263644b Fix DescribeCommand with multiple match options
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.

See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt

Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-11-26 18:49:30 +01:00
Matthias Sohn 3f745e40a1 Add a method to get all values of HTTP header defined as list
According to RFC 2616 [1] header field names are case insensitive.
Header fields defined as a comma separated list can have multiple header
fields with the same field name. Add a method to HttpConnection which
retrieves all values with a given header field name with the field name
compared case insensitive.

[1] https://tools.ietf.org/html/rfc2616#section-4.2"

Change-Id: I7f601b21cda99e84f43f866c7c7cb4cb0e3cf5c3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-11-26 03:07:58 -05:00