Commit Graph

4444 Commits

Author SHA1 Message Date
Matthias Sohn b0332d0133 Ignore API warnings
The following commits introduced in stable-4.5 and stable-4.9
introduced some minor API additions in service releases.

f7ceeaa2 FileRepository: Add pack-based inserter implementation
085d1f95 Make PackInserter public
10e65cb4 Fix LockFile semantics when running on NFS

Change-Id: I4afed7e0395cf93d828e671080e3ec9ddf20987d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-24 17:23:00 +02:00
Ivan Frade 0b84c5b29e UploadPack: shallowExcludeRefs doesn't need to be nullable
Code can check size instead of null, and that makes the initialization
trivial.

Change-Id: Icbe655816429a7a680926b0e871d96f3b2f1f7ba
Signed-off-by: Ivan Frade <ifrade@google.com>
2018-08-23 15:22:06 -07:00
David Pursehouse e1b971848a RefDirectory: Add missing $NON-NLS suppression
Change-Id: I6afb4f86c3a8a06e5ea3128bf1b9b20c46ca158c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-08-23 16:51:19 +09:00
Michael Keppler 2fc00af44e refactor: simplify collection.toArray()
On recent VMs, collection.toArray(new T[0]) is faster than
collection.toArray(new T[collection.size()]). Since it is also more
readable, it should now be the preferred way of collection to array
conversion.

https://shipilev.net/blog/2016/arrays-wisdom-ancients/

Change-Id: I80388532fb4b2b0663ee1fe8baa94f5df55c8442
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2018-08-23 01:34:39 -04:00
David Pursehouse 7058493fea UploadPack: Add missing $NON-NLS suppressions
Change-Id: Ica2cc0ed23a7667bcb8d2006fd0a6ba6bddfee9a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-08-23 11:35:06 +09:00
David Pursehouse 985e6342b3 UploadPack: Remove redundant specification of type
Change-Id: I257635ed3f9ce012e03a3c45ed666fb9195019e0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-08-23 11:33:17 +09:00
Thomas Wolf d9e767b431 Suppress warning for trying to delete non-empty directory
This is actually a fairly common occurrence; deleting the parent
directories can work only if the file deleted was the last one
in the directory.

Bug: 537872
Change-Id: I86d1d45e1e2631332025ff24af8dfd46c9725711
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-22 18:36:31 +02:00
Thomas Wolf ffd1ac5dde Fix fetching with duplicate ref updates
If packed refs are used, duplicate updates result in an exception
because JGit tries to lock the same lock file twice. With non-atomic
ref updates, this used to work, since the same ref would simply be
locked and updated twice in succession.

Let's be more lenient in this case and remove duplicates before
trying to do the ref updates. Silently skip duplicate updates
for the same ref, if they both would update the ref to the same
object ID. (If they don't, behavior is undefined anyway, and we
still throw an exception.)

Add a test that results in a duplicate ref update for a tag.

Bug: 529400
Change-Id: Ide97f20b219646ac24c22e28de0c194a29cb62a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-22 11:09:01 +02:00
Marc Strapetz 65a0cfc82a Fetch(Process): should tolerate duplicate refspecs
Bug: 529314
Change-Id: I91eaeda8a988d4786908fba6de00478cfc47a2a2
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-22 11:09:00 +02:00
Masaya Suzuki 36346bd81c Add SuppressWarnings for FutureReturnValueIgnored
https://errorprone.info/bugpattern/FutureReturnValueIgnored

The submitted task logs the error in case it fails.

Change-Id: I280442ddb8046da47aeb79d005886f65d2e3067b
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2018-08-21 10:28:37 -07:00
Jonathan Nieder af75250a45 Merge "Support remove-project node in the manifest parser" 2018-08-20 22:14:35 -04:00
Masaya Suzuki 58a90f7a70 Support remove-project node in the manifest parser
This directive removes the projects added by other directives.

https://gerrit.googlesource.com/git-repo/+/65b0ba5aa0447f7ee25103828115662b1eb80ff9/manifest_xml.py#614
https://gerrit.googlesource.com/git-repo/+/e469a0c741832f6584513f4a382d6b93f417b8d2/docs/manifest-format.txt#339

Change-Id: I90d7b9fc711b586dec05f475422d34d67d909986
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2018-08-20 18:27:32 -07:00
Matthias Sohn 0c12df54f1 Remove unnecessary maven dependency of org.eclipse.jgit to httpclient
Change-Id: Icc3c696a46adaf0e7b633913a3f36f0440a81e3a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-20 04:54:51 -04:00
Andrey Loskutov 5a95e7e74b SubmoduleDeinitTest fails on Windows
Bug: 538068
Change-Id: I982e8b12936582e8e119f45d7a3bf4888d3e0e1a
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2018-08-19 20:53:31 +02:00
Matthias Sohn 60c7791918 Merge branch 'stable-5.0'
* stable-5.0:
  Fix GC run in foreground to not use executor

Change-Id: I484391e5589d70de136126d153f886f5afa12e88
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:57:42 +02:00
Matthias Sohn 78d7d9916c Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Fix GC run in foreground to not use executor

Change-Id: I9bb1506805a2d5e79a420a1152bc45321c081229
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:55:24 +02:00
Matthias Sohn c415e93e8a Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Fix GC run in foreground to not use executor

Change-Id: I565b95de6c89f021475667caaacdbb08caddd881
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:51:45 +02:00
Matthias Sohn 9b55f1a278 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Fix GC run in foreground to not use executor

Change-Id: Ib4d76125fca7eec9e88666688b5e614e7e20dde7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:51:00 +02:00
Matthias Sohn ce84d0dfc6 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Fix GC run in foreground to not use executor

Change-Id: Id9d864a8e727fefa35ca87eccb4e3801eb689c3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:47:51 +02:00
Matthias Sohn 96beadbbc1 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Fix GC run in foreground to not use executor

Change-Id: Ib150d132e2ce055d36ddffb2dbc37b5cb355e77a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-18 13:40:33 +02:00
Matthias Sohn 7a536cef06 Merge branch 'stable-5.0'
* stable-5.0:
  Fix wrong warning message in case of lock failures

Change-Id: Ia9b813652a2f5b6c78e9587367c35c3a6c177a7a
2018-08-17 01:24:10 +02:00
Christian Halstrick 982be5db14 Fix wrong warning message in case of lock failures
Bug: 537969
Change-Id: I0854bbbe7393abaca36640d429886c30d95d5bf1
2018-08-16 14:30:46 -05:00
Masaya Suzuki 3aa2b8064c Introduce ProtocolV2Hook
In Git protocol v2, UploadPack and ReceivePack have the same
capabilities and can process any protocol v2 request. For example, a
client can sent a "fetch" command to the "/git-receive-pack" endpoint.

This makes it difficult for existing hook interfaces. For example,
PreUploadHook takes UploadPack, but a "fetch" command may be received by
ReceivePack.

To resolve this skew, this change introduce a different hook interface
for the protocol v2. The hook takes a request that is independent to the
handlers (UploadPack, ReceivePack). Also this makes it clear what
parameters the hook is counting on, instead of keep track of the hook
using getters from UploadPack / ReceivePack.

Bug: 534847
Change-Id: I71f3266584483db1e2b2edfc1a72d0bdf1bb6041
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2018-08-16 11:22:17 -07:00
Hugo Arès d4a19c328f Fix GC run in foreground to not use executor
Since I3870cadb4, GC task was always delegated to an executor even when
background option was set to false. This was an issue because if more
than one GC object was instantiated and executed in parallel, only one GC
was actually running because of the single thread executor.

Change-Id: I8c587d22d63c1601b7d75914692644a385cd86d6
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
2018-08-15 19:53:33 -05:00
Dave Borowitz 62562295c0 Add EMPTY_TREE_ID as a sibling of EMPTY_BLOB_ID
Change-Id: Id68b3e1db7294cf9adb8b4ce8eecb1d828687eef
2018-08-15 15:59:55 -07:00
Dave Borowitz 289f116dbb Update javadoc for EMPTY_BLOB_ID
Found original version with:

  $ git describe --contains $(git blame fc4d4b5340 org.eclipse.jgit/src/org/eclipse/jgit/lib/Constants.java | grep EMPTY_BLOB_ID | cut -d' ' -f1)
  v0.9.1~164

Change-Id: I1f86bde42615c8b64064894bde44f82660b38d0f
2018-08-15 15:59:23 -07:00
Jonathan Nieder fc4d4b5340 Merge "Declare ConfigConstants and GitProtocolConstants final" 2018-08-14 18:55:26 -04:00
Matthias Sohn 31a72f5da7 Declare ConfigConstants and GitProtocolConstants final
This avoids that we have to suppress API errors whenever we add a new
constant in a minor release. This change affects implementors only which
is ok to do in a minor release following OSGi semantic versioning rules.

Change-Id: Iece841886fbe00f1ba567c5ff68093c542ba265e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-15 00:32:41 +02:00
Jonathan Tan 3227348e5e Respond with shallow-info before wanted-refs
When fetching with protocol v2, git expects the shallow-info section to
appear before wanted-refs if both appear in the response. Teach
UploadPack to do this.

Change-Id: Ie26a91edcce5d27a1d727d7fba5c30e1144e118b
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2018-08-13 16:57:25 -07:00
Han-Wen Nienhuys d1dc49e968 Use a TreeMap to construct the list of renamed projects
This eliminates one source of unpredictable ordering of entries in .gitmodules.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I834ae57bd18f51966ef76c039f5212ebf60a33e8
2018-08-13 12:17:58 -04:00
Gunnar Wagenknecht 82e0c4a084 Merge "Fix ResolveMerger: rebase with autocrlf=true, direct checkout" 2018-08-13 07:36:44 -04:00
Thomas Wolf 4027c5c9ff Fix ResolveMerger: rebase with autocrlf=true, direct checkout
ResolveMerger.checkout() and cleanUp() check out files directly and
must honor CR/LF settings and also smudge filters.

Deprecate the 3-argument version of DirCacheCheckout.checkoutEntry().
It isn't used anymore anywhere in JGit (nor in EGit).

Bug: 537410
Change-Id: I062b35401c8bd5bc99deb2f68f91089a0643504c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-13 13:15:24 +02:00
Jonathan Tan c477b0ddcb Support protocol v2 want-ref in UploadPack
UploadPack already allows the client to send wanted OIDs as "want"
lines. Extend UploadPack to also allow the client to send wanted ref
names as "want-ref" lines when the fetch is done using protocol v2.

The corresponding Git commit is 516e2b76bd ("upload-pack: implement
ref-in-want", 2018-06-28).

To support a two-stage rollout, two configuration variables are
provided: uploadpack.allowrefinwant (default "false") allows clients to
specify "want-ref" in their requests, and uploadpack.advertiserefinwant
(default "true") makes UploadPack advertise this capability. If
uploadpack.allowrefinwant is true but uploadpack.advertiserefinwant is
false, UploadPack will not advertise that it supports "want-ref", but it
will support it.

Change-Id: I3c24077949640d453af90d81a7f48ce4b8ac9833
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2018-08-10 18:44:49 -04:00
Thomas Wolf bd1a825026 Merge remote-tracking branch 'origin/stable-5.0'
* stable-5.0:
  Ensure that JSch knows HostKeyAlgorithms ssh-rsa and ssh-dss

Change-Id: I76899a1f6b3d9348d138266e05e1320b7429bfd1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-09 12:06:49 +02:00
Thomas Wolf 09753591fa Ensure that JSch knows HostKeyAlgorithms ssh-rsa and ssh-dss
Without these registrations, JSch's up-front checks which algorithms
are available at all fail if the ssh config explicitly sets only these
algorithms.

Bug: 537790
Change-Id: Idb0431190a7f101913363ee95af6c8fcbda6c923
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-08 13:36:35 +02:00
Matthias Sohn cf6463bddc Fix API breakage introduced by da254106
Use org.eclipse.jgit.errors.CancelledException which is a subclass of
IOException instead of org.eclipse.jgit.api.errors.CanceledException in
order to avoid breaking API. We can reconsider this with the next major
version 6.0.

Bug: 536324
Change-Id: Ia6f84f59aa6b7d78b8fccaba24ade320a54f7458
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-08-08 11:44:55 +02:00
Matthias Sohn da254106a7 Abort rename detection in a timely manner if cancelled
If progress monitor is cancelled break loops in rename detection by
throwing a CanceledException.

Bug: 536324
Change-Id: Ia3511fb749d2a5d45005e72c156b874ab7a0da26
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-07 00:09:43 +02:00
Jonathan Nieder 26c5d0e56a RebaseCommand: Remove unnecessary fall through comments
It's easier to read a list of cases

	case A:
	case B:
	case C:
		... do something ...;
		break;

without intervening "// fall through" lines separating the cases.

This also makes it clearer that JGit uses "$FALL-THROUGH$", not "fall
through", to suppress fallthrough warnings in switch statements.
Eclipse does not pay attention to the second comment style.

Change-Id: I5279a727aee2868854d06bfcaac8cb8186b4299e
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-08-04 14:04:29 -07:00
Matthias Sohn 2f7bf6b536 Prepare 5.0.3-SNAPSHOT builds
Change-Id: Iae9d763a65481b09af6e19b33e41375f27038357
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-01 01:30:05 +02:00
Matthias Sohn 40813388e7 JGit v5.0.2.201807311906-r
Change-Id: Iccb98c03a069fee85a52325e1ed22cba8b76bcd8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-08-01 01:06:30 +02:00
Matthias Sohn a8c7c42ff1 Prepare 4.11.2-SNAPSHOT builds
Change-Id: I02630900ab131478b30abe68b2133d2c8a5a67ab
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-31 22:07:17 +02:00
Matthias Sohn e51d2fcda8 JGit v4.11.1.201807311124-r
Change-Id: Ic58a916d7e2c380b84690afa5e3517c6c178af8c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-31 17:31:30 +02:00
Matthias Sohn 6a813c64ff Prepare 4.9.4-SNAPSHOT builds
Change-Id: Ibb2ee9d00d5d049f681e1abbb449656105e553a7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-31 16:57:13 +02:00
Matthias Sohn 56b435a61c JGit v4.9.3.201807311005-r
Change-Id: I3f4f437649736c6a27e78d536344c7b31c8a0f65
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-31 16:05:49 +02:00
Jonathan Nieder 8ef7cd4df4 Merge "Use project names instead of paths for the submodule name" 2018-07-27 18:15:14 -04:00
Masaya Suzuki 6658f36768 Use project names instead of paths for the submodule name
Two submodules at the same path on different branches need not represent
the same repository, and two submodules at different paths can represent
the same one.

The C Git implementation uses the submodule name to internally manage
the submodule repositories under .git/modules. When a submodule
represents different repositories in different branches, it makes a
conflict inside .git/modules.

The current RepoCommand implementation uses submodule paths as the
submodule names. When the manifest file mounts different repositories to
the same path in different branches, this makes a situation described
above. To solve this issue, we can use the project name instead of
the path as the submodule name.

On the other hand, since repo v1.12.8~3^2 (repo: Support multiple
branches for the same project., 2013-10-11), a manifest file can mount
the same project to different paths. If we naively use the project
name as the submodule name, it makes a conflict in .git/modules, too.

This patch uses the project name as the submodule name basically, but
when the same project is mounted to different paths, it uses the project
name and path as the submodule name.

Change-Id: I09dc7d62ba59016fe28852d3139a56ef7ef49b8f
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Reported-by: JP Sugarbroad <jpsugar@google.com>
2018-07-27 15:07:30 -07:00
Matthias Sohn 435975e7a9 Merge branch 'stable-5.0'
* stable-5.0:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: I9717444c42ea89195dfcde8bb10121b9bb178e87
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 14:16:49 +02:00
Matthias Sohn 0db128acd5 Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: I780d4a1624fbb97a2acdf5321b9c3244b0626444
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 14:06:22 +02:00
Matthias Sohn fd4a62fdf0 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: I2b44d862869d4453c57db668fc7c925da591f671
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 13:38:21 +02:00
Matthias Sohn 8b97c266c3 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: Ie7029bc91621af32e7bfd2e0d76a424b991b1995
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 11:03:24 +02:00
Matthias Sohn a0b818ce01 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: I6ab2b951dd94a9fc1c4f5283847a3e2ec37d0895
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 10:52:13 +02:00
David Pursehouse e90a013e45 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.3-SNAPSHOT builds
  JGit v4.7.2.201807261330-r
  Delete all loose refs empty directories
  Use java.nio to delete path to get detailed errors
  GC: Remove empty references folders
  Do not ignore path deletion errors

Change-Id: Iadc8275fbaa3d6f7d08a96ab66d49f392f6aab78
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-27 08:22:51 +01:00
Matthias Sohn 29118ed117 Prepare 4.7.3-SNAPSHOT builds
Change-Id: I5c437f45d5bc469e3c32bef1180c127d96d24d23
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-27 01:00:35 +02:00
Matthias Sohn 01787756aa JGit v4.7.2.201807261330-r
Change-Id: I0d8c7ca756e6236e315c91da000fe8103ce83d05
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-26 19:28:38 +02:00
Luca Milanesio 977726e5bb Delete all loose refs empty directories
Remove completely the empty directories under refs/<namespace>
including the first level partition of the changes, when they are
completely empty.

Bug: 536777
Change-Id: I88304d34cc42435919c2d1480258684d993dfdca
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-26 01:13:56 +02:00
Luca Milanesio eea9a7a0ba Use java.nio to delete path to get detailed errors
Get the full IOException of the reason why a directory
cannot be removed during GC.

Change-Id: Ia555bce009fa48087a73d677f1ce3b9c0b685b57
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-26 01:02:42 +02:00
Jonathan Nieder 0c75097830 Merge "Send only 1 flush pkt at the end of v2 fetch" 2018-07-24 18:56:42 -04:00
Jonathan Tan 7e7b00f1a5 Send only 1 flush pkt at the end of v2 fetch
When processing a fetch using protocol v2, UploadPack#fetchV2 sends an
extraneous flush pkt when also sending a packfile (#sendPack sending its
own flush pkt). Update that method to only send the flush pkt if the
packfile is not being sent.

Change-Id: I7117a264bccd2d7f3a048645fcb8425a9d78d526
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2018-07-24 15:39:32 -07:00
Matthias Sohn e0e52cb011 Fix NPE in BlameGenerator.getSourceStart()
Bug: 499543
Change-Id: I99f6ebb1c3ceea20e8ca093acbe824c9f0362d45
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-07-24 00:41:34 +02:00
Jonathan Tan 32798dcfdb Always send refs' objects despite "filter" in pack
In a0c9016abd ("upload-pack: send refs' objects despite "filter"",
2018-07-09), Git updated the "filter" option in the fetch-pack
upload-pack protocol to not filter objects explicitly specified in
"want" lines, even if they match the criterion of the filter. Update
JGit to match that behavior.

Change-Id: Ia4d74326edb89e61062e397e05483298c50f9232
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2018-07-20 17:09:05 -07:00
Jonathan Nieder fb9031c956 RepoCommand: record remote branch, group, shallow bit by default
Propagating more information from the manifest to the superproject
tends to work out well.  These particular settings mean:

Recording the remote branch in .gitmodules allows
- "git submodule update --remote" to fetch the right branch
- Gerrit's submodule subscription feature to subscribe from the
  right branch, if requested

Recording groups in .gitattributes allows commands like "git clone
--recurse-submodules=:(attr:platform-linux) $superproject" to clone
the correct set of projects.

Recording the shallow bit in .gitmodules means that "git clone
--recurse-submodules" will use shallow clone in submodules where that
was requested in the manifest.

Gerrit's supermanifest plugin records the remote branch and shallow
bit by default already, and most users of that plugin configure it to
record groups in .gitattributes as well, so for most users this will
be a no-op.

Change-Id: Id2ed47cbca5ce822bde517494673c86ab8c58da9
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-07-20 00:48:22 -07:00
Thomas Wolf a9b54b026d ResolveMerger: don't try needlessly to delete directories
Don't try to delete folders if the merger chooses THEIRS, but all of
BASE, OURS, and THEIRS contain the folder.

Add a test for rebase with auto-stash and subdirectories that
verifies this case. The needless directory deletion and reporting
such directories in getModifiedFiles() was the root cause of bug
536880.

Note even with this fix, bug 536880 will not be fixed in all cases
yet. There may still be cases where the set of modified files ends
up containing directories. This will be dealt with in EGit where
this set is used. (See https://git.eclipse.org/r/#/c/126242/ .)

Bug: 536880
Change-Id: I62b4571a1c1d4415934a6cb4270e0c8036deb2e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-07-19 12:38:28 +02:00
Terry Parker e4774f45c4 GC: Trim more EWAHCompressedBitmaps to free unused memory
04b9f4436 fixed places where compressed bitmaps were holding on to their
full buffers, but missed this StoredBitmap.getBitmap() case where a
bitmap is resonstituted from an xor chain.

Change-Id: I7cf75d9e49c18a1a8a880a4df7e821502edc68a4
Signed-off-by: Terry Parker <tparker@google.com>
2018-07-16 14:44:43 -07:00
Masaya Suzuki 579bff6653 Add API to specify the submodule name
Currently SubmoduleAddCommand always uses the path as submodule name.
This patch lets the caller specify a submodule name.

SubmoduleUpdateCommand still does not make use of the submodule name
(see bug 535027) but Git does.  To avoid triggering CVE-2018-11235,
do some validation on the name to avoid '..' path components.

[jn: fleshed out commit message, mostly to work around flaky CI]

Change-Id: I6879c043c6d7973556e2080387f23c246e3d76a5
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-07-12 12:27:28 -07:00
Minh Thai 263a8c1c06 Make Reftable seek* and has* method names more consistent
Make the method names more consistent and their semantics simpler:
hasRef and seekRef to look up a single exact reference by name and
hasRefsByPrefix and seekRefsByPrefix to look up multiple references by
name prefix.

In particular, splitting hasRef into two separate methods for its
different uses makes DfsReftableDatabase.isNameConflicting easier to
follow.

[jn: fleshed out commit message]

Change-Id: I71106068ff3ec4f7e14dd9eb6ee6b5fab8d14d0b
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-07-11 14:08:35 -07:00
Minh Thai e04d96e3fa Seek references by prefix in reftable
Reftable implementation of RefDatabase.getRefsByPrefix() should be
more performant, as references are filtered directly by prefix;
instead of fetching the whole subtree then filter by prefix.

Change-Id: If4f5f8c08285ea1eaec9efb83c3d864cea7a1321
Signed-off-by: Minh Thai <mthai@google.com>
2018-07-10 19:40:29 -04:00
Terry Parker 04b9f44367 GC: Trim EWAHCompressedBitmaps to free unused memory
The "Building bitmaps" GC phase fails for large repositories (repos with
10M objects use 1.25MB per uncompressed bitmap, and those with long
histories may build >25k bitmaps). Since these bitmaps xor well against
each other, the actual space needed for each compressed bitmap is
usually no more than a few KB. Calling trim() will ensure we aren't
holding on to excess memory.

Change-Id: I40bf78c730b9f6051da6025f9777ce27220a5b0a
Signed-off-by: Terry Parker <tparker@google.com>
2018-07-10 14:10:11 -04:00
Terry Parker dd7d479bd4 Merge "Add progress monitor step to first phase of GC commit selection" 2018-07-10 11:23:38 -04:00
Dave Borowitz 3447e2a776 Make DfsConfig public
This may be convenient for downstream implementers who require a dummy
StoredConfig implementation, rather than making them reimplement the two
abstract StoredConfig methods.

Change-Id: I2b7bc6250d722c2b95d9f99e4eff1e5bf97cb567
2018-07-10 20:28:39 +09:00
Hector Caballero 4eea4ea508 GC: Remove empty references folders
After packaging references, the folders containing these references are
not deleted. In a busy repository, this causes operations to slow down
as traversing the references tree becomes longer.

Delete empty reference folders after the loose references have been
packed.
To avoid deleting a folder that was just created by another concurrent
operation, only delete folders that were not modified in the last 30
seconds.

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ie79447d6121271cf5e25171be377ea396c7028e0
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-10 20:24:59 +09:00
Terry Parker 7fe15d4428 Add progress monitor step to first phase of GC commit selection
Change-Id: I10e3d53de5e2b1fc1be9a59b236da5780473d841
Signed-off-by: Terry Parker <tparker@google.com>
2018-07-09 17:51:39 -07:00
Luca Milanesio 5a8ad44208 Do not ignore path deletion errors
Log as warning when an attempt to remove a directory
fails. This helps troubleshooting some bugs like the GC leaving
behind empty directories.

Change-Id: Idb94ce17f8be9668a970c7ecae31436bf434073c
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2018-07-08 23:44:17 +01:00
David Pursehouse 67f7ca5bb6 Merge branch 'stable-5.0'
* stable-5.0:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: I8e6d0c92144db3bf57ea05f93852ac015458260b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:16:51 +09:00
David Pursehouse 6ba60a3127 Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: I40b7b4b3fd6d53bbad32f8f550e885c0e698faf1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:13:27 +09:00
David Pursehouse 55b1335084 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: I2f02298d0ff7caafeca4020cde4fdfa29a46e585
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:11:11 +09:00
David Pursehouse a96d640365 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: Ibd8f2a041b0de6e008a1ea84b92823f8cbc6e3d2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:10:29 +09:00
David Pursehouse b6a6ba10c7 Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: Id6a85804695d5dcb32f26ed1d861b7c93577c5e4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:09:53 +09:00
David Pursehouse c296342333 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  ResolveMerger: Fix encoding with string; use bytes

Change-Id: If17328fbd101d596a8a16d9c4a190e9b6e120902
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-07-04 14:09:01 +09:00
Thomas Wolf 55ebb83c98 Add response message, if any, on HTTP status 404
Try to give as much information as possible. The connection's
response message might contain additional hints as to why the
connection could not be established.

Bug: 536541
Change-Id: I7230e4e0be9417be8cedeb8aaab35186fcbf00a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-07-03 13:54:32 +02:00
Terry Parker 115a740e2f Correctly handle initialization of shallow commits
In a new RevWalk, if the first object parsed is one of the
shallow commits, the following happens:
1) RevCommit.parseCanonical() is called on a new "r1" RevCommit.
2) RevCommit.parseCanonical() immediately calls
   RevWalk.initializeShallowCommits().
3) RevWalk.initializeShallowCommits() calls lookupCommit(id),
   creating and adding a new "r2" version of this same object and
   marking its parents empty.
4) RevCommit.parseCanonical() initializes the "r1" RevCommit's
   fields, including the parents.
5) RevCommit.parseCanonical()'s caller uses the "r1" commit that
   has parents, losing the fact that it is a shallow commit.

This change passes the current RevCommit as an argument to
RevWalk.initializeShallowCommits() so that method can set its
parents empty rather than creating the duplicate "r2" commit.

Change-Id: I67b79aa2927dd71ac7b0d8f8917f423dcaf08c8a
Signed-off-by: Terry Parker <tparker@google.com>
2018-06-26 20:13:47 -07:00
Terry Parker e8e4fe7af6 Merge changes Ib6019b10,I82c71b52
* changes:
  Fix a GC scalability issue when selecting commit bitmaps
  Test uniform bitmap commit selection across multiple branches
2018-06-26 12:29:11 -04:00
tparker 2070d146cb Fix a GC scalability issue when selecting commit bitmaps
The previous algorithm selected commits by creating bitmaps at
each branch tip, doing a revwalk to populate each bitmap, and
looping in this way:
1) Select the remaining branch with the most commits (the branch
   whose bitmap has the highest cardinality)
2) Select well-spaced bitmaps in that branch
3) Remove commits in the selected branch from the remaining
   branch-tip bitmaps
4) Repeat at #1

This algorithm gave good commit selection on all branches but
a more uniform selection on "important" branches, where branch
length is the proxy for "important". However the algorithm
required N bitmaps of size M solely for the purpose of commit
selection, where N is the number of branch tips in the primary
GC pack, and M is the number of objects in the pack.

This new algorithm uses branch modification date as the proxy for
"important" branches, replacing the N*M memory allocation with a
single M-sized bitmap and N revwalks from new branch tips to
shared history (which will be short when there is a lot of shared
history).

GcCommitSelectionTest.testDistributionOnMultipleBranches verifies
that this algorithm still yields good coverage on all branches.

Change-Id: Ib6019b102b67eabb379e6b85623e4b5549590e6e
Signed-off-by: Terry Parker <tparker@google.com>
2018-06-26 09:23:46 -07:00
Thomas Wolf f508a00176 CleanCommand: don't fire WorkingTreeModifiedEvent on dry run
Since no files are actually deleted it makes no sense to fire such an
event.

Change-Id: I66e87afc1791f27fddaa873bafe8bb8b61662535
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-06-26 08:26:38 +02:00
Matthias Sohn ff15df05c3 Prepare 5.0.2-SNAPSHOT builds
Change-Id: I65e83f39a4082278c4cb809dbb08d44a119cebe0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-22 01:42:14 +02:00
Matthias Sohn a7e38fb0b0 JGit v5.0.1.201806211838-r
Change-Id: I594cd85d46698953c27da3687e4ec40aeb73b445
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-22 00:40:01 +02:00
Marco Miller 1dcb0688c7 ResolveMerger: Fix encoding with string; use bytes
This change fixes the issue [1]. Before this fix, a merge involving
the caching of consecutive yet similar filenames with Norwegian
characters [2] used to throw an IllegalStateException: Duplicate
stages not allowed. This was caused by inaccurate decoding of the
filenames, using string values assuming default encoding. In the
toString method of DirCacheEntry, used before through getPathString,
UTF-8 encoding is used, but the end result becomes default encoding,
through Object's default toString usage. The special characters in
those two consecutive (particular) filenames [2] were becoming the
very same decoded /single character, lending consecutive -but then
identical- filenames. Thus the perceived duplicate 0-staging of the
file(s).

Replace getPathString usage with getRawPath for this specific case,
or use byte array representations of cached entries instead of string.

Adding a test for this change is not possible, as there is no known
way to change the default encoding for filenames such as [2] (e.g.).
JGitTestUtil does write file contents through UTF-8, but encoding like
so does not apply to the actual file name. Hence there is no way to
create files with names properly made of special characters such as
[2]'s. And the test that is necessary for this case assumes such
Norwegian (or similar characters) filenames. Changing the default
locale programmatically in a test has no effect either. And changing
the LANG value passed to the JVM is only possible upon starting it.

[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=9153

[2] <=>
(...)
"a/b/SíÒr-Norge.map",
"a/b/Sør-Norge.map",
(...)

Change-Id: Ib9f2f5297932337c9817064cc09d9f774dd168f4
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
2018-06-21 15:38:56 -04:00
Jonathan Nieder 82ae431e45 Merge branch 'stable-5.0'
* stable-5.0:
  Respect "unshallow" lines in protocol v2
  Temporarily @Ignore flaky CommitCommandTest methods

Change-Id: I14bd522d2f6bafb1b41e3c730d6dbef827e3e88a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-06-20 21:24:05 -07:00
Jonathan Nieder 7df024b276 Respect "unshallow" lines in protocol v2
If I run

 git config --global protocol.version 2
 mkdir repo
 cd repo
 git init --bare
 git remote add origin https://go.googlesource.com/proposal
 git fetch --depth=1
 git fetch --unshallow

then I expect to have a full history, just as though I had fetched
without --depth in the first place.  Instead, it reports success
but does not fetch enough objects:

 $ git fsck
 notice: HEAD points to an unborn branch (master)
 Checking object directories: 100% (256/256), done.
 Checking objects: 100% (468/468), done.
 broken link from  commit 2c6bc83f234085c8eadb7ea33405ce6223c44d1b
               to  commit 878975cf2b600675b4c905e5d9591bd24541ae9e
 missing commit 878975cf2b600675b4c905e5d9591bd24541ae9e
 dangling commit 314be00dae78dd526851f5635e6349014e2ad0c2

The false success indicates problems in the client and the server.
Git 2.18-rc2 (the client) ought to have been more defensive, noticing
the incomplete history.  The greater error is in JGit (the server),
which neglects to send the objects requested.

When serving protocol v0 requests, JGit sends the correct objects by
taking unshallowCommits into account when generating the pack to send
to the client.  Do the same in the protocol v2 code path.  I forgot to
do this in v5.0.0.201806050710-rc3~6 (Teach UploadPack shallow fetch
in protocol v2, 2018-03-15).

Reported-by: Russ Cox <rsc@golang.org>
Change-Id: I282b45f47616a641b9e8d6210b4a070d3efdbb9b
Signed-off-by: Jonathan Nieder <jrn@google.com>
2018-06-20 21:20:41 -07:00
Thomas Wolf 2e76daec14 Avoid expensive getAllRefsByPeeledObjectId() in PlotWalk constructor
Instead, do it when we return the first PlotCommit from next().
On a repository with many refs, getAllRefsByPeeledObjectId() can
take a while. Doing a late initialization simplifies the handling
of a PlotWalk.

EGit, for instance, creates and configures an instance, and then
does the real walk in a background job. With late initialization,
the potentially expensive getAllRefsByPeeledObjectId() also occurs
in that background job.

Bug: 485743
Change-Id: I84c020cf8f7afda6f181778786612b8e6ddd7ed8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-06-15 16:11:10 +02:00
Matthias Sohn 331f1a8e49 Merge branch 'stable-5.0'
* stable-5.0:
  Prepare 5.0.1-SNAPSHOT builds
  JGit v5.0.0.201806131550-r
  JGit v5.0.0.201806131210-r
  Downgrade Apache httpclient to 4.5.2.v20170210-0925
  RefUpdateTest: Refactor to not use deprecated Repository#getAllRefs
  Propagate failure of ssh command to caller of SshSupport
  Make JGit describe behaves same as c-git for lightweight tags
  Fix issues with LFS on GitHub (SSH)

Change-Id: I0471440919adfdbfc72996711d9e0bbd1f3cf477
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-14 00:00:30 +02:00
Matthias Sohn 0dbaf9d951 Prepare 5.0.1-SNAPSHOT builds
Change-Id: Iadf12dbdd63ef17c3f712c0fc18495304aa35016
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-13 23:24:42 +02:00
Matthias Sohn e729a83bd2 JGit v5.0.0.201806131550-r
Change-Id: Ia2d548676217250593c3f0c383aec19e12e5e4a4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-13 21:42:40 +02:00
Matthias Sohn 94a585e6e8 JGit v5.0.0.201806131210-r
Change-Id: Iedccfcaf57c11822f1faf5d5195357de42b2a14e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-13 18:11:23 +02:00
Matthias Sohn f7fbc7fcd7 Propagate failure of ssh command to caller of SshSupport
When SshSupport.runSshCommand fails since the executed external ssh
command failed throw a CommandFailedException.

If discovery of LFS server fails due to failure of the
git-lfs-authenticate command chain the CommandFailureException to the
LfsConfigInvalidException in order to allow root cause analysis in the
application using that.

Change-Id: I2f9ea2be11274549f6d845937164c248b3d840b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-13 11:24:24 +02:00
Marcel Trautwein 5429d1a0cf Make JGit describe behaves same as c-git for lightweight tags
JGit now considers lightweight tags only if the --tags option is set
i.e. `git.describe().setAllTags(true)` has to be set, else the default
is now as in c git:

Only annotated tags are evaluated unless you pass true
equivalent to --tags (or --all) by the option setAllTags.

Hint: This (still) doesn't address any difference between c-git
`--all` and `!--all --tags` behavior;
perhaps this might be a follow up request

Bug: 423206
Change-Id: I9a3699756df0b9c6a7c74a7e8887dea0df17c8e7
Signed-off-by: Marcel Trautwein <me+eclipse@childno.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-13 09:37:47 +02:00
Markus Duft 01c52a58f6 Fix issues with LFS on GitHub (SSH)
* URIish seems to have a tiny feature (bug?). The path of the URI
   starts with a '/' only if the URI has a port set (it seems).
 * GitHub does not return SSH authorization on a single line as Gerrit
   does - need to account for that.
 * Increase the SSH git-lfs-authenticate timeout, as GitHub sometimes
   responds slower than expected.
 * Guard against NPE in case the download action does not contain any
   additional headers.

Change-Id: Icd1ead3d015479fd4b8bbd42ed42129b0abfb95c
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
2018-06-12 09:49:15 +02:00
Matthias Sohn e512d919ec Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Ensure DirectoryStream is closed promptly

Change-Id: I62674a1db9266c04fb353ab697e2c0a24a7369b7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-11 10:54:59 +02:00
David Pursehouse 0bcf88bc0d Merge branch 'stable-5.0'
* stable-5.0:
  Fix Javadoc typo

Change-Id: Ib864f5871f8e99d2a776b7ae9f782ec3eb312d50
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-06-11 08:09:09 +09:00
Michael Keppler e710f14ed1 Fix Javadoc typo
Change-Id: Ib4ebc57236bdea663f27295764886413e2550580
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-06-10 23:19:20 +09:00
Matthias Sohn 0f8f6746ed Merge branch 'stable-5.0'
* stable-5.0:
  Ensure Jsch checks all configured algorithms
  RawTextTest#testBinary: use array comparison to compare arrays
  LFS: Better SSH authentication token timeout handling
  Ensure DirectoryStream is closed promptly
  Validate branch names on branch creation

Change-Id: Ic4f6a24b6ccee6730eee3fd5dcb0d1f3e291c478
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-06-10 12:12:16 +02:00