Commit Graph

8335 Commits

Author SHA1 Message Date
Matthias Sohn c9b0606782 Merge "pgm: add missing dependency to org.apache.commons.logging" 2021-01-18 02:44:39 -05:00
Matthias Sohn 5e2c976233 Merge "[spotbugs]: Fix potential NPE in FileSnapshot constructor" 2021-01-18 02:44:13 -05:00
Matthias Sohn d4fca2e81c pgm: add missing dependency to org.apache.commons.logging
Without this dependency I get class loading exceptions when trying to
run org.eclipse.jgit.pgm.Clone in Eclipse.

Change-Id: Ia9ecb385d3baccbcd041114287af5076fefd3d71
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-17 18:04:38 -05:00
Thomas Wolf 3ef9f2c764 [pgm] Use Apache sshd by default
Change-Id: I2a00059415fc2674469bc921827362801583af83
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-15 18:42:45 -05:00
Thomas Wolf 58f4e6e7f8 TransportHttp: support preemptive Basic authentication
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.

Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.

Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.

No preemptive authentication will be done if the connection is
redirected to a different host.

Add tests.

Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 471ad49546 TransportHttp: shared SSLContext during fetch or push
TransportHttp makes several HTTP requests. The SSLContext and socket
factory must be shared over these requests, otherwise authentication
information may not be propagated correctly from one request to the
next. This is important for authentication mechanisms that rely on
client-side state, like NEGOTIATE (either NTLM, if the underlying HTTP
library supports it, or Kerberos). In particular, SPNEGO cannot
authenticate on a POST request; the authentication must come from the
initial GET request, which implies that the POST request must use the
same SSLContext and socket factory that was used for the GET.

Change the way HTTPS connections are configured. Introduce the concept
of a GitSession, which is a client-side HTTP session over several HTTPS
requests. TransportHttp creates such a session and uses it to configure
all HTTP requests during that session (fetch or push). This gives a way
to abstract away the differences between JDK and Apache HTTP connections
and to configure SSL setup outside.

A GitSession can maintain state and thus give all HTTP requests in a
session the same socket factory.

Introduce an extension interface HttpConnectionFactory2 that adds a
method to obtain a new GitSession. Implement this for both existing
HTTP connection factories. Change TransportHttp to use the new
GitSession to configure HTTP connections.

The old methods for disabling SSL verification still exist to support
possibly external connection and connection factory implementations
that do not make use of the new GitSession yet.

Bug: 535850
Change-Id: Iedf67464e4e353c1883447c13c86b5a838e678f1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 224aaa0be7 TransportHttp: make the connection factory configurable
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.

Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.

Make the global static factory reference volatile.

Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:44 +01:00
Matthias Sohn 312ab4f7f6 Add target platform for eclipse 4.19 staging
Change-Id: I172aa01203edc8cd069c93762c482a3f21950acc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-12 23:26:09 +01:00
Matthias Sohn a3d1bb0445 Update orbit to S20210105214148 and com.google.gson to 2.8.6
Also fix target name of jgit-4.18 target.

Change-Id: Ib5a13281398e7c5589de2e4a34fae0d08346db9c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-12 23:26:09 +01:00
Marco Miller 63f4de721c Update spotbugs-maven-plugin to 4.2.0
Change-Id: I094d92f95ec2ab1cefa8f1504f2a8c1aa6e28d96
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
2021-01-12 16:00:12 -05:00
Matthias Sohn 4120d17b0b Add org.eclipse.jetty.util.ajax to target platform and bazel deps
It is required by org.eclipse.jetty.servlet [1].

[1] https://search.maven.org/artifact/org.eclipse.jetty/jetty-servlet/9.4.35.v20201120/jar

Change-Id: I307ec8bad17f3a703bf25cc16c87ab9b524a84ff
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-12 10:14:42 +01:00
Thomas Wolf dd3846513b Tag message must not include the signature
Signatures on tags are just tacked onto the end of the message.
Getting the message must not return the signature. Compare [1]
and [2] in C git, which both drop a signature at the end of an
object body.

[1] https://github.com/git/git/blob/21bf933/builtin/tag.c#L173
[2] https://github.com/git/git/blob/21bf933/ref-filter.c#L1276

Change-Id: Ic8a1062b8bc77f2d7c138c3fe8a7fd13b1253f38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-10 10:19:40 -05:00
Matthias Sohn db48fcedbc Update jetty to 9.4.35.v20201120
Change-Id: I203778ea0536defffb720bcf7cdcbc6258540e65
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-07 15:51:18 -05:00
Thomas Wolf fb3ae37e26 Protocol V2: don't log spurious ACKs in UploadPack
UploadPack may log ACKs in protocol V2 that it doesn't send (if it
got a "done" from the client), or may log ACKs twice. That makes
packet log analysis difficult.

Add a new constructor to PacketLineOut to omit all logging from an
instance, and use it in UploadPack.

Change-Id: Ic29ef5f9a05cbcf5f4858a4e1b206ef0e6421c65
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-06 12:17:23 +01:00
Thomas Wolf fc9f866a17 Merge "Protocol V2: respect MAX_HAVES only once we got at least one ACK" 2021-01-05 17:59:27 -05:00
Christian Halstrick 5aaaad5cc1 Merge "FileSnapshot: don't try to read file attributes twice" 2021-01-05 02:11:48 -05:00
Thomas Wolf 0d7d98620f Protocol V2: respect MAX_HAVES only once we got at least one ACK
The negotiation in the git protocol contains a cutoff: if the client
has sent more than MAX_HAVES "have" lines without getting an ACK, it
gives up and sends a "done". MAX_HAVES is 256.

However, this cutoff must kick in only if at least one ACK has been
received. Otherwise the client may give up way too early, which makes
the server send all its history. See [1].

This was missed when protocol V2 was implemented for fetching in JGit
in commit 0853a241.

Compare also C git commit 0b07eecf6ed.[2] C git had the same bug.[3][4]

[1] https://github.com/git/git/blob/6c430a647cb9/Documentation/technical/pack-protocol.txt#L385
[2] https://github.com/git/git/commit/0b07eecf6ed
[3] https://lore.kernel.org/git/b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz/
[4] https://lore.kernel.org/git/20200422084254.GA27502@furthur.local/

Bug: 553083
Change-Id: I1f4e2cc16b5eed6971d981d472329185abb9e4a9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-04 10:00:28 +01:00
Matthias Sohn 74d5a1c172 RepositoryCache: declare schedulerLock final
This fixes errorprone error [SynchronizeOnNonFinalField]: Synchronizing
on non-final fields is not safe: if the field is ever updated, different
threads may end up locking on different objects.

Change-Id: I42fe5bde825151693e2da2d5b6cd6e1d34038dbc
2021-01-03 19:59:35 -05:00
David Ostrovsky d9143287b7 Enable git wire protocol version 2 on server side per default
Bug: 563145
Change-Id: Id5030c2b85466da0a8ccf3d78ae78df16d64ffc5
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2021-01-03 16:25:01 +01:00
Matthias Sohn ef04b3b883 [spotbugs]: Fix potential NPE in FileSnapshot constructor
File#getParent can return null which caused this spotbugs warning.

FS.FileStoreAttributes#get already gets the parent directory if the
passed File is not a directory and checks for null. Hence there is no
need to get the parent directory in the FileSnapshot constructor.

Change-Id: I77f71503cffb05970ab8d9ba55b69c96c53098b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-03 16:08:59 +01:00
Matthias Sohn 877ce01d29 FileSnapshot: don't try to read file attributes twice
If file doesn't exist set state to MISSING_FILE immediately. Doing that
by calling File#lastModified and File#length effectively does the same
since they set the value to 0 if the file doesn't exist.

Log an error if a different exception than NoSuchFileException is
caught.

Change-Id: I0d4396b9f80446692a088d17522d64f735ce6708
2021-01-03 15:33:36 +01:00
Thomas Wolf 0853a2410f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.

Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 21:22:30 +01:00
Thomas Wolf 0f442d7083 Use Map interface instead of ConcurrentHashMap class
On Android, the co-variant override of ConcurrentHashMap.keySet()
introduced in Java 8 was undone. [1] If compiled Java code calls that
co-variant override directly, one gets a NoSuchMethodError exception
at run-time on Android.

Making the code call that method via Map.keySet() side-steps this
problem.

This is similar to bug 496262, where the same problem cropped up when
compiling with Java 8 against a Java 7 target, but here we cannot use
bootclasspath. We build against Java 8, not against the Android version
of it.

Recent Android versions should have some bytecode "magic" that adds the
co-variant override in bytecode (see the commit referenced in [1]), but
on older Android version this problem may still occur. (Or perhaps the
"magic" is ineffective...) There are two pull requests on Github for
this problem, both from 2020, [2][3] while the Android commit [1] is
from March 2018. Apparently people still occasionally run into this
problem in the wild.

[1] 0e8b937ded/ojluni/src/main/java/java/util/concurrent/ConcurrentHashMap.java (1244)
[2] https://github.com/eclipse/jgit/pull/104
[3] https://github.com/eclipse/jgit/pull/100

Change-Id: I7c07e0cc59871cb7fe60795e22867827fa9c2458
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 12:49:05 -05:00
Thomas Wolf 5b1a6e0e38 Fix NPE in DirCacheCheckout
If a file exists in head, merge, and the working tree, but not in
the index, and we're doing a force checkout, the checkout must be
an "update", not a "keep".

This is a follow-up on If3a9b9e60064459d187c7db04eb4471a72c6cece.

Bug: 569962
Change-Id: I59a7ac41898ddc1dd90e86b09b621a41fdf45667
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-30 10:51:14 +01:00
Christian Halstrick 086f474054 Merge "Added check for null on DirCacheEntry in checkoutEntry method" 2020-12-30 03:05:00 -05:00
Thomas Wolf 8caaaf956a GPG user ID matching: use case-insensitive matching
Although not mentioned in the GPG documentation at [1], GPG uses
case-insensitive matching also for the '<' (exact e-mail) and '@'
(partial e-mail) operators. Matching for '=' (full exact match) is
case-sensitive. Compare [2].

[1] https://www.gnupg.org/documentation/manuals/gnupg/Specify-a-User-ID.html
[2] https://dev.gnupg.org/source/gnupg/browse/master/g10/keyring.c;22f7dddc34446a8c3e9eddf6cb281f16802351d7$890

Bug: 547789
Change-Id: I2f5ab65807d5dde3aa00ff032894701bbd8418c9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-29 10:15:20 +01:00
Thomas Wolf 616a7eeaec Don't export package from test bundle
Do not export the test-only package org.eclipse.jgit.transport from
bundle org.eclipse.jgit.ssh.jsch.test. Doing so can confuse the build
in Eclipse: other bundles that import this package may then also pick
up this test package, leading to non-test sources depending on test
sources and to build cycles.

Change-Id: I9f73b7a8d13bc4a2fe58bd2f1d33068164a13991
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-28 22:53:50 +01:00
Matthias Sohn 8d3b939cfd Update eclipse-jarsigner-plugin to 1.1.7
Change-Id: I6ac33e662aac68a01455113d8abbe0bcdd725ca2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 16:09:40 +01:00
Matthias Sohn c90fba54ec Update build-helper-maven-plugin to 3.2.0
Change-Id: I41e5645fe6eb9f477ec9e0653a75279d927a64f8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 16:08:20 +01:00
Matthias Sohn 8df3639b09 Update maven-enforcer-plugin to 3.0.0-M3
Change-Id: I5121415523b01994338c4097a8437a677d08b954
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:57:15 +01:00
Matthias Sohn eecaebec3f Update maven-shade-plugin to 3.2.4
Change-Id: Id3eb7c8ab5ce37b6079bbf9a4cb1974ba70ace8a
2020-12-24 15:57:15 +01:00
Matthias Sohn f61b7e5406 Require latest Maven version 3.6.3
Change-Id: Iab08ddcdb7edc3c9ac55343d3d40012e19792ea0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:55:40 +01:00
Matthias Sohn 06ea895967 Update spring-boot-maven-plugin to 2.4.1
Change-Id: I68a6c04a5c7e674410736a45ef0031a2c369a321
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:30:18 +01:00
Matthias Sohn f93b1b5e6a Update maven-surefire plugins to 3.0.0-M5
Change-Id: I0425029eac630f4e8fb47d112d086e74283b2ebd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:30:18 +01:00
Matthias Sohn 41158b2147 Update japicmp-maven-plugin to 0.14.4
Change-Id: I33ba5388153e6941713b9c9425cd917e9769a177
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:02:25 +01:00
Matthias Sohn 00882e03d6 Update spotbugs-maven-plugin to 4.1.4
Change-Id: Ie231dc03051f3c8d799f037eeeed8dcaad258fe0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-24 15:00:06 +01:00
Marco Miller 6a3bae670e Upgrade ecj to 3.24.0
Change-Id: I81c03c5f916e97f91cf06dbe3ce355e430b2981a
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
2020-12-24 14:44:20 +01:00
Marco Miller 1d7829eb82 Upgrade wagon-ssh to 3.4.2; align maven-site-plugin
Upgrade wagon-ssh to 3.4.2 in all pom files, thus align
maven-site-plugin versions to 3.9.1 across as well, consistently.

Change-Id: I38f7a6ded3517d1b116169dee1c12deb86eed3a1
Signed-off-by: Marco Miller <marco.miller@ericsson.com>
2020-12-24 14:44:20 +01:00
Matthias Sohn 3482e50134 [spotbugs] Fix incorrect lazy initialization in SystemReader
This fixes two warnings of type LI_LAZY_INIT_STATIC.

Change-Id: I26a7a48aed9d0a0547e908a56b7014a7620fadd8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:01 +01:00
Matthias Sohn 13e0ffbd3d [spotbugs] Don't use class from java.util.concurrent for locking
Use a dedicated Lock object to lock the scheduler in
RepositoryCache#configureEviction to fix spotbugs warning
JLM_JSR166_UTILCONCURRENT_MONITORENTER.

Change-Id: I003dcf0ed1a0a3f4eea5d8a2f51a07473d28a928
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:01 +01:00
Matthias Sohn 50f0347ea3 [spotbugs] Fix potential NPE in OpenSshServerKeyDatabase
If oldLine is null #updateModifiedServerKey shouldn't be called since it
would derefence it. Spotbugs raised this as problem
RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE. Fix it by checking if
oldLine is null before calling #updateModifiedServerKey.

Change-Id: I8a2000492986e52ce7dbe25f48b321c05fd371e4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:00 +01:00
Matthias Sohn bceb202319 [spotbugs] Fix potential NPE in FS.FileStoreAttributes#get
Path#getParent can return null, return fallback filestore attributes in
that case.

Change-Id: Ic09484d527bc87b27964b625e07373b82412f2da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:00 +01:00
Matthias Sohn d7b94f0194 [spotbugs] Silence CN_IMPLEMENTS_CLONE_BUT_NOT_CLONEABLE in jgit tests
We want complete control over clone behavior and don't want to use
Object's clone implementation.

Change-Id: I34a4e967f8aa3879c69a4978768bd35712c4105c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:00 +01:00
Matthias Sohn 21262e98fe [spotbugs] Silence NP_BOOLEAN_RETURN_NULL in FakeUserAuthGSS#doAuth
Also mark the return value @Nullable to enable null analysis in Eclipse.

Change-Id: Ib954b231d743da6ea122adb2cc4880b5f99824cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:00 +01:00
Matthias Sohn 0132666d5a [spotbugs] Fix FileReftableStack#equals to check for null
This fixes spotbugs warning NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT.

This implementation violated the contract defined by
java.lang.Object.equals() because it did not check for null being passed
as the argument. All equals() methods should return false if passed a
null value.

Change-Id: I607f6979613d390aae2f3546b587f63133d6d73c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn fa0e77e8f9 [spotbugs] FileReftableDatabase: extract lock to local variable
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#compactFully.

Change-Id: I370578ad9a027c5c9709d60a1dfafdac0cfca908
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 39cbc574d1 [spotbugs] DfsReftableDatabase: extract lock to local variable
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#DfsReftableDatabase and #clearCache.

Change-Id: Ifd3189288d2a8e64139c02cd105eb335fa2f68cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 1fd0a49ce0 [spotbugs] Silence NP_BOOLEAN_RETURN_NULL in IgnoreNode#checkIgnored
Also mark the return value @Nullable to enable null analysis in Eclipse.

Change-Id: I5b286d657d432f4b32afd4dd370f76892b115422
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 5eabd6995a [spotbugs] Silence warning DLS_DEAD_LOCAL_STORE in generated classes
Change-Id: I714f5fa48a2443d1a8e2c640b204d3e32438910e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:58 +01:00
Matthias Sohn d6fb42ebd6 [spotbugs] Silence warning about unwritten field written by gson
It seems spotbugs does not recognize that this data is written by gson.

Change-Id: I0704d489971c0ab6ec398083f211c409061e7661
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:58 +01:00