Commit Graph

83 Commits

Author SHA1 Message Date
Matthias Sohn 1d280db147 Enable compiler option --release
This ensures the compiler compiles against the public, supported and
documented API for a specific VM version (here 11) [1]. This also means
that
we don't need EE descriptors in Eclipse anymore in order to ensure that
only supported APIs of the selected Java version can be used.

According to [2] if option --release is used --source and --target
options can't be used.

While we are at it also add default value for all new jdt core options
added in Eclipse 4.21.

[1] https://docs.oracle.com/en/java/javase/11/tools/javac.html
[2] https://docs.oracle.com/en/java/javase/14/docs/specs/man/javac.html#option-release

Change-Id: I852a5d7b0a3210751c15d79ec91915b4c01c41e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-09-29 17:13:01 +02:00
Matthias Sohn f81a3fc91a Bump minimum required Java version to 11
Bug: 569917
Change-Id: Ifdcdb022a3f29321b4d10da1cc34acca68ed7b03
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-09-29 17:12:12 +02:00
Matthias Sohn a19494b735 Merge branch 'master' into next
* master: (38 commits)
  Revert "DFS block cache: Refactor to enable parallel index loading"
  GitServlet: allow to override default error handlers
  Silence API error for new interface method ProtocolV2Hook#onObjectInfo
  transport: add object-info capability
  Ignore IllegalStateException if JVM is already shutting down
  Update orbit to R20210825222808 for 2021-09
  Update spotbugs-maven-plugin to 4.3.0
  Update ant to 1.10.11 also in pom.xml
  DFS block cache: add additional stats to DfsReaderIoStats
  Update Orbit to S20210817231813
  [gpg] Better GPG home directory determination
  FS: cleanup use of final modifier
  Ensure FS#searchPath only selects executable files
  RevWalk: getMergedInto's result is wrong on the second call
  DFS block cache: Refactor to enable parallel index loading
  [test] Create keystore with the keytool of the running JDK
  [gpg] Update to Bouncy Castle 1.69
  [test] Create keystore with the keytool of the running JDK
  [sshd] Minor code clean-up
  Support commit.template config property
  ...

Change-Id: I9f99e9a513a23c0c0d252334e79c351512d7355e
2021-09-13 23:53:34 +02:00
Matthias Sohn ca969ecc61 Relax version range for hamcrest and assertj
We currently cannot use hamcrest 2.2 and assertj 3.20.2 (which requires
hamcrest 2.2) in egit tests since other Eclipse dependencies (e.g.
swtbot) require hamcrest 1.1.

Hence relax version range for these components in jgit so that jgit
tests also work when using the egit target platform.

Change-Id: I521e2ec4491bd8d790609b8a66a8f14511a865a1
2021-07-19 12:32:26 +02:00
Matthias Sohn e0ba98edd2 Update orbit to I20210711110031
and update
- assertj to 3.20.2.v20210706-1104
- hamcrest to 2.2.0.v20210711-0821
  - classes which were in org.hamcrest.core 1.3 and org.hamcrest.library
    1.3 were all moved to org.hamcrest in 2.2
  - the annotation org.hamcrest.Factory was removed and is no longer
    needed
  - junit 4.13 requires hamcrest-core and hamcrest-library 1.3 therefore
    keep them in the target platform

CQ: 23501
Change-Id: Ife871c0343b611be9203aed7f86577e85bbf5c95
2021-07-16 23:47:29 +02:00
Thomas Wolf 27a1fa1872 [sshd] Implement SSH config KexAlgorithms
Make the used KEX algorithms configurable via the ssh config. Also
implement adding algorithms not in the default set: since sshd 2.6.0
deprecated SHA1-based algorithms, it is possible that the default set
has not all available algorithms, so adding algorithms makes sense.

This enables users who have to use a git server that only supports
old SHA1-based key exchange methods to enable those methods in the
ssh config:

  KexAlgorithms +diffie-hellman-group1-sha1

There are two more SHA1 algorithms that are not enabled by default:
diffie-hellman-group14-sha1 and diffie-hellman-group-exchange-sha1.
KeyAlgorithms accepts a comma-separated list of algorithm names.

Since adding algorithms is now supported, adapt the handling of
signature algorithms, too. Make sure that definitions for the KEX
exchange signature (HostKeyAlgorithms) don't conflict with the
definition for signatures for pubkey auth (PubkeyAcceptedAlgorithms).

HostKeyAlgorithms updates the signature factories set on the session
to include the default factories plus any that might have been added
via the SSH config. Move the handling of PubkeyAcceptedAlgorithms
from the client to the JGitPubkeyAuthentication, where it can be done
only if pubkey auth is attempted at all and where it can store its
adapted list of factories locally.

Bug: 574636
Change-Id: Ia5d5f174bbc8e5b41e10ec2c25216d861174e7c3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-07-16 08:45:23 +02:00
Thomas Wolf 66c9c7bf87 Update target platform to I20210626190330
Update
- com.google.gson to 2.8.7.v20210624-1215
- javaewah to 1.1.12.v20210622-2206
- org.apache.sshd.osgi to 2.7.0.v20210623-0618
- org.apache.sshd.sftp to 2.7.0.v20210623-0618
- org.tukaani.xz to 1.9.0.v20210624-1259
- Apache MINA sshd to 2.7.0
  - Remove work-arounds for problems resolved upstream since 2.6.0, and
    adapt to upstream API changes.
- update DEPENDENCIES.

CQ: 23469
CQ: 23470
CQ: 23496
CQ: 23497
CQ: 23498
Bug: 574220
Change-Id: I898b216c3492f8488fbf25fa4b49f1250f86f3c8
Also-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-07-06 13:22:38 +02:00
Matthias Sohn 1cbfea9ece Prepare 5.13.0-SNAPSHOT builds
Change-Id: Ie9cfc1eeb0eda7b2bbe744a22a7e4cfe6d59bc37
2021-06-14 01:49:05 +02:00
Matthias Sohn a397727e4a Prepare 5.12.1-SNAPSHOT builds
Change-Id: Idf266c34aa9a04cf9c5e0e09bcb415c13d773d4c
2021-06-13 23:59:14 +02:00
Matthias Sohn 1f062c64be JGit v5.12.0.202106070339-r
Change-Id: I0fbfea2c83f1ce83f75130cc97591547032f1104
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-06-07 09:38:29 +02:00
Matthias Sohn 044c5f215c Prepare 5.12.0-SNAPSHOT builds
Change-Id: Ifc72d3f3ac84b9c4055b95ec0093d877ffb09ab0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-06-03 20:04:28 +02:00
Matthias Sohn 45a4c131ae JGit v5.12.0.202106021050-rc1
Change-Id: I622ee049f14f37504ff4a062f03d6fc25465d0ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-06-02 16:49:17 +02:00
Matthias Sohn 1f733663bf Prepare 5.12.0-SNAPSHOT builds
Change-Id: I25e4efc9b40ae4e7168b37385445c73992c5beb0
2021-06-02 08:47:28 +02:00
Matthias Sohn 94aa245023 JGit v5.12.0.202106011439-rc1
Change-Id: Ieac1d02879defe0f4791062448d4efc328a2f652
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-06-01 20:38:31 +02:00
Matthias Sohn 677d7794cc Prepare 5.12.0-SNAPSHOT builds
Change-Id: If563be77aab768ac1f31ae2211fb0892d0205a2a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-06-01 16:13:41 +02:00
Matthias Sohn ff4c10bf90 JGit v5.12.0.202105261145-m3
Change-Id: I3b1af2032227900e6e0c6189f47bace1df67f0ab
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-05-26 17:43:55 +02:00
Matthias Sohn 1aa3cf7f41 Merge branch 'master' into next
* master: (34 commits)
  Remove texts which were added by mistake in 00386272
  Fix formatting which was broken in 00386272
  LockFile: create OutputStream only when needed
  Add a cgit interoperability test for LockFile
  Add TemporaryBuffer.toString(int limit)
  LockFile: create OutputStream only when needed
  Prepare 5.12.0-SNAPSHOT builds
  JGit v5.12.0.202105051250-m2
  Update jetty to 9.4.40.v20210413
  [releng] Update eclipse-jarsigner-plugin to 1.3.1
  Implement ours/theirs content conflict resolution
  ssh: ensure list is modifiable before using Iterator.remove().
  Update orbit to S20210406213021 and add 4.20-staging target
  Fix typo in test method name
  Allow file mode conflicts in virtual base commit on recursive merge.
  sshd: don't lock the known_hosts files on reading
  Allow info messages in UsernamePasswordCredentialsProvider
  ssh config: do environment variable replacement
  sshd: implement server-sig-algs SSH extension (client side)
  Upgrade ecj to 3.25.0
  ...

Change-Id: Ibc39a9c4e431d15b67ab4a307241f47a7f3740a9
2021-05-12 08:59:07 +02:00
Matthias Sohn 520b6d7dcf Prepare 5.12.0-SNAPSHOT builds
Change-Id: I2fc5305e7eaaa4593d418fc3b31d20e4b6e1e585
2021-05-06 09:53:25 +02:00
Matthias Sohn 92679d792f JGit v5.12.0.202105051250-m2
Change-Id: Ic7d86c91ec0ff9aa0678dcb971c197e62a4ca2dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-05-05 17:50:19 +02:00
Matthias Sohn beecca02bb Merge branch 'stable-5.11'
* stable-5.11:
  Refactor CommitCommand to improve readability
  CommitCommand: fix formatting
  CommitCommand: remove unncessary comment
  Ensure post-commit hook is called after index lock was released
  sshd: try all configured signature algorithms for a key
  sshd: modernize ssh config file parsing
  sshd: implement ssh config PubkeyAcceptedAlgorithms

Change-Id: Ic3235ffd84c9d7537a1fe5ff4f216578e6e26724
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-26 09:56:19 +01:00
Thomas Wolf fd3edc7bfc sshd: try all configured signature algorithms for a key
For RSA keys, there may be several configured signature algorithms:
rsa-sha2-512, rsa-sha2-256, and ssh-rsa. Upstream sshd has bug
SSHD-1105 [1] and always and unconditionally uses only the first
configured algorithm. With the default order, this means that it cannot
connect to a server that knows only ssh-rsa, like for instance Apache
MINA sshd servers older than 2.6.0.

This affects for instance bitbucket.org or also AWS Code Commit.

Re-introduce our own pubkey authenticator that fixes this.

Note that a server may impose a penalty (back-off delay) for subsequent
authentication attempts with signature algorithms unknown to the server.
In such cases, users can re-order the signature algorithm list via the
PubkeyAcceptedAlgorithms (formerly PubkeyAcceptedKeyTypes) ssh config.

[1] https://issues.apache.org/jira/browse/SSHD-1105

Bug: 572056
Change-Id: I7fb9c759ab6532e5f3b6524e9084085ddb2f30d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-03-19 17:28:24 +01:00
Thomas Wolf ffc1f9b026 sshd: implement ssh config PubkeyAcceptedAlgorithms
Apache MINA sshd 2.6.0 appears to use only the first appropriate
public key signature algorithm for a particular key. See [1]. For
RSA keys, that is rsa-sha2-512. This breaks authentication at servers
that only know the older (and deprecated) ssh-rsa algorithm.

With PubkeyAcceptedAlgorithms, users can re-order algorithms in
the ssh config file per host, if needed. Setting

  PubkeyAcceptedAlgorithms ^ssh-rsa

will put "ssh-rsa" at the front of the list of algorithms, and then
authentication at such servers with RSA keys works again.

[1] https://issues.apache.org/jira/browse/SSHD-1105

Bug: 572056
Change-Id: I86c3b93f05960c68936e80642965815926bb2532
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-03-19 17:27:03 +01:00
Matthias Sohn 232876421d Prepare 5.12.0-SNAPSHOT builds
Change-Id: I736de7c3deb11da75777d459f47332df0b486443
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-10 16:34:28 +01:00
Matthias Sohn 1f368f8867 Prepare 5.11.1-SNAPSHOT builds
Change-Id: I94628ccbb5099a65aa4345cfd28a141ff5555b68
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-09 23:42:31 +01:00
Matthias Sohn 30b6887d44 JGit v5.11.0.202103091610-r
Change-Id: I8e6855eaf7228459f492036feb4e34ca085698a7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-09 22:10:22 +01:00
Matthias Sohn 1172c0b283 Prepare 5.11.0-SNAPSHOT builds
Change-Id: I89ed49a6acc53dd75d16f40c99e1140e0c18f646
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-04 16:27:51 +01:00
Matthias Sohn d60dd72310 JGit v5.11.0.202103031150-rc1
Change-Id: I0a86fa59645888f9f36ea6938c9121e095f02fc6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-03-03 17:48:00 +01:00
Matthias Sohn f659797199 Merge branch 'master' into next
* master: (143 commits)
  Prepare 5.11.0-SNAPSHOT builds
  JGit v5.11.0.202102240950-m3
  [releng] japicmp: update last release version
  IgnoreNode: include path to file for invalid .gitignore patterns
  FastIgnoreRule: include bad pattern in log message
  init: add config option to set default for the initial branch name
  init: allow specifying the initial branch name for the new repository
  Fail clone if initial branch doesn't exist in remote repository
  GPG: fix reading unprotected old-format secret keys
  Update Orbit to S20210216215844
  Add missing bazel dependency for o.e.j.gpg.bc.test
  GPG: handle extended private key format
  dfs: handle short copies
  [GPG] Provide a factory for the BouncyCastleGpgSigner
  Fix boxing warnings
  GPG: compute the keygrip to find a secret key
  GPG signature verification via BouncyCastle
  Post commit hook failure should not cause commit failure
  Allow to define additional Hook classes outside JGit
  GitHook: use default charset for output and error streams
  ...

Change-Id: I689f4070e79f4a0ac1c02b35698ccaab68ad2f34
2021-02-28 00:58:04 +01:00
Matthias Sohn 83d4f19a30 Prepare 5.11.0-SNAPSHOT builds
Change-Id: If3dbe084ee37ae4b993d3a10ec48b14e8709ff6d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-02-24 21:21:34 +01:00
Matthias Sohn 9eacae28df JGit v5.11.0.202102240950-m3
Change-Id: Iea6b3515fa63db497989194b6bf50fe7324086d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-02-24 15:50:41 +01:00
David Ostrovsky 4560bdf7e2 Migrate to Apache MINA sshd 2.6.0 and Orbit I20210203173513
Re-enable DSA, DSA_CERT, and RSA_CERT public key authentication.
DSA is discouraged for a long time already, but it might still be
way too disruptive to completely drop it. RSA is discouraged for
far less long, and dropping that would be really disruptive.

Adapt to the changed property handling. Remove work-arounds for
shortcomings of earlier sshd versions.

Use Orbit I20210203173513, which includes sshd 2.6.0. This also bumps
apache.httpclient to 4.5.13 and apache.httpcore to 4.4.14.

Change-Id: I2d24a1ce4cc9f616a94bb5c4bdaedbf20dc6638e
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-02-04 08:35:12 +01:00
Thomas Wolf 0853a2410f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.

Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 21:22:30 +01:00
Matthias Sohn 9034c7d423 Prepare 5.11.0-SNAPSHOT builds
Change-Id: I91e5532526775191fbd34f81e2ef777cba605e3b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:57:16 +01:00
Matthias Sohn 4f30dc5eb9 Ignore missing javadoc tags in test bundles
It seems Eclipse 4.18 reports them as error whereas earlier versions
ignored this maybe since we don't require javadoc comments for all the
test bundles.

Change-Id: I3f4d42ce681ea5c2b4b302991d2641290ac8561d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-29 20:16:38 +01:00
Matthias Sohn 286ad23cb5 Merge branch 'master' into next
* master:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Add support for reading symrefs from pack capabilities
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Prepare 5.10.0-SNAPSHOT builds
  JGit v5.10.0.202011251205-m3
  PacketLineIn: ensure that END != DELIM
  Update Orbit to S20201118210000 and add target for 4.18
  PacketLineIn: ensure that END != DELIM
  PacketLineIn: ensure that END != DELIM
  Allow to resolve a conflict by checking out a file
  Update Orbit to I20201111205634
  Document that setLastModified sets time of symlink target
  Fix bug in PerformanceLogContext
  Fix IOException occurring during gc
  Prepare 5.10.0-SNAPSHOT builds
  JGit v5.10.0.202011041322-m2
  Revert "Client-side protocol V2 support for fetching"
  Close Repository to fix tests failing on Windows
  Client-side protocol V2 support for fetching
  Update slf4j to 1.7.30
  Update Orbit to S20201027182932 (2020-12 M2)
  Fix formatting of config option values
  Document options in core section supported by JGit
  Ensure .gitmodules is loaded when accessing submodule name
  Export new package org.eclipse.jgit.logging and import it where used
  Ensure GC.deleteOrphans() can delete read-only orphaned files on Windows
  Add new performance logging
  Implement git describe --all
  Compute time differences with Duration
  Override config http.userAgent from environment GIT_HTTP_USER_AGENT
  Upgrade spotbugs-maven-plugin to 4.1.3
  Fix OperatorPrecedence warning flagged by error prone
  UploadPackTest#testUploadRedundantBytes: ensure test repo is closed
  ObjectDirectory#selectObjectRepresentation: fix formatting
  Upgrade ecj to 3.23.0
  Support "http.userAgent" and "http.extraHeader" from the git config
  sshd: better error report when user cancels authentication
  API filters for PackStatistics.Accumulator
  Add TypedConfigGetter.getPath()
  Make Javadoc consistent for PackStatistics fields
  Measure time taken for reachability checks
  Measure time taken for negotiation in protocol V2
  IndexDiffFilter: handle path prefixes correctly
  sshd: support the ProxyJump ssh config
  Upgrade jacoco-maven-plugin to 0.8.6
  ReceivePackStats: Add size and count of unnecessary pushed objects
  Upgrade maven-project-info-reports-plugin to 3.1.1
  Prepare 5.9.1-SNAPSHOT builds
  JGit v5.9.0.202009080501-r
  [releng] Enable japicmp for the fragments added in 5.8.0
  GitlinkMergeTest: fix boxing warnings
  Remove unused API problem filters
  Add missing since tag on BundleWriter#addObjectsAsIs
  SshdSession: close channel gracefully
  GPG: include signer's user ID in the signature
  jgit: Add DfsBundleWriter
  Bump Bazel version to 3.5.0
  Upgrade maven-resources-plugin to 3.2.0
  Upgrade plexus-compiler version to 2.8.8
  [bazel] Add missing dependency to slf4j-api
  [errorprone] DirCacheEntry: make clear operator precedence
  [errorprone] PackWriter#parallelDeltaSearch: avoid suppressed exception
  [errorprone] Declare DirCache#version final
  Add jgit-4.17-staging target platform for 2020-09
  Update target platform to R20200831200620
  Prepare 5.10.0-SNAPSHOT builds
  Prepare 5.9.0-SNAPSHOT builds
  ResolveMerger: do not content-merge gitlinks on del/mod conflicts
  ResolveMerger: Adding test cases for GITLINK deletion
  ResolveMerger: choose OURS on gitlink when ignoreConflicts
  ResolveMerger: improving content merge readability
  ResolveMerger: extracting createGitLinksMergeResult method
  ResolveMerger: Adding test cases for GITLINK merge
  JGit v5.9.0.202008260805-m3
  Fix possible NegativeArraySizeException in PackIndexV1
  FS: use binary search to determine filesystem timestamp resolution
  Do not prematurely create directory of jgit's XDG config file
  FS: write to JGit config in a background thread
  FS: don't cache fallback if running in background
  Keep line endings for text files committed with CR/LF on text=auto
  Delay WindowCache statistics JMX MBean registration
  [releng] Update plexus-compiler to 2.8.7
  DirCache: support index V4
  Update javadoc for RemoteSession and SshSessionFactory
  Fix JSchProcess.waitFor() with time-out
  sshd: work around a race condition in Apache MINA sshd 2.4.0/2.5.x
  sshd: store per-session data on the sshd session object
  FilterSpec: Use BigInteger.ZERO instead of valueOf(0)
  Do not send empty blob in response to blob:none filter
  Add support for tree filters when fetching
  sshd: use PropertyResolver in test
  FS_POSIX: avoid prompt to install the XCode tools on OS X
  Remove dependency on JSch from SSH test framework
  Use LinkedBlockingQueue for executor determining filesystem attributes
  Update API warning filters
  Remove unused imports
  Bazel: Add workspace status command to stamp final artifact
  DiffFormatter: correctly deal with tracked files in ignored folders
  Prepare 5.8.2-SNAPSHOT builds
  JGit v5.8.1.202007141445-r
  Update Jetty to 9.4.30.v20200611
  Fix writing GPG signatures with trailing newline
  Rename a test method
  Add a test for upstream bug SSHD-1028
  Improve error message when receive.maxCommandBytes is exceeded
  LfsConnectionFactory#getLfsUrl: Fix unconditional break in for-loop
  DiffFormatterTest: Add a test to confirm the default rename detection settings
  Upgrade maven-site-plugin to 3.9.1
  Upgrade build-helper-maven-plugin to 3.2.0
  Upgrade spotbugs to 4.0.4
  MergedReftable: Include the last reftable in determining minUpdateIndex
  Add new osgi fragments to maven-central deploy scripts
  PackBitmapIndex: Not buffer inflated bitmap during bitmap creation.
  Do not require org.assertj.core.annotations
  Upgrade ecj to 3.22.0
  Remove workaround for signing jars using Tycho plugins
  Use https for URL of jgit website
  Fix CI information in pom.xml
  Use gitiles as scm url in pom.xml for browsing source code
  Update API baseline to 5.8.0.202006091008-r
  Remove trailing whitespace

Change-Id: Ie6bc6954741a47cfbd32c0886bdbd7b594f08b31
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-28 21:51:50 +01:00
Thomas Wolf d69fb4d4ac Revert "Client-side protocol V2 support for fetching"
This reverts commit f802f06e7f.

I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.

Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does

  if protocol V2
    doFetchV2()
  else
    doFetchV0()

with doFetchV0 the old code, and doFetchV2 completely new.

Plus there would need to be a HTTP test case requiring several
negotiation rounds.

This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.

Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-11-03 23:50:21 +01:00
Thomas Wolf f802f06e7f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.

Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-10-29 00:36:21 +01:00
Thomas Wolf 566e49d7d3 sshd: support the ProxyJump ssh config
This is useful to access git repositories behind a bastion server
(jump host).

Add a constant for the config; rewrite the whole connection initiation
to parse the value and (recursively) set up the chain of hops. Add
tests for a single hop and two different ways to configure a two-hop
chain.

The connection timeout applies to each hop in the chain individually.

Change-Id: Idd25af95aa2ec5367404587e4e530b0663c03665
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-09-19 15:17:00 -04:00
Matthias Sohn 6ae1a30004 Prepare 5.10.0-SNAPSHOT builds
Change-Id: I9a2b39e9e85f27179ceb3b1709d75c466089a3bc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-08-27 16:14:36 +02:00
Thomas Wolf cc9975ff68 sshd: work around a race condition in Apache MINA sshd 2.4.0/2.5.x
When exceptions occur very early in the SSH connection setup, it's
possible that an exception gets lost. A subsequent authentication
attempt may then never be notified of the failure, and then wait
indefinitely or until its timeout expires.

This is caused by race conditions in sshd. The issue has been reported
upstream as SSHD-1050,[1] but will be fixed at the earliest in sshd
2.6.0.

[1] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-1050

Bug: 565394
Change-Id: If9b62839db38f9e59a5e1137c2257039ba82de98
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-10 10:20:06 +02:00
Thomas Wolf 835e3225a8 sshd: use PropertyResolver in test
Improve the SshTestGitServer API for accessing the server properties.
Instead of returning the raw property map, return the proper sshd API
abstraction PropertyResolver.

This makes the interface more resilient against upstream changes.

Change-Id: Ie5b685bddc4e59f3eb6c121026d3658d57618ca4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-07-28 09:47:07 +02:00
Thomas Wolf 0b9370bec8 Rename a test method
Change-Id: Ibb0bf0b1895a07a80a1f73d2fd6df2cb2d09fd77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-07-04 15:14:17 +02:00
Thomas Wolf 151f0cb82b Add a test for upstream bug SSHD-1028
SSHD-1028:[1] server doesn't close server-side sessions properly when
client disconnects.

[1] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-1028

Change-Id: I0d67f49e35abe8375cb1370a494dc01d0fb2c9b1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-07-03 20:53:59 +02:00
Matthias Sohn 4887894ffd Merge branch 'master' into next
* master:
  SimpleMergeTest: Clean up code style
  Prepare 5.8.1-SNAPSHOT builds
  Handle Fragment-Host declaration when updating version
  JGit v5.8.0.202006091008-r
  Prepare 5.9.0-SNAPSHOT builds
  Handle Fragment-Host declaration when updating version
  Add benchmark for strategies how to move a file
  Add getter for unpackErrorHandler in ReceivePack
  Upgrade maven-project-info-reports-plugin to 3.1.0
  Upgrade maven-shade-plugin to 3.2.4
  ObjectDirectoryInserter: Open FileOutputStream in try-with-resource
  ObjectDirectoryInserter: Remove redundant 'throws' declarations
  ObjectDirectory: Further clean up insertUnpackedObject
  Add Git#shutdown for releasing resources held by JGit process
  ApplyCommand: use context lines to determine hunk location
  GPG: don't prompt for a passphrase for unprotected keys
  Fix typo in org.eclipse.jgit.ssh.jsch.test MANIFEST
  Fix ProtectedMembersInFinalClass warning flagged by error prone
  Use version range to define fragment host bundle version
  ObjectDirectory: Explicitly handle NoSuchFileException
  ObjectDirectory: Fail immediately when atomic move is not supported
  Fix jgit packaging
  Fix InvalidInlineTag error flagged by error prone
  Fix BadComparable error flagged by error prone
  Add tests for RawTextComparator.WS_IGNORE_CHANGE.hash()
  Update Orbit to R20200529191137 for final Eclipse release 2020-06
  Organize manifest of org.eclipse.jgit.pgm
  Do not include log4j implementation in jgit
  Decouple JSch from JGit Core
  Decouple BouncyCastle from JGit Core
  Verify that the user home directory is valid
  WindowCache: conditional JMX setup
  RawTextComparator.WS_IGNORE_CHANGE must not compare whitespace
  Revert "PackBitmapIndex: Not buffer inflated bitmap in
BasePackBitmapIndex"
  Update jetty to 9.4.28.v20200408
  Add 4.16 staging target platform
  In-memory SSH keys for the "no files" sshd tests
  Builder API to configure SshdSessionFactories
  TransportHttp: abort on time-out or on SocketException
  Ignore core.eol if core.autocrlf=input
  Attributes: fix handling of text=auto in combination with eol
  Bazel: Remove superfluous dependencies flagged by unused_deps
  Log stack trace if CachingKeyPairProvider hits unexpected exception
  Update Orbit to S20200519202422 and ant to 1.10.8
  Include full IssuerFingerprint in GPG signature
  Bazel: Fix src_sha1 of bcpg-jdk15on
  Suppress API error for new method
BitmapIndex.Bitmap#retrieveCompressed
  Fix wrong @since tags added in dcb0265
  PackBitmapIndex: Set distance threshold
  PackBitmapIndex: Not buffer inflated bitmap in BasePackBitmapIndex
  PackBitmapIndex: Remove convertedBitmaps in the Remapper
  PackBitmapIndex: Reduce memory usage in GC
  PackBitmapIndex: Add AddToBitmapWithCacheFilter class
  PackBitmapIndex: Add util methods and builder to BitmapCommit
  PackBitmapIndex: Move BitmapCommit to a top-level class
  Refactor: Make retriveCompressed an method of the Bitmap class
  Fix downloading LFS Object fails behind proxy
  Allow for using custom s3 host with lfs server
  ReceivePack: adding IterativeConnectivityChecker
  Moving transport/internal -> internal/transport
  Fix error occurring during checkout

Change-Id: Ic11286e16ed6a72c6372297b310336dd040689d1
2020-06-10 10:33:50 +02:00
Matthias Sohn 855842af19 Prepare 5.9.0-SNAPSHOT builds
Change-Id: Ia998e2772df1285a4c674b07201f15d53156eb78
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-06-09 12:51:12 +02:00
Matthias Sohn 8d2d683655 Decouple JSch from JGit Core
Motivation: JSch serves as 'default' implementations of the SSH
transport. If a client application does not use it then there is no need
to pull in this dependency.

Move the classes depending on JSch to an OSGi fragment extending the
org.eclipse.jgit bundle and keep them in the same package as before
since moving them to another package would break API. Defer moving them
to a separate package to the next major release.

Add a new feature org.eclipse.jgit.ssh.jsch feature to enable
installation. With that users can now decide which of the ssh client
integrations (JCraft JSch or Apache Mina SSHD) they want to install.
We will remove the JCraft JSch integration in a later step due to the
reasons discussed in bug 520927.

Bug: 553625
Change-Id: I5979c8a9dbbe878a2e8ac0fbfde7230059d74dc2
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:46:59 +02:00
Thomas Wolf 5a5d85a4a3 In-memory SSH keys for the "no files" sshd tests
Avoid using a key written to a file. This makes it clearer that
the test does not rely on files being present.

Change-Id: I31cf4f404aab5b891c32fc4bda906b7f8fe03777
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-23 16:46:22 +02:00
Thomas Wolf 3a499606b1 Builder API to configure SshdSessionFactories
A builder API provides a more convenient way to define a customized
SshdSessionFactory by hiding the subclassing.

Also provide a new interface SshConfigStore to abstract away the
specifics of reading a ssh config file, and provide a way to customize
the concrete ssh config implementation to be used. This facilitates
using an alternate ssh config implementation that may or may not be
based on files.

Change-Id: Ib9038e8ff2a4eb3a9ce7b3554d1450befec8e1e1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-23 16:46:22 +02:00
Matthias Sohn 629fa260c0 Merge branch 'master' into next
* master:
  Prepare 5.8.0-SNAPSHOT builds
  JGit v5.8.0.202005061305-m2
  Update to bouncycastle 1.65 and orbit I20200506000552
  ApplyCommand: use Files#copy to copy file
  Apply hunks when renaming or copying from patch files
  Create parent directories when renaming a file in ApplyCommand
  Reduce BitmappedObjectReachabilityChecker visibility
  Add missing @since to new API
  Add missing test source file to the build
  Upgrade Tycho to 1.7.0
  ObjectReachabilityCheckers: Make walk member final
  Upgrade wagon-ssh to 3.4.0
  UploadPack: Use more relevant refs first in object reachability check
  UploadPack: Refactor to generalize the object reachability checks
  UploadPack: Use BitmappedReachabilityChecker for not advertised wants
  revwalk: Introduce bitmap-based object reachability checker
  Bump Bazel version to 3.1.0
  revwalk: Extract ObjectReachabilityChecker interface
  UploadPack: Extract walk-based reachability check
  Enable passing java options to jgit command line executable
  RefTreeBatch: fix unclosed resource warning
  CherryPickCommand: fix unclosed resource warning
  URIish: suppress non-localized message warning
  Always use https to access download.eclipse.org
  UploadPack: Clear advertised ref map after negotiation
  Use Map directly in MetaFilter
  Fix human name for local .bundle files
  Bazel: Disable SecurityManagerMissingPermissionsTest test
  Remove double blank from sentence start
  Upgrade maven-antrun-plugin to 3.0.0
  Upgrade maven-shade-plugin to 3.2.3
  Remove double blank from sentence start
  Bump Bazel version to 3.0.0
  Scan through all merged reftables for max/min update indices
  FileUtils: improve delete (Windows)
  FS.runInShell(): handle quoted filters and hooksPath containing blanks
  Document gc and pack relevant options
  Define constants for pack config option keys
  Fix javadoc typo
  Upgrade ecj to 3.21.0
  ReceivePack: Use error message if set
  Handle non-normalized index also for executable files
  Update to org.apache.sshd 2.4.0
  Scan through all merged reftables for max/min update indices
  ResolveMerger: Ignore merge conflicts if asked so
  Upgrade spotbugs-maven-plugin to 4.0.0
  Upgrade maven-javadoc-plugin to 3.2.0
  Upgrade maven-dependency-plugin to 3.1.2
  tag option for clone command
  Set baseline for japicmp to 5.7.0.202003110725-r
  RevWalk: fix bad topo flags error message
  RevWalk: new topo sort to not mix lines of history
  Upgrade maven-site-plugin to 3.9.0
  Upgrade build-helper-maven-plugin to 3.1.0
  Prepare 5.7.1-SNAPSHOT builds
  JGit v5.7.0.202003110725-r
  TransportHttp: support HTTP response 308 Permanent Redirect
  Remove unused API problem filters

Change-Id: Ifc0c42fd3881b6026b0dcf7a2eb599e7cdede67e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-05-07 01:12:00 +02:00
Thomas Wolf fd3778b935 Update to org.apache.sshd 2.4.0
Change target platforms to Orbit I20200319180910 and regenerate them.
Change package imports to [2.4.0,2.5.0); adapt code to upstream API
changes.

Maven build: update version in root pom.

Bazel build: update version & hash in WORKSPACE file.

Proxy functionality verified manually using 3proxy (HTTP & SOCKS,
with basic authentication) and ssh -vvv -D7020 localhost (SOCKS, no
authentication).

Bug: 561078
Change-Id: I582f6b98055b013c006f2c749890fe6db801cbaa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-03-22 00:37:05 +01:00