Commit Graph

222 Commits

Author SHA1 Message Date
Terry Parker dbd05433ec Move reachability checker generation into the ObjectReader object
Reachability checkers are retrieved from RevWalk and ObjectWalk objects:
* RevWalk.createReachabilityChecker()
* ObjectWalk.createObjectReachabilityChecker()

Since RevWalks and ObjectWalks are themselves directly instantiated
in hundreds of places (e.g. UploadPack...) overriding them in a
consistent way requires overloading 100s of methods, which isn't
feasible. Moving reachability checker generation to a more central
place solves that problem.

The ObjectReader object seems a good place from which to get
reachability checkers, because reachability checkers return
information about relationships between objects. ObjectDatabases
delegate many operations to ObjectReaders, and reachability bitmaps
are attached to ObjectReaders.

The Bitmapped and Pedestrian reachability checker objects were
package private in the org.eclipse.jgit.revwalk package. This change
makes them public and moves them to the
org.eclipse.jgit.internal.revwalk package. Corresponding tests are
also moved.

Motivation:
1) Reachability checking algorithms need to scale. One of the
   internal Android repositories has ~2.4 million refs/changes/*
   references, causing bad long tail performance in reachability
   checks.
2) Reachability check performance is impacted by repository
   topography: number of refs, number of objects, amounts of
   related vs. unrelated history.
3) Reachability check performance is also affected by per-branch
   access (Gerrit branch permissions) since different users can
   see different branches.
4) Reachability check performance isn't affected by any state in a
   RevWalk or ObjectWalk.

I don't yet know if a single algorithm will work for all cases in #2
and #3. We may need to evolve the ReachabilityChecker interfaces
over time to solve the Gerrit branch permissions case, or use
Gerrit-specific identity information to solve that in an efficient
way.

This change takes the existing public API and moves it to the
ObjectReader/whole repository level, which is where we can do
consistent customizations for #2 and #3. We intend to upstream the
best of whatever works, but anticipate the need for multiple rounds
of experimentation.

Change-Id: I9185feff43551fb387957c436112d5250486833d
Signed-off-by: Terry Parker <tparker@google.com>
2021-01-28 22:17:26 -08:00
Han-Wen Nienhuys 7386f73bb0 Remove reftree and ketch
This was experimental code and never used in production.

Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 02:00:35 +01:00
Matthias Sohn 9034c7d423 Prepare 5.11.0-SNAPSHOT builds
Change-Id: I91e5532526775191fbd34f81e2ef777cba605e3b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:57:16 +01:00
Matthias Sohn 8269587f49 Export new package org.eclipse.jgit.logging and import it where used
Change-Id: Ib81b8733e184e75c2cba21ab27c4bbe8dbe0b04d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-10-25 15:45:49 +01:00
Matthias Sohn 6ae1a30004 Prepare 5.10.0-SNAPSHOT builds
Change-Id: I9a2b39e9e85f27179ceb3b1709d75c466089a3bc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-08-27 16:14:36 +02:00
Matthias Sohn 855842af19 Prepare 5.9.0-SNAPSHOT builds
Change-Id: Ia998e2772df1285a4c674b07201f15d53156eb78
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-06-09 12:51:12 +02:00
Matthias Sohn 8d2d683655 Decouple JSch from JGit Core
Motivation: JSch serves as 'default' implementations of the SSH
transport. If a client application does not use it then there is no need
to pull in this dependency.

Move the classes depending on JSch to an OSGi fragment extending the
org.eclipse.jgit bundle and keep them in the same package as before
since moving them to another package would break API. Defer moving them
to a separate package to the next major release.

Add a new feature org.eclipse.jgit.ssh.jsch feature to enable
installation. With that users can now decide which of the ssh client
integrations (JCraft JSch or Apache Mina SSHD) they want to install.
We will remove the JCraft JSch integration in a later step due to the
reasons discussed in bug 520927.

Bug: 553625
Change-Id: I5979c8a9dbbe878a2e8ac0fbfde7230059d74dc2
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:46:59 +02:00
Matthias Sohn 77848d635b Decouple BouncyCastle from JGit Core
Motivation: BouncyCastle serves as 'default' implementation of
the GPG Signer. If a client application does not use it there is no need
to pull in this dependency, especially since BouncyCastle is a large
library.

Move the classes depending on BouncyCastle to an OSGi fragment extending
the org.eclipse.jgit bundle. They are moved to a distinct internal
package in order to avoid split packages. This doesn't break public API
since these classes were already in an internal package before this
change.

Add a new feature org.eclipse.jgit.gpg.bc to enable installation. With
that users can now decide if they want to install it.

Attempts to sign a commit if org.eclipse.jgit.gpg.bc isn't available
will result in ServiceUnavailableException being thrown.

Bug: 559106
Change-Id: I42fd6c00002e17aa9a7be96ae434b538ea86ccf8
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:26:22 +02:00
Thomas Wolf 4d7a16257f Include full IssuerFingerprint in GPG signature
Update dependency to Bouncy Castle to 1.65.

Add the IssuerFingerprint as a hashed sub-packet in the signature. If
added unhashed, GPG ignores it.

Bug: 553206
Change-Id: I6807e8e2385e6ec5790f388e4753a44aa9474ebb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-18 23:25:58 +02:00
Demetr Starshov 519cb1e91b Moving transport/internal -> internal/transport
Moving transport related internal classes into dedicated subpackage in
o/e/j/internal package.

Signed-off-by: Demetr Starshov <dstarshov@google.com>
Change-Id: I21ed029d359f5f7d8298f102efbb4b1dcdf404ad
2020-05-08 17:57:20 -07:00
Matthias Sohn 8fab712f75 Prepare 5.8.0-SNAPSHOT builds
Change-Id: I056b45806a82eae80177932e42e3dc806015351a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-09 22:47:10 +01:00
Michael Dardis 4055522359 Remove use of org.bouncycastle.util.io.TeeOutputStream
Bug: 559106
Change-Id: Ife06f21a00d258780a2030745738194098c631cf
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-21 23:45:43 +01:00
Matthias Sohn 23c5ebff79 Merge branch 'stable-5.6'
* stable-5.6:
  Add ability to redirect stderr from git hooks
  Add possibility to get pure stderr output from AbortedByHookException

Change-Id: Ifc02675542dad6ced25fdd8b9fae80b5736db688
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-09 10:15:05 +01:00
Tim Neumann 23125abc20 Add ability to redirect stderr from git hooks
This will change the behavior in the CLI to resemble that of C-Git more
closely by printing the stderr of the hooks to the CLI stderr
independently of the exit code of the hook.

This is also useful for the corresponding EGIT-Change, which will add
the ability to show the hook output in eclipse.
With this also the stderr can be shown even if the exit code is 0.

Bug: 553471
Change-Id: Ie7bc503fe39e270e9b93dd1108b5879f02a12b4c
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
2019-12-09 00:56:40 +01:00
Matthias Sohn f487a9eb2e Prepare 5.7.0-SNAPSHOT builds
Change-Id: I7efa0cd08dee711d6fefc3e6bb478a3cb11fe85c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-05 01:32:59 +01:00
Matthias Sohn 473527e42c Prepare 5.6.0-SNAPSHOT builds
Change-Id: I5e737ff7f262fdd43fc975a0b3594c8b33919663
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 22:13:56 +01:00
Matthias Sohn a7e454bc51 JGit v5.6.0.201912041214-rc1
Change-Id: I5ed21fbc5f83096bf0b79f2aa751db415cbcc7e8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-04 18:14:43 +01:00
Matthias Sohn 4ea42bc816 Prepare 5.6.0-SNAPSHOT builds
Change-Id: I1e52d2dfb202b87ecb9d0273deaa2c8d8ce1864e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 21:25:38 +01:00
Matthias Sohn 6c9aa82f70 JGit v5.6.0.201911271000-m3
Change-Id: I3810892c2bed947e4dfaa615a7b4d93eeb46abb9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-27 16:01:20 +01:00
Thomas Wolf 6a39da37fe GPG: implement more OpenPGP UserId matching formats
Instead of just looking for a substring match of user.signingKey
in a key's user ID implement the GPG matching formats[1] for:

'=' Full exact match
'<' Full exact match of the e-mail address
'@' Substring match within the e-mail address only
'*' General case-insensitive substring match (default)

When user.signingKey is not set, the committer's e-mail address is
used by default. In that case, use '<', i.e., require an exact match
on the OpenPGP e-mail address.

Also handle the optional "0x" prefix for (partial) key fingerprints.

[1] https://www.gnupg.org/documentation/manuals/gnupg/Specify-a-User-ID.html

Bug: 550335
Change-Id: I6ce482a099ff1a0dc9de45435cd4d3ec5b504f12
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-23 11:17:24 +02:00
Matthias Sohn 5e24753ab0 Prepare 5.6.0-SNAPSHOT builds
Change-Id: I9eceb7b1272b27b979144041c75eb09ab4eeca6b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-29 10:08:52 +02:00
Michael Keppler 4b0c192f2d Fix bundle localization of Apache SSH bundle
The placeholders in manifest and plugin.properties did not match. To
avoid similar issues, all placeholders have been changed to
Bundle-Vendor and Bundle-Name now.

Bug:548503
Change-Id: Ibd4b9bc237b323e614506b97e5fbc99416365040
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-06-21 17:54:06 +02:00
Matthias Sohn 16fd3feefc Prepare 5.5.0-SNAPSHOT builds
Change-Id: I177d637e552a79014816dc5d2ef5ccda506adb39
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-13 03:01:15 +02:00
Matthias Sohn f3b7c2beae Prepare 5.4.1-SNAPSHOT builds
Change-Id: I8620d03c1cd5c1d0ad04e7607553e9aa18def9bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-12 17:35:11 +02:00
Matthias Sohn 3dad0682be JGit v5.4.0.201906121030-r
Change-Id: Ie68828af68d07cf8e3fe778d39436f539f1c73d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-12 16:29:11 +02:00
Matthias Sohn 0e9361f066 JGit v5.4.0.201906120450-r
Change-Id: Iea17cba848d4e53d69d34e952a2476c375721aa4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-12 10:50:14 +02:00
Konrad Windszus d7bd2e700c Support reading and writing cookies.
The git config entries "http.cookieFile" and
"http.saveCookies" are correctly evaluated.

Bug: 488572
Change-Id: Icfeeea95e1a5bac3fa4438849d4ac2306d7d5562
Signed-off-by: Konrad Windszus <konrad_w@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-06 19:02:17 +02:00
Brandon Weeks f4443ef3f9 Upgrade Bouncy Castle to 1.61
Also now use JcaKeyBoxBuilder constructor in
BouncyCastleGpgKeyLocator.readKeyBoxFile(Path).

CQ: 19868
CQ: 19869
CQ: 19870
Change-Id: I45bd80e158aecd90448b0c7e59615db27aaef892
Signed-off-by: Brandon Weeks <bweeks@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-01 01:39:26 +02:00
Matthias Sohn 90107d53eb Prepare 5.4.0-SNAPSHOT builds
Change-Id: I90a4791f63d0eba23da744c720e869f1830b86e7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-07 00:34:14 +01:00
Medha Bhargav Prabhala 137e91a465 Implement signing commits using BouncyCastle
This also includes a change to generating the jgit CLI jar. Shading is
no longer possible because it breaks the signature of BouncyCastle.
Instead, the Spring Boot Loader Maven plug-in is now used to generate an
executable jar.

Bug: 382212
Change-Id: I35ee3d4b06d9d479475ab2e51b29bed49661bbdc
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-01-16 10:05:10 +01:00
Michael Keppler 0d9e486714 Replace deprecated FirstLine by FirstCommand
and allow package org.eclipse.jgit.http.server to use package
org.eclipse.jgit.internal.transport.parser.

Change-Id: Ief330c3e75a735853d0a5a265a9ff56fb5128b99
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-31 16:31:37 +01:00
Jonathan Nieder b4e415f8a9 Move first line parsing for v0 push out of BaseReceivePack
This simplifies the BaseReceivePack class and decreases its API
surface, which should make merging with ReceivePack easier.

Inspired by 6aca8899a5 (Move first line
parsing for v0/v1 pack negotiation out of UploadPack, 2018-09-17).

Change-Id: I1fc175d15aa7cb5968c26fc83a95480403af617c
2018-12-27 14:07:41 -08:00
Matthias Sohn 38da21d6b0 Prepare 5.3.0-SNAPSHOT builds
Change-Id: I8951c2cf650cc3e41d2baa0b330b94468cfed5c2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-03 02:32:49 +01:00
Thomas Wolf 488d95571f Apache MINA sshd client
Add a new ssh client implementation based on Apach MINA sshd 2.0.0.

This implementation uses JGit's own config file parser and host entry
resolver. Code inspection of the Apache MINA implementation revealed
a few bugs or idiosyncrasies that immediately would re-introduce bugs
already fixed in the past in JGit.

Apache MINA sshd is not without quirks either, and I had to configure
and override more than I had expected. But at least it was all doable
in clean ways.

Apache MINA boasts support for Bouncy Castle, so in theory this should
open the way to using more ssh key algorithms, such as ed25519.

The implementation is in a separate bundle and is still not used in
the core org.eclipse.jgit bundle. The tests re-use the ssh tests from
the core test bundle.

Bug: 520927
Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-11-13 10:49:26 -08:00
Thomas Wolf cc000f93a8 Factor out a JSch-independent ssh config parser
Move the bulk of the basic parsing and host entry handling into a
new class OpenSshConfigFile that has no dependencies on any concrete
ssh implementation. Make the existing OpenSshConfig use the new
parser.

Introduce a new class SshConstants collecting all the various ssh-
related string literals. Also use TreeMaps with a case-insensitive
key comparator instead of converting keys to uppercase. Add a test
to verify that keys are matched case-insensitively.

Most of the parsing code was simply moved, except that the new
parser supports looking up entries given host name, port, and user
name, and can thus handle more %-substitutions correctly. This
feature is not yet used and cannot be used with JSch since JSch
only has a ConfigRepository.getConfig(String) interface.

The split is still worth the trouble as it opens the way to using
another ssh client altogether. Apache MINA sshd, for instance,
resolves host entries giving host name, port, and user name.

(Apache MINA has a built-in ssh config handling, but that has
problems, too: its pattern matching is case-insensitive, and its
merging of host entries if several match is not the same as in
OpenSsh. But with this refactoring, it will be possible to plug in
OpenSshConfigFile into an Apache MINA sshd  client without dragging
along JSch.)

One test case that doesn't make sense anymore has been removed. It
tested that repeatedly querying for a host entry returned the same
object. That is no longer true since the caching has been moved to
a deeper level.

Bug: 520927
Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-11-06 13:17:22 +01:00
Thomas Wolf f8d2738ff4 Fix package exports in org.eclipse.jgit MANIFEST.MF
Do not export o.e.j.internal.transport.parser as public package;
restrict visibility to org.eclipse.jgit.test only.

Add two packages that were not listed at all (o.e.j.internal.revwalk
and o.e.j.internal.submodule) marked as x-internal:=true.

Change-Id: I9188356075515ad354b724102fbd6304b682de6a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-11-05 12:56:36 +01:00
Ivan Frade 6aca8899a5 Move first line parsing for v0/v1 pack negotiation out of UploadPack
In protocol v0/v1 pack negotiation, the first want line contains the
options the client wants in effect. This parsing is done in UploadPack
but it doesn't have any interaction with that class.

Move the code to its own class and package, mark the current one
as deprecated (it is public API) and add unit tests.

Take the chance to move the parsing code from the constructor to a
factory method, making the class a simple container of results.

Change-Id: I1757f535dda78a4111a1c12c3a3b455a4b6f0c51
Signed-off-by: Ivan Frade <ifrade@google.com>
2018-10-12 13:59:06 -07:00
Matthias Sohn 2fae326aa9 Prepare 5.2.0-SNAPSHOT builds
Change-Id: If8e8ca20e745901a5e0239c16cb4acc7934b4ec8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-09-06 09:36:00 +02:00
Matthias Sohn 2f1b4ffcd2 Prepare 5.1.0-SNAPSHOT builds
Change-Id: I8523a993ae1f7b62573d7547273bc1356bf64fa7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-05-23 10:18:18 +02:00
Matthias Sohn 10d03a77a4 Merge branch 'stable-4.11'
* stable-4.11:
  Remove package import for javax.servlet.http from org.eclipse.jgit
  Add missing @since tag and silence API error

Change-Id: I2783a15ead26ab19de31a8fb3bfb148ef19de91a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-26 16:35:30 -07:00
Matthias Sohn a263bb1173 Remove package import for javax.servlet.http from org.eclipse.jgit
This package import is unused.

Change-Id: I9f202bb3162736a1ef8054516f1a3145d3a7bb9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-27 00:36:20 +02:00
Matthias Sohn d0f1f5eef6 Prepare 5.0.0-SNAPSHOT builds
Change-Id: I2d2f50ed8a12f310e7cac68eed5536bd460c403f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-08 23:09:18 +01:00
Matthias Sohn f389e18a28 Prepare 4.11.1-SNAPSHOT builds
Change-Id: Id9aa6b7e8f56de5183b6cd57ef0e790ec9debd4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-08 21:10:08 +01:00
Matthias Sohn f944063517 JGit v4.11.0.201803080745-r
Change-Id: Ie24a33bc8a24c30db06fe7b175f405efb95776ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-08 13:43:46 +01:00
Markus Duft 94bcde663c LFS: Add remote download to SmudgeFilter
Transfer data in chunks of 8k Transferring data byte per byte is slow,
running checkout with CleanFilter on a 2.9MB file takes 20 seconds.
Using a buffer of 8k shrinks this time to 70ms.

Also register the filter commands in a way that the native GIT LFS can
be used alongside with JGit.

Implements auto-discovery of LFS server URL when cloning from a Gerrit
LFS server.

Change-Id: I452a5aa177dcb346d92af08b27c2e35200f246fd
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
2018-02-16 18:27:25 +01:00
Matthias Sohn 5a4b6fd237 Prepare 4.11.0-SNAPSHOT builds
Change-Id: I5d5e2befcf530d93457d44684bd9e4fc2392e5eb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-31 04:05:07 +01:00
Karsten Thoms 3832527b02 Add header Automatic-Module-Name for Java 9
Bug: 529075
Change-Id: I4532ce2c80eb91531d46026676502d636ccda706
Signed-off-by: Karsten Thoms <karsten.thoms@itemis.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-23 01:00:01 +01:00
Matthias Sohn ced98986f9 Fix discouraged access warnings in RefsUnreadableInMemoryRepository
Change-Id: I16e291f533625c80b1d39f7cbaf40efbdfcc2c3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:23:07 +01:00
Matthias Sohn 6b544da293 Prepare 4.10.0-SNAPSHOT builds
Change-Id: I5ca462d1db18a2c5c9382cfb9c83972510fa2b88
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-10-08 11:35:54 +02:00
Shawn Pearce 0398f3dd6e reftable: debug tools
Simple debug programs to experiment with the reftable file format:

  debug-read-reftable
  debug-write-reftable
  debug-verify-reftable
  debug-benchmark-reftable

Change-Id: I79db351d86900f1e58b17e922e195dff06ee71f1
2017-08-17 15:06:51 -07:00