Commit Graph

413 Commits

Author SHA1 Message Date
Thomas Wolf 3774fcc848 GPG signature verification via BouncyCastle
Add a GpgSignatureVerifier interface, plus a factory to create
instances thereof that is provided via the ServiceLoader mechanism.

Implement the new interface for BouncyCastle. A verifier maintains
an internal LRU cache of previously found public keys to speed up
verifying multiple objects (tag or commits). Mergetags are not handled.

Provide a new VerifySignatureCommand in org.eclipse.jgit.api together
with a factory method Git.verifySignature(). The command can verify
signatures on tags or commits, and can be limited to accept only tags
or commits. Provide a new public WrongObjectTypeException thrown when
the command is limited to either tags or commits and a name resolves
to some other object kind.

In jgit.pgm, implement "git tag -v", "git log --show-signature", and
"git show --show-signature". The output is similar to command-line
gpg invoked via git, but not identical. In particular, lines are not
prefixed by "gpg:" but by "bc:".

Trust levels for public keys are read from the keys' trust packets,
not from GPG's internal trust database. A trust packet may or may
not be set. Command-line GPG produces more warning lines depending
on the trust level, warning about keys with a trust level below
"full".

There are no unit tests because JGit still doesn't have any setup to
do signing unit tests; this would require at least a faked .gpg
directory with pre-created key rings and keys, and a way to make the
BouncyCastle classes use that directory instead of the default. See
bug 547538 and also bug 544847.

Tested manually with a small test repository containing signed and
unsigned commits and tags, with signatures made with different keys
and made by command-line git using GPG 2.2.25 and by JGit using
BouncyCastle 1.65.

Bug: 547751
Change-Id: If7e34aeed6ca6636a92bf774d893d98f6d459181
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-02-16 00:37:00 +01:00
Thomas Wolf 58f4e6e7f8 TransportHttp: support preemptive Basic authentication
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.

Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.

Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.

No preemptive authentication will be done if the connection is
redirected to a different host.

Add tests.

Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 471ad49546 TransportHttp: shared SSLContext during fetch or push
TransportHttp makes several HTTP requests. The SSLContext and socket
factory must be shared over these requests, otherwise authentication
information may not be propagated correctly from one request to the
next. This is important for authentication mechanisms that rely on
client-side state, like NEGOTIATE (either NTLM, if the underlying HTTP
library supports it, or Kerberos). In particular, SPNEGO cannot
authenticate on a POST request; the authentication must come from the
initial GET request, which implies that the POST request must use the
same SSLContext and socket factory that was used for the GET.

Change the way HTTPS connections are configured. Introduce the concept
of a GitSession, which is a client-side HTTP session over several HTTPS
requests. TransportHttp creates such a session and uses it to configure
all HTTP requests during that session (fetch or push). This gives a way
to abstract away the differences between JDK and Apache HTTP connections
and to configure SSL setup outside.

A GitSession can maintain state and thus give all HTTP requests in a
session the same socket factory.

Introduce an extension interface HttpConnectionFactory2 that adds a
method to obtain a new GitSession. Implement this for both existing
HTTP connection factories. Change TransportHttp to use the new
GitSession to configure HTTP connections.

The old methods for disabling SSL verification still exist to support
possibly external connection and connection factory implementations
that do not make use of the new GitSession yet.

Bug: 535850
Change-Id: Iedf67464e4e353c1883447c13c86b5a838e678f1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 224aaa0be7 TransportHttp: make the connection factory configurable
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.

Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.

Make the global static factory reference volatile.

Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:44 +01:00
Thomas Wolf 0853a2410f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.

Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 21:22:30 +01:00
Han-Wen Nienhuys 7386f73bb0 Remove reftree and ketch
This was experimental code and never used in production.

Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 02:00:35 +01:00
Thomas Wolf 41b9159795 TagCommand: support signing annotated tags
Add the two config constants from C git that can switch on signing
of annotated tags. Add them to the GpgConfig, and implement actually
signing a tag in TagCommand.

The interactions between command line options for "git tag" and config
options is a bit murky in C git. There are two config settings for it:

* tag.gpgSign is the main option, if set to true, it kicks in if
  neither -s nor -u are given on the command line.
* tag.forceSignAnnotated signs only tags created via "git tag -m",
  but only if command-line option "-a" is not present. It applies
  even if tag.gpgSign is set explicitly to false.

Giving -s or -u on the command line also forces an annotated tag
since lightweight tags cannot be signed.

Bug: 386908
Change-Id: Ic8a1a44b5f12f47d5cdf3aae2456c1f6ca9ef057
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-07 09:33:57 +01:00
Thomas Wolf 5abd8a4feb Enable GpgSigner to also sign tags
Factor out a common ObjectBuilder as super class of CommitBuilder
and TagBuilder, and make the GpgSigner work on ObjectBuilder.

In order not to break API, add the new method for signing an
ObjectBuilder in a new interface GpgObjectSigner.

The signature for a tag is just tacked onto the end of the tag
message. The message of a signed tag must end in LF.

Bug: 386908
Change-Id: I5e021e3c927f4051825cd7355b129113b949455e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-07 09:04:33 +01:00
Matthias Sohn d4fe93f55b Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  PacketLineIn: ensure that END != DELIM
  Update API warning filters
  Remove unused imports

Change-Id: Icf415ac5bab2f69f71189c942424ee69f8a64d4e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 10:10:44 +01:00
Matthias Sohn 5cd485e5dd Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  PacketLineIn: ensure that END != DELIM
  Update API warning filters
  Remove unused imports

Change-Id: I70b399eb3df02aa2fb112e86e844cf5a0daa5515
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 09:38:42 +01:00
Matthias Sohn d1801402fe Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I9b94938f5c09bd726e8e368c98c56da8280fd0b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 09:07:30 +01:00
Matthias Sohn 99f5329c38 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Ie24d381f295cccfb99068c7ed5817179da29c1db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 01:03:02 +01:00
Matthias Sohn 480b00f1c7 Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I00c8ddad0059a20a978743dfb7ad1b513dc7fef6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 01:01:31 +01:00
Matthias Sohn 95846863cb Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Icc34f809a3bb019d8d640b9bdb71363e617942e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 00:40:26 +01:00
Matthias Sohn a74c0cd333 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I82576ad0f61cf3ff11b54691b32666c61401ad9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 00:29:53 +01:00
Matthias Sohn 3f85d3b75c Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Id9386846a202b5ae98dd602744963f8897ddaa8c
2020-11-26 15:37:19 +01:00
Matthias Sohn 65598b9516 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I91cfe2820c40d2d773cbf018cc2a6c36b062801e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 15:34:24 +01:00
Matthias Sohn 0f1735de0b GC#deleteOrphans: log warning for deleted orphaned files
Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 01:34:42 +01:00
Thomas Wolf d69fb4d4ac Revert "Client-side protocol V2 support for fetching"
This reverts commit f802f06e7f.

I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.

Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does

  if protocol V2
    doFetchV2()
  else
    doFetchV0()

with doFetchV0 the old code, and doFetchV2 completely new.

Plus there would need to be a HTTP test case requiring several
negotiation rounds.

This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.

Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-11-03 23:50:21 +01:00
Thomas Wolf f802f06e7f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.

Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-10-29 00:36:21 +01:00
James Wynn 2171f868d9 Support "http.userAgent" and "http.extraHeader" from the git config
Validate the extra headers and log but otherwise ignore invalid
headers. An empty http.extraHeader starts the list afresh.

The http.userAgent is restricted to printable 7-bit ASCII, other
characters are replaced by '.'.

Moves a support method from the ssh.apache bundle to HttpSupport in
the main JGit bundle.

Bug:541500
Change-Id: Id2d8df12914e2cdbd936ff00dc824d8f871bd580
Signed-off-by: James Wynn <james@jameswynn.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-09-26 23:42:38 +02:00
Thomas Wolf 2990ad66ad FS: use binary search to determine filesystem timestamp resolution
Previous code used a minimum granularity of 1 microsecond and would
iterate 233 times on a system where the resolution is 1 second (for
instance, Java 8 on Mac APFS).

New code uses a binary search between the maximum we care about (2
seconds) and zero, with a minimum granularity of also 1 microsecond.
This takes at most 19 iterations (guaranteed). For a file system with 1
second resolution, it takes 4 iterations (1s, 0.5s, 0.8s, 0.9s). With
an up-front check at 1 microsecond and at 1 millisecond this performs
equally well as the old code on file systems with a fine resolution.
(For instance, Java 11 on Mac APFS.)

Also handle obscure cases where the file timestamp implementation may
yield bogus values (as observed on HP NonStop). If such an error case
occurs, log a warning and abort the measurement at the last good value.

Bug: 565707
Change-Id: I82a96729b50c284be7c23fbdf3d0df1bddf60e41
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-23 11:39:24 +02:00
Matthias Sohn e9c7ba6fdc Do not prematurely create directory of jgit's XDG config file
LockFile.lock() will create it anyway when the config file is created.

Bug: 565637
Change-Id: I078b89a695193fd76f130f6de7ac1cf26d2f8f0f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-22 22:57:14 +02:00
Thomas Wolf e9cb0a8e47 DirCache: support index V4
Index format version 4 was introduced in C git in 2012. It's about
time that JGit can deal with it.

Version 4 added prefix path compression. Instead of writing the full
path for each index entry to disk, only the difference to the previous
entry's path is written: a variable-encoded int telling how many bytes
to remove from the previous entry's path to get the common prefix,
followed by the new suffix.

Also, cache entries in a version 4 index are not padded anymore.

Internally, version 3 and version 4 index entries are identical; it's
only the stored format that changes.

Implement this path compression, and make sure we write an index file
that we read previously in the same format. (Only changing from version
2 to version 3 if there are extended flags.)

Add support for the "feature.manyFiles" and the "index.version" git
configs, and honor them when writing a new index file.

Add tests, including a compatibility test that verifies that JGit can
read a version 4 index generated by C git and write an identical
version 4 index.

Bug: 565774
Change-Id: Id83241cf009e50f950eb42f8d56b834fb47da1ed
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-15 12:47:45 +02:00
Thomas Wolf 24fdc1d039 Fix JSchProcess.waitFor() with time-out
SshSupport.runSshCommand() had a comment that wait with time-out
could not be used because JSchProcess.exitValue() threw the wrong
unchecked exception when the process was still running.

Fix this and make JSchProcess.exitValue() throw the right exception,
then wait with a time-out in SshSupport.

The Apache sshd client's SshdExecProcess has always used the correct
IllegalThreadStateException.

Add tests for SshSupport.runCommand().

Change-Id: Id30893174ae8be3b9a16119674049337b0cf4381
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-08-10 22:51:34 +02:00
David Pursehouse 8774f54190 Improve error message when receive.maxCommandBytes is exceeded
The message "Too many commands" implies there is a hard limit on the
number of commands, which isn't the case. The limit is on the total
size of the received data, as explained in change I84317d396 which
introduced the configuration setting receive.maxCommandBytes:

  shorter reference names allow for more commands, longer reference
  names permit fewer commands per batch.

Change the message to:

  Commands size exceeds limit defined in receive.maxCommandBytes

Change-Id: I678b78f919b2fec8f8058f3403f2541c26a5d00e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-06-29 08:57:42 +09:00
Matthias Sohn 8d2d683655 Decouple JSch from JGit Core
Motivation: JSch serves as 'default' implementations of the SSH
transport. If a client application does not use it then there is no need
to pull in this dependency.

Move the classes depending on JSch to an OSGi fragment extending the
org.eclipse.jgit bundle and keep them in the same package as before
since moving them to another package would break API. Defer moving them
to a separate package to the next major release.

Add a new feature org.eclipse.jgit.ssh.jsch feature to enable
installation. With that users can now decide which of the ssh client
integrations (JCraft JSch or Apache Mina SSHD) they want to install.
We will remove the JCraft JSch integration in a later step due to the
reasons discussed in bug 520927.

Bug: 553625
Change-Id: I5979c8a9dbbe878a2e8ac0fbfde7230059d74dc2
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:46:59 +02:00
Matthias Sohn 77848d635b Decouple BouncyCastle from JGit Core
Motivation: BouncyCastle serves as 'default' implementation of
the GPG Signer. If a client application does not use it there is no need
to pull in this dependency, especially since BouncyCastle is a large
library.

Move the classes depending on BouncyCastle to an OSGi fragment extending
the org.eclipse.jgit bundle. They are moved to a distinct internal
package in order to avoid split packages. This doesn't break public API
since these classes were already in an internal package before this
change.

Add a new feature org.eclipse.jgit.gpg.bc to enable installation. With
that users can now decide if they want to install it.

Attempts to sign a commit if org.eclipse.jgit.gpg.bc isn't available
will result in ServiceUnavailableException being thrown.

Bug: 559106
Change-Id: I42fd6c00002e17aa9a7be96ae434b538ea86ccf8
Also-by: Michael Dardis <git@md-5.net>
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2020-06-01 01:26:22 +02:00
Thomas Wolf 0b2d41b858 Verify that the user home directory is valid
If the determination of the user home directory produces a Java File
object with an invalid path, spurious exceptions may occur at the
most inopportune moments anytime later. In the case in the linked bug
report, start-up of EGit failed, leading to numerous user-visible
problems in Eclipse.

So validate the return value of FS.userHomeImpl(). If converting that
File to a Path throws an exception, log the problem and fall back to
Java system property user.home. If that also is not valid, use null.

(A null user home directory is allowed by FS, and calling in Java
new File(null, "some_string") is fine and produces a File relative
to the current working directory.)

Bug: 563739
Change-Id: If9eec0f9a31a45bd815231706285c71b09f8cf56
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-05-31 12:47:21 -04:00
Matthias Sohn a0aebbd528 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Remove double blank from sentence start
  Bump Bazel version to 3.0.0

Change-Id: I26c3a8345020239d1c2ec5c6f70a633b43ddab86
2020-04-18 00:30:15 +02:00
Michael Keppler a79c5b1f10 Remove double blank from sentence start
Multiple whitespaces are not normalized when reading properties files,
therefore leading to unwanted space/indentation in console or UI output.

Change-Id: I1f5224fe359e0cac493e0237872afc75dc8b9fbe
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ebbc3efce7)
2020-04-17 14:36:50 +09:00
Michael Keppler ebbc3efce7 Remove double blank from sentence start
Multiple whitespaces are not normalized when reading properties files,
therefore leading to unwanted space/indentation in console or UI output.

Change-Id: I1f5224fe359e0cac493e0237872afc75dc8b9fbe
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2020-04-14 09:31:50 +02:00
Alex Spradlin cf8c84c6d1 RevWalk: fix bad topo flags error message
The error message for an Exception thrown by StartGenerator when given
both the TOPO flag and the TOPO_KEEP_BRANCH_TOGETHER flag mentions a
non-existent flag, TOPO_NON_INTERMIX. The error message was introduced
in commit e498d43.

Replace TOPO_NON_INTERMIX with TOPO_KEEP_BRANCH_TOGETHER in the error
message of an Exception thrown by the StartGenerator when the TOPO flag
is provided together with the TOPO_KEEP_BRANCH_TOGETHER flag.

Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Id24640dc08e96a196508fe38ce144aa7e035082f
2020-03-12 09:04:36 -07:00
Alex Spradlin e498d43186 RevWalk: new topo sort to not mix lines of history
The topological sort algorithm in TopoSortGenerator for RevWalk may mix
multiple lines of history, producing results that differ from C git's
git-log whose man page states: "Show no parents before all of its
children are shown, and avoid showing commits on multiple lines of
history intermixed." Lines of history are mixed because
TopoSortGenerator merely delays producing a commit until all of its
children have been produced; it does not immediately produce a commit
after its last child has been produced.

Therefore, add a new RevSort option called TOPO_KEEP_BRANCH_TOGETHER
with a new topo sort algorithm in TopoNonIntermixGenerator. In the
Generator, when the last child of a commit has been produced, unpop
that commit so that it will be returned upon the subsequent call to
next(). To avoid producing duplicates, mark commits that have not yet
been produced as TOPO_QUEUED so that when a commit is popped, it is
produced if and only if TOPO_QUEUED is set.

To support nesting with other generators that may produce the same
commit multiple times like DepthGenerator (for example, StartGenerator
does this), do not increment parent inDegree for the same child commit
more than once.

Commit b5e764abd2 modified the existing
TopoSortGenerator to avoid mixing lines of history, but it was reverted
in e40c38ab08 because the new behavior
caused problems for EGit users. This motivated adding a new Generator
for the new behavior.

Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: Icbb24eac98c00e45c175b01e1c8122554f617933
2020-03-11 15:39:38 -07:00
Michael Dardis d7f304045a Add validation to hex decoder
Does not fix any issue but prevents user from shooting themselves in the
foot with improper configuration.

Suggested by Demetr Starshov at https://git.eclipse.org/r/#/c/157681/

Change-Id: I006d65022f0a7d4066970825d00080c59404fdc3
Signed-off-by: Michael Dardis <git@md-5.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-03-04 16:34:01 +01:00
Matthias Sohn 7e3e740cc7 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I79d615dff66493b60d3a4bcbdc57b9455e8d6673
2020-02-01 02:09:37 +01:00
Matthias Sohn 3d59d1b80c Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I73d16b53df02bf735c2431588143efe225a4b5b4
2020-02-01 02:06:47 +01:00
Matthias Sohn 68b0645a2e Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I76a62da98182f0c504b1ea8b7d37cecdf4eea7e0
2020-02-01 01:53:27 +01:00
Matthias Sohn 195b8fc94c Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I12002dbfed9dff14fc6d2df9787d92eab5b1fa78
2020-02-01 01:28:40 +01:00
Matthias Sohn 48dd3385ab Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows
  Replace usage of ArrayIndexOutOfBoundsException in treewalk
  Add config constants for WindowCache configuration options

Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
2020-02-01 01:14:01 +01:00
David Pursehouse f57a7bd230 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix string format parameter for invalidRefAdvertisementLine
  WindowCache: add metric for cached bytes per repository
  pgm daemon: fallback to user and system config if no config specified
  WindowCache: add option to use strong refs to reference ByteWindows

Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-02-01 01:09:46 +01:00
David Pursehouse 7d5300023e Fix string format parameter for invalidRefAdvertisementLine
The externalized error message added in f4fc640 ("BasePackConnection:
Check for expected length of ref advertisement", Dec 18, 2019) uses a
malformed string format. Since there is only one formatting argument,
it should be referenced with '{0}' rather than '{1}'.

Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2020-01-31 08:38:49 +09:00
Thomas Wolf d661b9f43a TLS support on IBM JDKs
SSLContext.getInstance("TLS") by default behaves differently on IBM
JDK than on Oracle or OpenJDK.[1] On IBM JDK one gets sockets that
have only TLSv1 enabled, which makes HTTPS connections fail since most
servers refuse this old protocol version. On Oracle JDK/OpenJDK, one
gets sockets with all available protocol versions enabled.

Explicitly enable all available TLS protocol versions to make
HTTPS connections work also on IBM JDK.

[1] https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.security.component.80.doc/security-component/jsse2Docs/matchsslcontext_tls.html#matchsslcontext_tls

Bug: 558709
Change-Id: I5ffc57a78e67a6239b9dad54840a49a8ed28930a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-01-03 20:48:25 +01:00
Matthias Sohn 57f22d6db1 Externalize error message in FileRepository#convertRefStorage
Change-Id: I5d3acaa883ee7eb8d244b55b3e708cbe018690c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-12-14 22:10:12 +01:00
Thomas Wolf 59f9d206c9 Make blame work correctly on merge conflicts
When a conflicting file was blamed, JGit would not identify lines
coming from the merge parents. The main cause for this was that
Blame and BlameCommand simply added the first DirCacheEntry found
for a file to its queue of candidates (blobs or commits) to consider.
In case of a conflict this typically is the merge base commit, and
comparing a auto-merged contents against that base would yield
incorrect results.

Such cases have to be handled specially. The candidate to be
considered by the blame must use the working tree contents, but
at the same time behave like a merge commit/candidate with HEAD
and the MERGE_HEADs as parents. Canonical git does something very
similar, see [1].

Implement that and add tests.

I first did this for the JGit pgm Blame command. When I then tried
to do the same in BlameCommand, I noticed that the latter also
included some fancy but incomplete CR-LF handling. In order to
be able to use the new BlameGenerator.prepareHead() also in
BlameCommand this CR-LF handling was also moved into BlameGenerator
and corrected in doing so.

(Just considering the git config settings was not good enough,
CR-LF behavior can also be influenced by .gitattributes, and even
by whether the file in the index has CR-LF. To correctly determine
CR-LF handling for check-in one needs to do a TreeWalk with at
least a FileTreeIterator and a DirCacheIterator.)

[1] https://github.com/git/git/blob/v2.22.0/blame.c#L174

Bug: 434330
Change-Id: I9d763dd6ba478b0b6ebf9456049d6301f478ef7c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-11-27 03:03:06 +01:00
Han-Wen Nienhuys 38586d54d0 file: implement FileReftableDatabase
Reftable is a binary, block-based storage format for the ref-database.
It provides several advantages over the traditional packed + loose
storage format:

 * O(1) write performance, even for deletions and transactions.

 * atomic updates to the ref database.

 * O(log N) lookup and prefix scans

 * free from restrictions imposed by the file system: it is
   case-sensitive even on case-insensitive file systems, and has
   no inherent limitations for directory/file conflicts

 * prefix compression reduces space usage for repetitive ref names,
   such as gerrit's refs/changes/xx/xxxxx format.

FileReftableDatabase is based on FileReftableStack, which does
compactions inline. This is simple, and has good median performance,
but every so often it will rewrite the entire ref database.

For testing, a FileReftableTest (mirroring RefUpdateTest) is added to
check for Reftable specific behavior. This must be done separately, as
reflogs have different semantics.

Add a reftable flavor of BatchRefUpdateTest.

Add a FileReftableStackTest to exercise compaction.

Add FileRepository#convertToReftable so existing testdata can be
reused.

CQ: 21007
Change-Id: I1837f268e91c6b446cb0155061727dbaccb714b8
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-16 23:16:01 -08:00
Matthias Sohn 826317942f Merge branch 'stable-5.5'
* stable-5.5:
  BaseReceivePack: Fix the format
  Prepend hostname to subsection used to store file timestamp resolution
  Store filesystem timestamp resolution in extra jgit config
  SystemReader: extract updating config and its parents if outdated

Change-Id: Iecfddce8081303af29badcdcd3d72a0da50c964f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-11 11:07:17 -08:00
Matthias Sohn 838b5a84b5 Store filesystem timestamp resolution in extra jgit config
This avoids polluting hand-crafted user level config with
auto-configured options which might disturb in environments where
the user level config is replicated between different machines.

Add a jgit config as parent of the system level config. Persist
measured timestamp resolutions always in this jgit config and read it
via the user global config. This has the effect that auto-configured
timestamp resolution will be used by default and can be overridden in
either the system level or user level config.

Store the jgit config under the XDG_CONFIG_HOME directory following the
XDG base directory specification [1] in order to ensure that we have
write permissions to persist the file. This has the effect that each OS
user will use its jgit config since they typically use different
XDG_CONFIG_HOME directories.

If the environment variable XDG_CONFIG_HOME is defined the jgit config
file is located at $XDG_CONFIG_HOME/jgit/config otherwise the default is
~/.config/jgit/config.

If you want to avoid redundant measurement for different OS users
manually copy the values measured and auto-configured for one OS user to
the system level git config.

[1] https://wiki.archlinux.org/index.php/XDG_Base_Directory

Bug: 551850
Change-Id: I0022bd40ae62f82e5b964c2ea25822eb55d94687
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-11-09 18:01:19 +01:00
Matthias Sohn 271051ef06 Replace "Rebase interactive" with "Interactive rebase"
The RepositoryState's description is shown in EGit's interactive rebase
view. Use the grammatically correct form.

Bug: 552359
Change-Id: I860ef730a67f2b34d7fabbfbda5ea65ead9270d2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-10-25 17:46:58 +02:00
Thomas Wolf 385b503ae8 Support for core.hooksPath
Support the core.hooksPath git config. This can be an absolute or
relative path of a directory where to find git hooks; a relative
path is resolved relative to the directory the hook will run in.

Bug: 500266
Change-Id: I671999a6386a837e897c31718583c91d8035f3ba
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-10-21 18:05:31 -04:00