Commit Graph

5267 Commits

Author SHA1 Message Date
Robin Stocker c02c9e8042 Fix compilation errors with args4j 2.0.23 and later
The multiValued attribute on @Option was removed. When the field is a
List, it's not actually needed (even with earlier versions of args4j),
see RmTest. In other cases, we have a custom handler, where it's also
not needed.

Bug: 413163
Change-Id: I4bb951e9fab5f4ae4271bd7e11be799dc234ab80
2017-08-30 03:07:35 +02:00
Matthias Sohn f4c5a54aa9 Cleanup: Remove redundant type arguments
Change-Id: I81ce2e25666f5acfce4d3ada6b71ae8c81a97a4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 03:07:34 +02:00
Matthias Sohn e271e2f688 Cleanup: Remove unnecessary $NON-NLS$ tags
Change-Id: I143c9d62b89322509123d4e06a221dd4dc158c55
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 03:07:34 +02:00
Matthias Sohn e9fb111182 Cleanup: Organize imports
Change-Id: I6065e59519bc42bd18f5cc5ee3ec5210764ab03c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 03:07:33 +02:00
Matthias Sohn 41baa7a791 Partially revert c0ad77d8 "Enhance Eclipse save actions"
Do not automatically organize imports using a save action since this
seems to be buggy and removed some annotations org.eclipse.jgit.pgm
needs to use args4j.

Change-Id: I5a91292c3b9241ce2dde3e4ecce14ad460097129
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 03:07:18 +02:00
Matthias Sohn 1052678372 Cleanup: Remove trailing whitespace
Change-Id: I7c08b60fb6afafd1874210320da3da7519632e08
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 01:24:48 +02:00
Matthias Sohn 9906f09868 Partially revert c0ad77d8 "Enhance Eclipse save actions"
Revert the following save actions which were introduced in c0ad77d8:
- always use braces around blocks
- remove unused imports

Other than I expected save actions are run globally on edited files -
and not only on edited code lines only.

Hence revert the save action "Convert control statement bodies to
blocks" which would affect a large number of code lines not affected by
the change editing some small part of a class. This would generate a
large number of changes which may lead to many unnecessary conflicts.
Total number of affected lines across jgit would be around 10k lines.

Also revert "Remove unused imports" since it erroneously removes imports
of some annotations needed by pgm classes using args4j.

Change-Id: I879a47f68e664129e6124cf25c1ae1f6a2d7a5aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 01:24:47 +02:00
Thomas Wolf dbef8e2537 Pass along the original exception when an ssh connection fails
Otherwise, the stack trace doesn't really tell anything.
See for instance [1].

[1] https://www.eclipse.org/forums/index.php/t/1088535/

Change-Id: If22f2c63c36fec6b32818d2c2acecf20531b4185
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-29 15:08:08 -04:00
Shawn Pearce d684ade3d3 Merge "reftable: explicitly store update_index per ref" 2017-08-28 17:57:13 -04:00
David Pursehouse 10a8df22fa Merge "Enhance Eclipse save actions" 2017-08-28 12:08:01 -04:00
Terry Parker cb24de07d0 Merge "Add BlobObjectChecker" 2017-08-28 12:00:53 -04:00
Matthias Sohn c0ad77d84c Enhance Eclipse save actions
Add the following Eclipse save actions executed when saving modified
lines. This should help to reduce manual work needed to maintain a clean
and consistent code style:
- organize imports
- always use braces around blocks
- add missing annotations
  - @Override including implementation of interface methods
  - @Deprecated
- remove
  - unused imports
  - unnecessary $NON-NLS$ tags
  - redundant type arguments

Also add default values for new settings that were introduced in recent
Eclipse versions up to Neon since we updated save rules the last time.

Change-Id: Idc90b249df044d0552f04edf01a5f607c4846f50
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-28 11:52:45 -04:00
Masaya Suzuki fd74cf2f78 Add BlobObjectChecker
Some repositories can have a policy that do not accept certain blobs. To
check if the incoming pack file contains such blobs, ObjectChecker can
be used. However, this ObjectChecker is not called by PackParser if the
blob is stored as a whole. This is because the object can be so large
that it doesn't fit in memory.

This change introduces BlobObjectChecker. This interface takes chunks of
a blob instead of the entire object. ObjectChecker can optionally return
a BlobObjectChecker. This won't change existing ObjectChecker
implementation; existing implementation continues to receive deltified
blob objects only.

Change-Id: Ic33a92c2de42bd7a89786a4da26b7a648b25218d
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2017-08-28 08:42:27 -07:00
Thomas Wolf 1637c44048 FetchCommand: pass on CredentialsProvider to submodule fetches
When a JGit API command is implemented in terms of other API
commands, the child command must "inherit" all relevant settings.
Calling configure() ensures that the CredentialsProvider and the
connection timeout are propagated correctly.

Bug: 515325
Change-Id: I948e306693a9edb7b199a735877413b6eddcfba4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:37:43 +02:00
Thomas Wolf d031b64667 Exclude file matching: fix backtracking on match failures after **
** matching always tries the empty match first. If a mismatch occurs
later, the ** must be extended by exactly one segment and matching must
resume with the matcher following the ** matcher.

Bug: 520920
Change-Id: Id019ad1c773bd645ae92e398021952f8e961f45c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:41 +02:00
Thomas Wolf d80b999c76 Fix path pattern matching to work also for gitattributes
Path pattern matching for attribute rules is different than matching
for excluded files.

The first difference concerns patterns without slashes. For
gitattributes those must match on the last component only, not on
any earlier segment. This is true also for directory-only patterns.

The second difference concerns directory-only patterns. Those also
must not match on a prefix or segment except the last one. They do
not apply recursively to all files beneath.

And third, matches only on a prefix must match for gitattributes
only if the last matcher was "/**".

Add a new parameter for such path matching to IMatcher.matches() and
pass it through as appropriate (false for gitignore, true for
gitattributes). As far as gitignore is concerned, there is no change.

New tests have been added, and some existing attribute matching tests
have been fixed since they operated on wrong assumptions.

Bug: 508568
Change-Id: Ie825dc2cac8a85a72a7eeb0abb888f3193d21dd2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:40 +02:00
Thomas Wolf 08d4bef6b7 Add new tests for gitignore/gitattribute pattern matching
These tests verify that JGit matches the same as C git, for
both attribute matching (.gitattributes) and file exclusion matching
(.gitignore). These tests work by setting up a test repository and
test rules, and then determine excluded files or attributes both with
JGit and with the native C git, and then compare the results.

For .gitignore tests, we run

  git ls-files --ignored --exclude-standard -o
  
and for attribute tests we use

  git check-attr --stdin --all
  
and pass the list of all files in the repository via stdin.

Change-Id: I5b40946e04ff4a97456be7dffe09374323b7c89d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:40 +02:00
Thomas Wolf 426caf99ee Ignore invalid TagOpt values
C git silently ignores invalid tagopt values; so make JGit behave the
same way.

Bug: 429625
Change-Id: I99587cc46c7e0c19348bcc63f602038fa9a7f378
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-26 09:11:03 +02:00
Thomas Wolf 8cbdf523cd Add a getter for a list of RefSpecs to Config
Reading RefSpecs from a Config can be seen as another typed value
conversion, so add a getter to Config and to TypedConfigGetter. Use
it in RemoteConfig.

Doing this allows clients of the JGit library to customize the
handling of invalid RefSpecs in git config files by installing a
custom TypedConfigGetter.

Bug: 517314
Change-Id: I0ebc0f073fabc85c2a693b43f5ba5962d8a795ff
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-26 09:11:02 +02:00
Thomas Wolf d32ad1cadd Improve getting typed values from a Config
Make the handling of typed values somewhat configurable by using
a separate converter. The default converter is the same as before;
just the implementations of the getters were moved. They also still
raise IllegalArgumentException on invalid values as before.

The converter can be set globally via Config.setTypedConfigGetter(),
which EGit can use in its core Activator to plug in a variant that
catches the IllegalArgumentException, logs the problem, and then
returns the default value.

In this way the behavior for other users of the JGit library is
unchanged, while EGit can deal gracefully with invalid git configs.

Bug: 520978
Change-Id: Ie8f81d206e358b6cc57aa29b9d7ad2a5d34b86a1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-26 09:11:02 +02:00
Matthias Sohn 7511bc886e Fix unthrown exception errors in CloneCommandTest
The Eclipse compiler raises errors for unthrown exceptions declared to
be thrown by test methods introduced in 88e45399.

Change-Id: I0d91c89e1b20ceff52c38b759abf906cc94e9902
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-26 09:11:02 +02:00
David Pursehouse e237c28936 Merge "Fix JGit set core.fileMode to false by default instead of true for non Windows OS." 2017-08-25 20:58:07 -04:00
David Pursehouse 40f40e496a Merge "Fix default directory set when setDirectory wasn't called." 2017-08-25 20:57:52 -04:00
David Pursehouse 0e12692d8c FileMode: Remove unnecessary @SuppressWarnings("synthetic-access")
In Eclipse Oxygen, the following warning is emitted:

  At least one of the problems in category 'synthetic-access' is not
  analysed due to a compiler option being ignored

Removing the suppression gets rid of the warning.

Change-Id: Ibfe5cc1e347150b699f54e2f204ab5ee770da202
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-25 20:57:11 -04:00
Matthias Sohn d979dfd00c Add toString() methods to OpenSshConfig to help debugging
Change-Id: I81b60a13a97e78d5ccd593ba8e4aa614df19f925
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-26 01:44:37 +02:00
Thomas Wolf c758a8cd37 Do most %-token substitutions in OpenSshConfig
Except for %p and %r and partially %C, we can do token substitutions
as defined by OpenSSH inside the config file parser. %p and %r can
be replaced only if specified in the config; if not, it would be the
caller's responsibility to replace them with values obtained from the
URI to connect to.

Jsch doesn't know about token substitutions at all. By doing the
replacements as good as we can in the config file parser, we can
make Jsch support most of these tokens.

%i is not handled at all as Java has no concept of a "user ID".

Includes unit tests.

Bug: 496170
Change-Id: If9d324090707de5d50c740b0d4455aefa8db46ee
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-26 01:44:36 +02:00
Thomas Wolf 9d2447063d Let Jsch know about ~/.ssh/config
Ensure the Jsch instance used knows about ~/.ssh/config. This
enables Jsch to honor more user configurations (see
com.jcraft.jsch.Session.applyConfig()), in particular also the
UserKnownHostsFile configuration, or additional identities given
via multiple IdentityFile entries.

Turn JGit's OpenSshConfig into a full parser that can be a
Jsch-compliant ConfigRepository. This avoids a few bugs
in Jsch's OpenSSHConfig and keeps the JGit-facing interface
unchanged. At the same time we can supply a JGit OpenSshConfig
instance as a ConfigRepository to Jsch. And since they'll both
work from the same object, we can also be sure that the parsing
behavior is identical.

The parser does not handle the "Match" and "Include" keys, and it
doesn't do %-token substitutions (yet).

Note that Jsch doesn't handle multi-valued UserKnownHostFile
entries as known by modern OpenSSH.[1]

[1] http://man.openbsd.org/OpenBSD-current/man5/ssh_config.5

Additional tests for new features are provided in OpenSshConfigTest.

Bug: 490939
Change-Id: Ic683bd412fa8c5632142aebba4a07fad4c64c637
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-26 01:41:50 +02:00
Jonathan Nieder 7e82be66cd Merge "Consume request body before flushing the buffer" 2017-08-25 19:31:05 -04:00
Masaya Suzuki e8d4f2598b Show error message for non-git clients
Change-Id: I56435d955348eb9acef6603d868f9add1c8781c9
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2017-08-25 15:52:59 -07:00
Masaya Suzuki 9fb6561e7a Consume request body before flushing the buffer
This is continuation from https://git.eclipse.org/r/#/c/94249/. When an
error happens, we might not read the entire stream. Consume the request
body before we flush the buffer.

Change-Id: Ia473a04ace600653b2d1f2822e3023570d992410
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2017-08-25 15:23:20 -07:00
Joan Goyeau 88e453995d Fix default directory set when setDirectory wasn't called.
Bug: 519883
Change-Id: I46716e9626b4c4adc0806a7c8df6914309040b94
Signed-off-by: Joan Goyeau <joan@goyeau.com>
2017-08-25 11:41:40 +01:00
David Pursehouse 65b2d0b2d9 ObjectToPack: Add missing @Override annotation
Change-Id: I65ed7b89312d58ea816b46d27707ff907df1c78b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-24 16:20:11 +09:00
Thomas Wolf 1b4daa2994 Cleanup: message reporting for HTTP redirect handling
The addition of "tooManyRedirects" in commit 7ac1bfc ("Do
authentication re-tries on HTTP POST") was an error I didn't
catch after rebasing that change. That message had been renamed
in the earlier commit e17bfc9 ("Add support to follow HTTP
redirects") to "redirectLimitExceeded".

Also make sure we always use the TransportException(URIish, ...)
constructor; it'll prefix the message given with the sanitized URI.
Change messages to remove the explicit mention of that URI inside the
message. Adapt tests that check the expected exception message text.

For the info logging of redirects, remove a potentially present
password component in the URI to avoid leaking it into the log.

Change-Id: I517112404757a9a947e92aaace743c6541dce6aa
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-23 12:20:55 +02:00
Thomas Wolf 7ac1bfc834 Do authentication re-tries on HTTP POST
There is at least one git server out there (GOGS) that does
not require authentication on the initial GET for
info/refs?service=git-receive-pack but that _does_ require
authentication for the subsequent POST to actually do the push.

This occurs on GOGS with public repositories; for private
repositories it wants authentication up front.

Handle this behavior by adding 401 handling to our POST request.
Note that this is suboptimal; we'll re-send the push data at
least twice if an authentication failure on POST occurs. It
would be much better if the server required authentication
up-front in the GET request.

Added authentication unit tests (using BASIC auth) to the
SmartClientSmartServerTest:

- clone with authentication
- clone with authentication but lacking CredentialsProvider
- clone with authentication and wrong password
- clone with authentication after redirect
- clone with authentication only on POST, but not on GET

Also tested manually in the wild using repositories at try.gogs.io.
That server offers only BASIC auth, so the other paths
(DIGEST, NEGOTIATE, fall back from DIGEST to BASIC) are untested
and I have no way to test them.

* public repository: GET unauthenticated, POST authenticated
  Also tested after clearing the credentials and then entering a
  wrong password: correctly asks three times during the HTTP
  POST for user name and password, then gives up.
* private repository: authentication already on GET; then gets
  applied correctly initially to the POST request, which succeeds.

Also fix the authentication to use the credentials for the redirected
URI if redirects had occurred. We must not present the credentials
for the original URI in that case. Consider a malicious redirect A->B:
this would allow server B to harvest the user credentials for server
A. The unit test for authentication after a redirect also tests for
this.

Bug: 513043
Change-Id: I97ee5058569efa1545a6c6f6edfd2b357c40592a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-22 23:57:09 +02:00
Shawn Pearce 44a75d9ea8 reftable: explicitly store update_index per ref
Add an update_index to every reference in a reftable, storing the
exact transaction that last modified the reference.  This is necessary
to fix some merge race conditions.

Consider updates at T1, T3 are present in two reftables.  Compacting
these will create a table with range [T1,T3].  If T2 arrives during
or after the compaction its impossible for readers to know how to
merge the [T1,T3] table with the T2 table.

With an explicit update_index per reference, MergedReftable is able to
individually sort each reference, merging individual entries at T3
from [T1,T3] ahead of identically named entries appearing in T2.

Change-Id: Ie4065d4176a5a0207dcab9696ae05d086e042140
2017-08-21 15:39:08 -07:00
David Pursehouse 231f5d9baf Merge changes Id3994e2d,I5e2a2868,I255af794
* changes:
  LongObjectIdTest: Add back self comparison test
  Format BUILD files with buildifier
  Bazel: Add missing dependency in org.eclipse.jgit.http.test
2017-08-18 17:05:41 -04:00
David Pursehouse f79911e4a2 LongObjectIdTest: Add back self comparison test
The test was removed in 4886621 to prevent a warning from
error-prone.

Add it back but rewrite it in a way that does not cause the
warning.

This reverts commit 4886621261.

Change-Id: Id3994e2d882a9d08bf548b7778406f8a80fbf830
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-18 10:01:06 +01:00
David Pursehouse 904e9f8b73 Format BUILD files with buildifier
Change-Id: I5e2a286866b63a8fa2bd29cc2fe432fab2bbe0af
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-18 10:01:06 +01:00
David Pursehouse 723d67b83e Bazel: Add missing dependency in org.eclipse.jgit.http.test
Change-Id: I255af794856371fdf1a1eceb6bca50a35b71b519
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-18 10:01:06 +01:00
Shawn Pearce 2d76df2442 reftable: reserve standard PackExt
Reserve "ref" extension for reftable files.  This allows them to be
used in a DFS repository as a stream in a DfsPackDescription.

Change-Id: Ife781bb64d0bb063333183ad2be70a41a2482513
2017-08-17 15:06:51 -07:00
Shawn Pearce 0aae64ce74 reftable: resolve symbolic references
resolve(Ref) helps callers recursively chase symbolic references and
is a useful function when wrapping a Reftable inside a RefDatabase, as
RefCursor does not resolve symbolic references during iteration.

Change-Id: I1ba143f403773497972e225dc92c35ecb989e154
2017-08-17 15:06:51 -07:00
Shawn Pearce 195541dd30 reftable: support threshold based compaction
Transactions may wish to merge several tables together as part of an
operation.  Setting a byte limit allows the transaction to consider
only some recent tables, bounding the cost of the compaction.

Change-Id: If037f2cbdc174ff1a215d5917178b33cde4ddaba
2017-08-17 15:06:51 -07:00
Shawn Pearce d48ac5bf01 reftable: compact merged tables
A compaction of reftables is just copying the results of a
MergedReftable into a ReftableWriter.  Wrap this up into a utility.

Change-Id: I6f5677d923e9628993a2d8b4b007a9b8662c9045
2017-08-17 15:06:51 -07:00
Shawn Pearce 77d8eead6d reftable: merge-join reftables
MergedReftable combines multiple reference tables together in a stack,
allowing higher/later tables to shadow earlier/lower tables.  This
forms the basis of a transaction system, where each transaction writes
a new reftable containing only the modified references, and readers
perform a merge on the fly to get the latest value.

Change-Id: Ic2cb750141e8c61a8b2726b2eb95195acb6ddc83
2017-08-17 15:06:51 -07:00
Shawn Pearce 7da2fe6be8 reftable: lookup by ObjectId unit tests
Change-Id: Ic819a04e285094e271435dcd027d8006e5897785
2017-08-17 15:06:51 -07:00
Shawn Pearce e8e8041e83 reftable: reflog unit tests
Change-Id: If719a63ead54ecbcaf7cbe12c71f00435706bc2b
2017-08-17 15:06:51 -07:00
Shawn Pearce da5a27cd4e reftable: namespace unit tests
Add additional test cases for looking up entries within a namespace
such as refs/heads/ or refs/tags/, where the seek is passed a name
that ends with '/'.

Change-Id: I5f944de7518cd0090374bddba48d4dd3955a8d72
2017-08-17 15:06:51 -07:00
Shawn Pearce 9da26a5a1a reftable: bulk operation unit tests
Add more test cases that cover larger collections of
references, verifying every reference is accessible
both by scan and by seek.

Change-Id: Icada59fdcfc92a4634f6df61baaebb1c37b75d98
2017-08-17 15:06:51 -07:00
Shawn Pearce b6e25e5d7e reftable: basic functionality unit tests
This set of tests covers primitive storage of an empty
file, and each type of supported reference.

Change-Id: I3bdff35cae8ae27283051932f20608b3ac353559
2017-08-17 15:06:51 -07:00
Shawn Pearce 0398f3dd6e reftable: debug tools
Simple debug programs to experiment with the reftable file format:

  debug-read-reftable
  debug-write-reftable
  debug-verify-reftable
  debug-benchmark-reftable

Change-Id: I79db351d86900f1e58b17e922e195dff06ee71f1
2017-08-17 15:06:51 -07:00