Commit Graph

3170 Commits

Author SHA1 Message Date
Shawn Pearce 163be57d0f Merge changes from topic 'add-df'
* changes:
  DirCache: Do not create duplicate tree entries
  DirCacheEditor: Replace file-with-tree and tree-with-file
  AddCommand: Use NameConflictTreeWalk to identify file-dir changes
2016-01-01 12:58:35 -05:00
Shawn Pearce 09500165a8 Fix "remote: Counting objects: ..." formatting
Trailing whitespace is usually removed in properties files so
JGitText did not supply a space between : and the remote message.
Ensure the space exists at runtime by reading the localized string
and appending a space if it is missing.

Messages should be dynamically fetched and not held in a static
class variable, as they can be changed using thread locals.

Change-Id: If6a3707d64094253b1a5304fbfafcf195db7497a
2015-12-31 17:37:16 -08:00
David Ostrovsky 13502fef8f Implement Buck driven build
Today there are plenty of modern build tool systems available in the
wild (in no particular order):

* http://bazel.io
* https://pantsbuild.github.io
* http://shakebuild.com
* https://ninja-build.org
* https://buckbuild.com

The attributes, that all these build tools have in common, are:

* reliable
* correct
* very fast
* reproducible

It must not always be the other build tool, this project is currently
using. Or, quoting Gerrit Code Review maintainer here:

  "Friends, don't let friends use <the other build tool system>!"

This change is non-complete implementation of JGit build in Buck,
needed by Gerrit Code Review to replace its dependency with standlone
JGit cell. This is very useful when a developer is working on both
projects and is trying to integrate changes made in JGit in Gerrit.

The supported workflow is:

  $ cd jgit
  $ emacs <hack>
  $ cd ../gerrit
  $ buck build --config repositories.jgit=../jgit gerrit

With --config repositories.jgit=../jgit jgit cell is routed through
JGit development tree.

To build jgit, issue:

  $ buck build //:jgit
  [-] PROCESSING BUCK FILES...FINISHED 0,0s

Yes, you can't measure no-op build time, given that Buck daemon is
used.

Change-Id: I301a71b19fba35a5093d8cc64d4ba970c2877a44
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2015-12-31 10:08:55 -08:00
Shawn Pearce 2fbd98dbe8 Merge "Fix encoding problem from curl repostory on github" 2015-12-30 23:47:28 -05:00
Shawn Pearce 80edcac06f Merge "Fix hanging fetch via SSH" 2015-12-30 23:27:09 -05:00
Eryk Szymanski 53c3bbe42f Fix encoding problem from curl repostory on github
Pushing curl repository to gerrit fails with a message:
remote: error: internal error while processing changes
java.nio.charset.IllegalCharsetNameException: 'utf8'

curl repository url: https://github.com/bagder/curl.git

To avoid this problem encodingAliases in RawParseUtils have
been extended to contain "'utf8'" (single quoted utf8) string.

Change-Id: I40f613cfdcabf0dc9455bee45116ab8d8c7dd6ee
Signed-off-by: Eryk Szymanski <eryksz@gmail.com>
2015-12-30 23:24:12 -05:00
Matthias Sohn 3fc93f8a56 Rename files using NIO2 atomic rename
Bug: 319233
Change-Id: I5137212f5cd3195a52f90ed5e4ce3cf194a13efd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-30 17:22:33 -05:00
Shawn Pearce e3acf01748 ObjectChecker: use java.text.Normalizer directly
Base Java version for JGit is now Java 7. The java.text.Normalizer
class was available in Java 6. Reflection is no longer required to
normalize strings for Mac OS X.

Change-Id: I98e14b72629a7a729a2d40a3aa275932841274e8
2015-12-30 15:45:13 -05:00
Shawn Pearce 7177b506b4 Merge changes I3db332bf,I614e7110,I37a4ccd6,I86b81e7f,Ic8b362e0, ...
* changes:
  Sort "eager" path-like options to the end of the help
  reset command: provide convenient and meaningful options help
  commit command: allow to specify path(s) argument(s)
  status command: consume more then one argument after --
  repo command: properly name the required 'path' argument
  Un-ignored existing CLI tests which run just fine on Java 7+
  Don't treat command termination due '-h' option as a fatal error
2015-12-30 15:44:30 -05:00
Shawn Pearce 018a91b715 Unify fetch and receive ObjectChecker setup
This avoids duplication of code between receive-pack and fetch-pack paths.
Separate methods are still required to check use of receive.fsckobjects vs.
fetch.fsckobjects, both of which default to transfer.fsckobjects.

Change-Id: I41193e093e981a79fc2f63914e273aaa44b82162
2015-12-29 16:55:41 -08:00
Shawn Pearce 29aa444760 PackWriter: use lib.ObjectIdSet to avoid wrapper
Hoist ObjectIdSet up to lib as part of the public API and add
the interface to some common types like PackIndex and JGit custom
ObjectId map types.  This cleans up wrapper code in a number of
places by allowing direct use of the types as an ObjectIdSet.

Future commits can now rely on ObjectIdSet as a simple read-only
type to check a set of objects from a number of storage options.

Change-Id: Ib62b062421d475bd52abd6c84a73916ef36e084b
2015-12-29 15:32:24 -08:00
Shawn Pearce 683c41af92 DirCache: Do not create duplicate tree entries
If a file (e.g.  "A") and a subtree file (e.g. "A/foo.c") both appear
in the DirCache this cache should not be written out as a tree object.
The "A" file and "A" subtree conflict with each other in the same tree
and will fail fsck.

Detect this condition during DirCacheBuilder and DirCacheEditor
finish() so the application can be halted early before it updates a
DirCache that might later write an invalid tree structure.

Change-Id: I95660787e88df336297949b383f4c5fda52e75f5
2015-12-29 11:33:39 -08:00
Shawn Pearce b71ba69410 DirCacheEditor: Replace file-with-tree and tree-with-file
If a PathEdit tries to store a file where a subtree was, or a subtree
where a file was, replace the entry in the DirCache with the new
name(s).  This supports switching between file and tree entry types
using a DirCacheEditor.

Add new unit tests to cover the conditions where these can happen.

Change-Id: Ie843d9388825f9e3d918a5666aa04e47cd6306e7
2015-12-29 11:33:39 -08:00
Shawn Pearce 3776b14ab4 AddCommand: Use NameConflictTreeWalk to identify file-dir changes
Adding a path that already exists but is changing type such as
from symlink to subdirectory requires a NameConflictTreeWalk to
match up the two different entry types that share the same name.

NameConflictTreeWalk needs a bug fix to pop conflicting entries
when PathFilterGroup aborts the walk early so that it does not
allow DirCacheBuilderIterator to copy conflicting entries into
the output cache.

Change-Id: I61b49cbe949ca8b4b98f9eb6dbe7b1f82eabb724
2015-12-29 11:33:35 -08:00
Andrey Loskutov 0505657d6a commit command: allow to specify path(s) argument(s)
This fixes the command below:

jgit commit a -m "added file a"

which currently fails with:

org.eclipse.jgit.api.errors.JGitInternalException: The combination of
arguments --all and --only is not allowed

Bug: 484973
Change-Id: I37a4ccd68101a66520ef99110f7aa0cbdcc8beba
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-29 15:00:09 +01:00
Andrey Loskutov 241b50be31 Simplify development of commands: added main() to CLIGitCommand
This will execute git commands (with arguments) specified on the command
line, handy for developing/debugging a sequence of arbitrary git
commands working on same repository.

The git working dir path can be specified via Java system property
"git_work_tree". If not specified, current directory will be used.

Change-Id: I621a9ec198c31e28a383818efeb4b3f835ba1d6f
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-28 22:59:59 +01:00
Shawn Pearce ef757a7e12 DirCacheEditor: Cleanup DeleteTree constructor
Neaten up formatting and avoid strings, which prevents the need for
NLS comment tags.  Instead check the last character using char
literal, and append a char literal instead of a string.

Change-Id: Ib68e017769a1f5c03200354a805769d585a48c8b
2015-12-28 11:43:31 -08:00
Shawn Pearce 110d3ca595 DirCacheEditor: Fix formatting to avoid , at start of line
Change-Id: I1b1d614470c67fe4736fdc9c26ae26fb38dd58b5
2015-12-28 11:43:30 -08:00
Shawn Pearce 21cec184b7 Merge "AddCommand: Avoid unnecessary string conversions" 2015-12-28 14:24:33 -05:00
Christian Halstrick 2fdce1ef8c Merge "AddCommand: Cleanup conditional logic" 2015-12-28 08:39:07 -05:00
Shawn Pearce a94e517940 AddCommand: Avoid unnecessary string conversions
Change-Id: I13634caeccd9f675a86adfdfa94099b6fb75463a
2015-12-23 22:26:50 -08:00
Shawn Pearce bb8627bd9b AddCommand: Cleanup conditional logic
Unnest and simplify conditional logic for handling entries.

Change-Id: I3093cab5f0edfaf3efbbd6c644e9c922edc67d38
2015-12-23 20:31:15 -08:00
Dmitry Neverov 9c71bf14b7 Close copy threads in case of errors
Bug: 484775
Change-Id: I3c7105188e615b6b994261f4ece0c8abc98eb444
Signed-off-by: Dmitry Neverov <dmitry.neverov@gmail.com>
2015-12-24 01:20:14 +01:00
Yuxuan 'fishy' Wang d88695e412 Skip nested copyfiles in RepoCommand.
Similar to nested directories, nested copyfiles won't work with git submodule
either.

Change-Id: Idbe965ec20a682fca0432802858162f8238f05de
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2015-12-22 14:05:35 -08:00
Andrey Loskutov 1aafa61958 Checkout should be able to override modified symbolic links
Handle existing symlink as a file, not as directory if deleting a file
before creating (overriding) a symlink.

Bug: 484491
Change-Id: I29dbf57d1daec2ba98454975b093e1d381d05196
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-17 10:42:44 +01:00
Shawn Pearce 8cfbbc9f54 Merge "ProgressSpinner: Simple busy wait entertainment" 2015-12-16 16:20:59 -05:00
Shawn Pearce a38531b21c ProgressSpinner: Simple busy wait entertainment
Keep a user amused while the server does work by spinning a
little ASCII-art object on a single line.

Change-Id: Ie8f181d1aa606d4ae69e5d3ca4db387cea739f38
2015-12-16 15:43:07 -05:00
James Kolb 3096a13340 Included cached deltas in delta packStatistics.
Previously, non-reuse deltas were only included in packStatistics if they
were not cached by the deltaWindow.

Change-Id: I7684d8214875f0a7569b34614f8a3ba341dbde9c
Signed-off-by: James Kolb <jkolb@google.com>
2015-12-16 14:06:16 -05:00
Jonathan Nieder cfa0b2fb3c Merge changes Ib4d53bdd,I55bd512c
* changes:
  Do not let PathFilter.create("a/b") match 'a' unless 'a' is a subtree
  Add tests for PathFilterGroup.Single
2015-12-16 12:09:49 -05:00
Shawn Pearce a3e9876d8f Merge "Fix InterruptTimer leak in BasePackConnection" 2015-12-16 10:29:02 -05:00
Matthias Sohn 6340c76d8b Fix InterruptTimer leak in BasePackConnection
When setting timeout on push, BasePackConnection creates a timer, which
will be terminated when push finishes. But, when using
SmartHttpPushConnection, it dropped the first timer created in the
constructor and then created another timer in doPush. If new threads are
created faster than the gc collects then this may stop the service if
it's hitting the max process limit. Hence don't create a new timer if it
already exists.

Bug: 474947
Change-Id: I6746ffe4584ad919369afd5bdbba66fe736be314
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-16 14:19:38 +01:00
Shawn Pearce 7c5b2761ed Merge "Refspec: loosen restrictions on wildcard "*"" 2015-12-15 23:22:12 -05:00
Shawn Pearce 9eda23e469 Merge "BaseRepositoryBuilder should trim CR from .git symref" 2015-12-15 23:18:14 -05:00
Shawn Pearce 8b749f7de5 Merge "Fix possible arithmetic overflow when setting a timeout" 2015-12-15 23:16:21 -05:00
Shawn Pearce fcd796a9a0 Merge "Accept UTF8 BOM with BlobBasedConfig" 2015-12-15 23:12:56 -05:00
Shawn Pearce 6297c2e92c Merge "Bug 484342: Support @ in username in SSH url." 2015-12-15 23:09:31 -05:00
Jonathan Nieder 75a0dcac18 Do not let PathFilter.create("a/b") match 'a' unless 'a' is a subtree
PathFilter and PathFilterGroup form JGit's implementation of git's
path-limiting feature in commands like log and diff.  To save time
when traversing trees, a path specification

	foo/bar/baz

tells the tree walker not to traverse unrelated trees like qux/.  It
does that by returning false from include when the tree walker is
visiting qux and true when it is visiting foo.

Unfortunately that test was implemented to be slightly over-eager: it
doesn't only return true when asked whether to visit a subtree "foo"
but when asked about a plain file "foo" as well.  As a result, diffs
and logs restricted to some-file/non-existing-suffix unexpectedly
match against some-file:

 $ jgit log -- LICENSE/no-such-file
 commit 629fd0d594
 Author: Shawn O. Pearce <spearce@spearce.org>
 Date:   Fri Jul 02 14:52:49 2010 -0700

     Clean up LICENSE file
[...]

Fix it by checking against the entry's mode.

Gitiles +log has the same bug and benefits from the same fix.

Callers know not to worry about what subtrees are included in the tree
walk because shouldBeRecursive() returns true in this case, so this
behavior change should be safe.  This also better matches the behavior
of C git:

 $ empty=$(git mktree </dev/null)
 $ git diff-tree --abbrev $empty HEAD -- LICENSE/no-such-file
 $ git diff-tree --abbrev $empty HEAD -- tools/no-such-file
 :000000 040000 0000000... b62648d... A  tools

Bug: 484266
Change-Id: Ib4d53bddd8413a9548622c7b25b338d287d8889d
2015-12-15 15:22:10 -08:00
Mike Gilbode 3e03907f65 Bug 484342: Support @ in username in SSH url.
Change-Id: I5795e925afff796488ba26c83694e806b76a374f
Signed-off-by: Mike Gilbode <gilbode@gmail.com>
2015-12-15 10:44:16 -05:00
Matthias Sohn 2647d024af Fix push with jgit pgm failing with "unauthorized"
Pushing with JGit commandline to e.g. Github failed with "unauthorized"
since HttpUrlConnection calls the configured authenticator implicitly.

The problem is that during a push two requests are sent to the server,
first a GET and then a POST (containing the pack data). The first GET
request sent anonymously is rejected with 401 (unauthorized). When an
Authenticator is installed the java.net classes will use the
Authenticator to ask the user for credentials and retry the request.
But this happens under the hood and JGit level code doesn't see that
this happens.

The next request is the POST but since JGit thinks the first GET request
went through anonymously it doesn't add authentication headers to the
POST request. This POST of course also fails with 401 but since this
request contains a lot of body-data streamed from JGit (the pack file!)
the java.net classes can't simply retry the request with authorization
headers. The whole process fails.

Fix this by using Apache httpclient which doesn't use Authenticator to
retrieve credentials. Instead initialize TransportCommand to use the
default credential provider if no other credentials provider was set
explicitly. org.eclipse.jgit.pgm.Main sets this default for the JGit
command line client.

Change-Id: Ic4e0f8b60d4bd6e69d91eae0c7e1b44cdf851b00
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-15 12:49:08 +01:00
Christian Halstrick 5c66ad5684 Merge "NetRCCredentialsProvider should return false if any item is missing" 2015-12-15 06:44:08 -05:00
Christian Halstrick 2f0b675a71 Merge "Fix ChainingCredentialsProvider" 2015-12-15 03:52:09 -05:00
Christian Halstrick 310e858f81 Fix possible arithmetic overflow when setting a timeout
BasePackPushConnection#readStringLongTimeout() was setting a timeout 10
times bigger than some other timeout or the pack transfer time. This
could lead to negative integer values when we hit an arithmetic
overflow. Add a check for this situation and set the timeout to
Integer.MAX_VALUE when overflow happens.

Bug: 484352
CC: Eugene Petrenko <eugene.petrenko@gmail.com>
Change-Id: Ie2a86312c1bcb1ec3e6388fa490ab3c845d41808
2015-12-15 09:44:09 +01:00
Andrey Loskutov 95b36b397b Null-annotated Ref class and fixed related compiler errors
This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.

Change-Id: I24c8a600ec962d61d5f40abf73eac4203e115240
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-15 09:07:05 +01:00
Shawn Pearce 8179185d11 Merge "push: Do not blindly overwrite peer" 2015-12-14 17:45:57 -05:00
Shawn Pearce 4f8c993c49 push: Do not blindly overwrite peer
If an application uses PushConnection directly on the native Git wire
protocols JGit should send along the application's expected oldId, not
the advertised value.  This allows the remote peer to compare-and-swap
since it was not tested inside JGit.

Discovered when I tried to use a PushConnection (bypassing the
standard PushProcess) and the client blindly overwrote the remote
reference, even though my app had supplied the wrong ObjectId for
the expectedOldObjectId. This was not expected and cost me over an
hour of debugging, plus "corruption" in the remote repository.

By passing along the exact expectedOldObjectId from the app the
remote side can do the check that the application skipped, and
avoid data loss.

Change-Id: Id3920837e6c47100376225bb4dd61fa3e88c64db
2015-12-14 09:18:27 -08:00
Christian Halstrick 5b55498b16 Fix FileTreeIterator.idSubmodule(Entry)
FileTreeIterator was calling by mistake
WorkingTreeIterator.idSubmodule(Entry). Instead it should always compute
idSubmodule on its own.

Change-Id: Id1b988aded06939b1d7edd2671e34bf756896c0e
2015-12-13 01:24:46 +01:00
Andrey Loskutov 85d09a9ec7 URIish: fixed full uri pattern not expecting end of line after host name
Bug: 483326
Change-Id: I8b6e3eb648c8ec2c38f73de22382537b1276b779
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-12 00:55:31 +01:00
Kaloyan Raev e23521f0d9 Add remote command to JGit CLI
Supported subcommands are:
 - <none> (lists available remotes)
 - add
 - remove
 - set-url
 - update
 
Supported options are:
 --verbose
 --push
 --prune

Bug: 481316
Change-Id: I57c34ed6daabb7d308bc383b17c1ef4af433e714
Signed-off-by: Kaloyan Raev <kaloyan.r@zend.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-11 22:54:48 +01:00
Doug Kelly 618b0c1ceb Accept UTF8 BOM with BlobBasedConfig
In I1f5dc07182dbf6bba2a9f4807fdd25b475da4ead, FileBasedConfig got
support for reading a configuration with UTF8 BOM.  Apply the same
support to BlobBasedConfig, to make SubmoduleWalk able to parse
.gitmodules configurations with BOM.

Change-Id: I25b5474779952fe2c076180b96fc2869eef190a8
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
2015-12-09 16:48:20 -06:00
Matthias Sohn afd167a1f2 NetRCCredentialsProvider should return false if any item is missing
Change-Id: I894d1621aaccd71dfe100fe83a1bd9d50a1e0808
2015-12-08 14:12:26 +01:00