Commit Graph

1856 Commits

Author SHA1 Message Date
Matthias Sohn 9906f09868 Partially revert c0ad77d8 "Enhance Eclipse save actions"
Revert the following save actions which were introduced in c0ad77d8:
- always use braces around blocks
- remove unused imports

Other than I expected save actions are run globally on edited files -
and not only on edited code lines only.

Hence revert the save action "Convert control statement bodies to
blocks" which would affect a large number of code lines not affected by
the change editing some small part of a class. This would generate a
large number of changes which may lead to many unnecessary conflicts.
Total number of affected lines across jgit would be around 10k lines.

Also revert "Remove unused imports" since it erroneously removes imports
of some annotations needed by pgm classes using args4j.

Change-Id: I879a47f68e664129e6124cf25c1ae1f6a2d7a5aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-30 01:24:47 +02:00
Shawn Pearce d684ade3d3 Merge "reftable: explicitly store update_index per ref" 2017-08-28 17:57:13 -04:00
David Pursehouse 10a8df22fa Merge "Enhance Eclipse save actions" 2017-08-28 12:08:01 -04:00
Terry Parker cb24de07d0 Merge "Add BlobObjectChecker" 2017-08-28 12:00:53 -04:00
Matthias Sohn c0ad77d84c Enhance Eclipse save actions
Add the following Eclipse save actions executed when saving modified
lines. This should help to reduce manual work needed to maintain a clean
and consistent code style:
- organize imports
- always use braces around blocks
- add missing annotations
  - @Override including implementation of interface methods
  - @Deprecated
- remove
  - unused imports
  - unnecessary $NON-NLS$ tags
  - redundant type arguments

Also add default values for new settings that were introduced in recent
Eclipse versions up to Neon since we updated save rules the last time.

Change-Id: Idc90b249df044d0552f04edf01a5f607c4846f50
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-28 11:52:45 -04:00
Masaya Suzuki fd74cf2f78 Add BlobObjectChecker
Some repositories can have a policy that do not accept certain blobs. To
check if the incoming pack file contains such blobs, ObjectChecker can
be used. However, this ObjectChecker is not called by PackParser if the
blob is stored as a whole. This is because the object can be so large
that it doesn't fit in memory.

This change introduces BlobObjectChecker. This interface takes chunks of
a blob instead of the entire object. ObjectChecker can optionally return
a BlobObjectChecker. This won't change existing ObjectChecker
implementation; existing implementation continues to receive deltified
blob objects only.

Change-Id: Ic33a92c2de42bd7a89786a4da26b7a648b25218d
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2017-08-28 08:42:27 -07:00
Thomas Wolf d031b64667 Exclude file matching: fix backtracking on match failures after **
** matching always tries the empty match first. If a mismatch occurs
later, the ** must be extended by exactly one segment and matching must
resume with the matcher following the ** matcher.

Bug: 520920
Change-Id: Id019ad1c773bd645ae92e398021952f8e961f45c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:41 +02:00
Thomas Wolf d80b999c76 Fix path pattern matching to work also for gitattributes
Path pattern matching for attribute rules is different than matching
for excluded files.

The first difference concerns patterns without slashes. For
gitattributes those must match on the last component only, not on
any earlier segment. This is true also for directory-only patterns.

The second difference concerns directory-only patterns. Those also
must not match on a prefix or segment except the last one. They do
not apply recursively to all files beneath.

And third, matches only on a prefix must match for gitattributes
only if the last matcher was "/**".

Add a new parameter for such path matching to IMatcher.matches() and
pass it through as appropriate (false for gitignore, true for
gitattributes). As far as gitignore is concerned, there is no change.

New tests have been added, and some existing attribute matching tests
have been fixed since they operated on wrong assumptions.

Bug: 508568
Change-Id: Ie825dc2cac8a85a72a7eeb0abb888f3193d21dd2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:40 +02:00
Thomas Wolf 08d4bef6b7 Add new tests for gitignore/gitattribute pattern matching
These tests verify that JGit matches the same as C git, for
both attribute matching (.gitattributes) and file exclusion matching
(.gitignore). These tests work by setting up a test repository and
test rules, and then determine excluded files or attributes both with
JGit and with the native C git, and then compare the results.

For .gitignore tests, we run

  git ls-files --ignored --exclude-standard -o
  
and for attribute tests we use

  git check-attr --stdin --all
  
and pass the list of all files in the repository via stdin.

Change-Id: I5b40946e04ff4a97456be7dffe09374323b7c89d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-27 16:02:40 +02:00
Matthias Sohn 7511bc886e Fix unthrown exception errors in CloneCommandTest
The Eclipse compiler raises errors for unthrown exceptions declared to
be thrown by test methods introduced in 88e45399.

Change-Id: I0d91c89e1b20ceff52c38b759abf906cc94e9902
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-26 09:11:02 +02:00
David Pursehouse 40f40e496a Merge "Fix default directory set when setDirectory wasn't called." 2017-08-25 20:57:52 -04:00
Thomas Wolf c758a8cd37 Do most %-token substitutions in OpenSshConfig
Except for %p and %r and partially %C, we can do token substitutions
as defined by OpenSSH inside the config file parser. %p and %r can
be replaced only if specified in the config; if not, it would be the
caller's responsibility to replace them with values obtained from the
URI to connect to.

Jsch doesn't know about token substitutions at all. By doing the
replacements as good as we can in the config file parser, we can
make Jsch support most of these tokens.

%i is not handled at all as Java has no concept of a "user ID".

Includes unit tests.

Bug: 496170
Change-Id: If9d324090707de5d50c740b0d4455aefa8db46ee
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-26 01:44:36 +02:00
Thomas Wolf 9d2447063d Let Jsch know about ~/.ssh/config
Ensure the Jsch instance used knows about ~/.ssh/config. This
enables Jsch to honor more user configurations (see
com.jcraft.jsch.Session.applyConfig()), in particular also the
UserKnownHostsFile configuration, or additional identities given
via multiple IdentityFile entries.

Turn JGit's OpenSshConfig into a full parser that can be a
Jsch-compliant ConfigRepository. This avoids a few bugs
in Jsch's OpenSSHConfig and keeps the JGit-facing interface
unchanged. At the same time we can supply a JGit OpenSshConfig
instance as a ConfigRepository to Jsch. And since they'll both
work from the same object, we can also be sure that the parsing
behavior is identical.

The parser does not handle the "Match" and "Include" keys, and it
doesn't do %-token substitutions (yet).

Note that Jsch doesn't handle multi-valued UserKnownHostFile
entries as known by modern OpenSSH.[1]

[1] http://man.openbsd.org/OpenBSD-current/man5/ssh_config.5

Additional tests for new features are provided in OpenSshConfigTest.

Bug: 490939
Change-Id: Ic683bd412fa8c5632142aebba4a07fad4c64c637
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-26 01:41:50 +02:00
Joan Goyeau 88e453995d Fix default directory set when setDirectory wasn't called.
Bug: 519883
Change-Id: I46716e9626b4c4adc0806a7c8df6914309040b94
Signed-off-by: Joan Goyeau <joan@goyeau.com>
2017-08-25 11:41:40 +01:00
Shawn Pearce 44a75d9ea8 reftable: explicitly store update_index per ref
Add an update_index to every reference in a reftable, storing the
exact transaction that last modified the reference.  This is necessary
to fix some merge race conditions.

Consider updates at T1, T3 are present in two reftables.  Compacting
these will create a table with range [T1,T3].  If T2 arrives during
or after the compaction its impossible for readers to know how to
merge the [T1,T3] table with the T2 table.

With an explicit update_index per reference, MergedReftable is able to
individually sort each reference, merging individual entries at T3
from [T1,T3] ahead of identically named entries appearing in T2.

Change-Id: Ie4065d4176a5a0207dcab9696ae05d086e042140
2017-08-21 15:39:08 -07:00
David Pursehouse 231f5d9baf Merge changes Id3994e2d,I5e2a2868,I255af794
* changes:
  LongObjectIdTest: Add back self comparison test
  Format BUILD files with buildifier
  Bazel: Add missing dependency in org.eclipse.jgit.http.test
2017-08-18 17:05:41 -04:00
David Pursehouse 904e9f8b73 Format BUILD files with buildifier
Change-Id: I5e2a286866b63a8fa2bd29cc2fe432fab2bbe0af
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-08-18 10:01:06 +01:00
Shawn Pearce 0aae64ce74 reftable: resolve symbolic references
resolve(Ref) helps callers recursively chase symbolic references and
is a useful function when wrapping a Reftable inside a RefDatabase, as
RefCursor does not resolve symbolic references during iteration.

Change-Id: I1ba143f403773497972e225dc92c35ecb989e154
2017-08-17 15:06:51 -07:00
Shawn Pearce d48ac5bf01 reftable: compact merged tables
A compaction of reftables is just copying the results of a
MergedReftable into a ReftableWriter.  Wrap this up into a utility.

Change-Id: I6f5677d923e9628993a2d8b4b007a9b8662c9045
2017-08-17 15:06:51 -07:00
Shawn Pearce 77d8eead6d reftable: merge-join reftables
MergedReftable combines multiple reference tables together in a stack,
allowing higher/later tables to shadow earlier/lower tables.  This
forms the basis of a transaction system, where each transaction writes
a new reftable containing only the modified references, and readers
perform a merge on the fly to get the latest value.

Change-Id: Ic2cb750141e8c61a8b2726b2eb95195acb6ddc83
2017-08-17 15:06:51 -07:00
Shawn Pearce 7da2fe6be8 reftable: lookup by ObjectId unit tests
Change-Id: Ic819a04e285094e271435dcd027d8006e5897785
2017-08-17 15:06:51 -07:00
Shawn Pearce e8e8041e83 reftable: reflog unit tests
Change-Id: If719a63ead54ecbcaf7cbe12c71f00435706bc2b
2017-08-17 15:06:51 -07:00
Shawn Pearce da5a27cd4e reftable: namespace unit tests
Add additional test cases for looking up entries within a namespace
such as refs/heads/ or refs/tags/, where the seek is passed a name
that ends with '/'.

Change-Id: I5f944de7518cd0090374bddba48d4dd3955a8d72
2017-08-17 15:06:51 -07:00
Shawn Pearce 9da26a5a1a reftable: bulk operation unit tests
Add more test cases that cover larger collections of
references, verifying every reference is accessible
both by scan and by seek.

Change-Id: Icada59fdcfc92a4634f6df61baaebb1c37b75d98
2017-08-17 15:06:51 -07:00
Shawn Pearce b6e25e5d7e reftable: basic functionality unit tests
This set of tests covers primitive storage of an empty
file, and each type of supported reference.

Change-Id: I3bdff35cae8ae27283051932f20608b3ac353559
2017-08-17 15:06:51 -07:00
Christian Halstrick be767fd7d9 Merge "Fix off-by-one error in Strings.count()" 2017-08-16 06:24:43 -04:00
Christian Halstrick c71af0c73a Merge "Use relative paths for attribute rule matching" 2017-08-16 06:24:33 -04:00
Thomas Wolf b13a285098 Send a detailed event on working tree modifications
Currently there is no way to determine the precise changes done
to the working tree by a JGit command. Only the CheckoutCommand
actually provides access to the lists of modified, deleted, and
to-be-deleted files, but those lists may be inaccurate (since they
are determined up-front before the working tree is modified) if
the actual checkout then fails halfway through. Moreover, other
JGit commands that modify the working tree do not offer any way to
figure out which files were changed.

This poses problems for EGit, which may need to refresh parts of the
Eclipse workspace when JGit has done java.io file operations.

Provide the foundations for better file change tracking: the working
tree is modified exclusively in DirCacheCheckout. Make it emit a new
type of RepositoryEvent that lists all files that were modified or
deleted, even if the checkout failed halfway through. We update the
'updated' and 'removed' lists determined up-front in case of file
system problems to reflect the actual state of changes made.

EGit thus can register a listener for these events and then knows
exactly which parts of the Eclipse workspace may need to be refreshed.

Two commands manage checking out individual DirCacheEntries themselves:
checkout specific paths, and applying a stash with untracked files.
Make those two also emit such a new WorkingTreeModifiedEvent.

Furthermore, merges may modify files, and clean, rm, and stash create
may delete files.

CQ: 13969
Bug: 500106
Change-Id: I7a100aee315791fa1201f43bbad61fbae60b35cb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-15 16:52:00 -04:00
Matthias Sohn 81d020aba9 Merge branch 'stable-4.8'
* stable-4.8:
  Update Oxygen Orbit p2 repository to R20170516192513
  Fix exception handling for opening bitmap index files

Change-Id: Ica20f5aa0d8a365fe3317765b93520b3abd5d342
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-15 00:48:44 +02:00
Matthias Sohn 758a181b82 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Update Oxygen Orbit p2 repository to R20170516192513
  Fix exception handling for opening bitmap index files

Change-Id: I1e4fcf84506ff4316567bbb1713e84d8d196c2a1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-15 00:24:49 +02:00
Matthias Sohn 53becf1f59 Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Update Oxygen Orbit p2 repository to R20170516192513
  Fix exception handling for opening bitmap index files

Change-Id: I669fe48ce0034f9ea1977d38ee39099497422c1c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-14 23:50:52 +02:00
Matthias Sohn 985e3c6414 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Fix exception handling for opening bitmap index files

Change-Id: Ifb511238e3e98b1bc9f79a990807b940a17ebaa6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-14 23:43:05 +02:00
Christian Halstrick 1ed1e40387 Fix exception handling for opening bitmap index files
When creating a new PackFile instance it is specified whether this pack
has an associated bitmap index file or not. This information is cached
and the public method getBitmapIndex() will always assume a bitmap index
file must exist if the cached data tells so. But it may happen that the
packfiles are repacked during a gc in a different process causing the
packfile, bitmap-index and index file to be deleted. Since JGit still
has an open FileHandle on the packfile this file is not really deleted
and can still be accessed. But index and bitmap index file are deleted.
Fix getBitmapIndex() to invalidate the cached packfile instance if such
a situation occurs.

This problem showed up when a gerrit server was serving repositories
which where garbage collected with native git regularly. Fetch and
clone commands for certain repositories failed permanently after a
native git gc had deleted old bitmap index files.

Change-Id: I8e620bec74dd3f310ba42024f9a657062f868f0e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-14 21:09:48 +02:00
Thomas Wolf 37908321c0 Do not apply pushInsteadOf to existing pushUris
Per the git config documentation[1], pushInsteadOf is ignored when
a remote has explicit pushUris.

Implement this, and adapt tests.

Up to now JGit mistakenly applied pushInsteadOf also to existing
pushUris. If some repositories had relied on this mis-feature,
pushes may newly suddenly fail (the uncritical case; the config
just needs to be fixed) or even still succeed but push to unexpected
places, namely to the non-rewritten pushUrls (the critical case).

The release notes should point out this change.

[1] https://git-scm.com/docs/git-config

Bug: 393170
Change-Id: I38c83204d2ac74f88f3d22d0550bf5ff7ee86daf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-14 17:27:05 +02:00
Thomas Wolf df3469f6ad Record submodule paths with untracked changes as FileMode.GITLINK
Bug: 520702
Change-Id: I9bb48af9e8f1f2ce7968a82297c7c16f1237f987
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-14 14:03:51 +02:00
Thomas Wolf f5a2c77dc4 Fix handling of pushInsteadOf
According to [1], pushInsteadOf is

1. applied to the uris, not to the pushUris
2. ignored if a remote has an explicit pushUri

JGit applied it only to the pushUris. As a result, pushInsteadOf was
ignored for remotes having only a uri, but no pushUri.

This commit implements (1) if there are no pushUris. I did not dare
implement (2) because:

* there are explicit tests for it that expect that pushInsteadOf gets
  applied to existing pushUrls, and
* people may actually use and rely on this JGit behavior.

[1] https://git-scm.com/docs/git-config

Bug: 393170
Change-Id: I6dacbf1768a105190c2a8c5272e7880c1c9c943a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-14 05:40:47 -04:00
Christian Halstrick 196915dde5 Merge "Ensure EOL stream type is DIRECT when -text attribute is present" 2017-08-14 03:34:57 -04:00
Thomas Wolf b07db60908 Fix off-by-one error in Strings.count()
Change-Id: I0667b1624827d1cf0cc1b81f86c7bb44eafd68a7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-14 08:04:56 +02:00
Shawn Pearce 53dd9a9e4b Rename extensions.refsStorage to refStorage
This matches the proposal that has been discussed at length on
git-core mailing list and seems to be the accepted convention.

Change-Id: I9f6ab15144826893d1e2a4b48a2d657d6dd445ec
2017-08-11 18:20:50 -07:00
Thomas Wolf a489a8ae9a Ensure EOL stream type is DIRECT when -text attribute is present
Otherwise fancy combinations of attributes (binary or -text in
combination with crlf or eol) may result in the corruption of binary
data.

Bug: 520910
Change-Id: I3ffc666c13d1b9d2ed987b69a67bfc7f42ccdbfc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-11 22:56:50 +02:00
Thomas Wolf 4bc539a814 Use relative paths for attribute rule matching
Attribute rules must match against the entry path relative to the
attribute node containing the rule. The global entry path is to be
used only for the init and the global node (and of course the root
node).

Bug: 520677
Change-Id: I80389a2dc272a72312729ccd5358d7c75e1ea20a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-11 21:59:49 +02:00
Shawn Pearce ed29dec1ea Expose LongMap in util package
This is a useful primitive collection type like IntList.

Change-Id: I04b9b2ba25247df056eb3a1725602f1be6d3b440
2017-08-09 10:42:09 -07:00
Shawn Pearce 40c9c59e07 NB: encode and decode 24-bit ints
Change-Id: Ie036dc46e5a88a4e87dc52e880505bbe34601ca7
2017-08-09 10:42:09 -07:00
Matthias Sohn ba85764223 Remove unused import introduced in a551b64
Change-Id: Ia6c3935cf061590e7305d0a80a1051e9aebcbb43
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-01 23:26:44 +02:00
David Pursehouse 4085646f6d Merge changes I424295df,Ib003f7c8
* changes:
  Treat RawText of binary data as file with one single line.
  Trim boilerplate in RawParseUtils_LineMapTest.
2017-08-01 10:18:48 -04:00
Han-Wen Nienhuys a551b64694 Treat RawText of binary data as file with one single line.
This avoids executing mergeAlgorithm.merge on binary data, which is
unlikely to be useful.

Arguably, binary data should not make it to
ResolveMerger#contentMerge, but this approach has the following
advantages:

* binary detection is exact, since it doesn't only look at the start
  of the blob.

* it is cheap, as we have to iterate over the bytes anyway to find
  '\n'.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I424295df1dc60a719859d9d7c599067891b15792
2017-08-01 16:00:46 +02:00
Terry Parker 8c6a9a286e Merge "Use w1 for hashCode of AbbreviatedObjectId" 2017-07-28 19:24:11 -04:00
David Pursehouse 8391cc233b Merge "IntList: support contains(int)" 2017-07-28 14:18:21 -04:00
Shawn Pearce 4a00f18e8e Use w1 for hashCode of AbbreviatedObjectId
Very short abbreviations that are under 8 hex digits do not
have values in w2. Use w1 as the Java hashCode() instead, so
that the prefix of the abbreviation is always included in the
hashing function used by any java.util.Collection type.

Change-Id: Idaf69f86b62630ba4a022d31b4c293c6d138f557
2017-07-28 10:20:45 -07:00
Shawn Pearce 652a6b0334 IntList: support contains(int)
LongList supports contains(long).
IntList should also support contains(int).

Change-Id: Ic7a81c3c25b0f10d92087b56e9f200b676060f63
2017-07-28 10:18:27 -07:00