Commit Graph

3449 Commits

Author SHA1 Message Date
Jonathan Nieder e89393ba3e Make BaseReceivePack.setAtomic public
This makes atomic more consistent with other properties like
allowNonFastForwards.

Change-Id: Ie3b41f14ffba1d0b206ffecc697377417daa6973
2016-05-04 15:04:55 -07:00
Jonathan Nieder 73b3c9c1d8 ReceivePack: Pass atomic setting from client to BatchRefUpdate
Change-Id: I0e4e6cef693e8138562f5ca345e0db61a8abef9c
2016-05-04 14:00:42 -07:00
Matthias Sohn b4c6a41adc Prepare 4.4.0-SNAPSHOT builds
Change-Id: Ib95be1300b1b95cc7e8153170d509282bdc4e328
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-05-04 22:00:58 +02:00
Matthias Sohn 1447159b92 JGit v4.4.0.201605041135-m1
Change-Id: Ic9b6d7dae293887c3ac3a11f92e153501abad1dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-05-04 17:35:15 +02:00
Dave Borowitz c9b00288fd PersonIdent: Make appendSanitized public
Change-Id: I337c90fff3c013819a42a7622ac6bacc72828c07
2016-05-03 14:40:15 -07:00
Stefan Beller 0406290eb4 Config: Improve documentation for setStringList
Change-Id: Idccfa0dfd6db324236f7cb26ab28bdbd80581a8b
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-05-03 11:22:11 -07:00
Dave Borowitz 125a8959d2 PersonIdent: Strip some special chars from external strings
The special characters <> and '\n' interfere with parsing of
identities. C git strips these special characters, so we should too.

Rather than allocating extra strings by calling String#trim(), add a
few lines to our sanitization method to perform the same trimming as
described in String's Javadoc.

Change-Id: I96edcb93a2fc194ee354d60566d352299742a52f
2016-04-29 11:27:14 -04:00
Dave Borowitz 773f9661d0 PersonIdent: Document that name and email aren't trimmed
This might be somewhat surprising behavior to users who might
naturally assume the following invariant:
  ident.equals(parseIdent(ident.toExternalString()))

This invariant does not hold since whitespace is only trimmed during
serialization. We don't want to mess with the strings during
initialization, as this is called during the highly-optimized commit
parsing codepath.

Change-Id: I081a603f0ac0e33167462244779b0ff3ad51e80c
2016-04-29 11:27:06 -04:00
Christian Halstrick ddc0e9e84a Fix possible NPEs when reporting transport errors
There was a bug in JGit which caused NPEs being thrown when Transport
errors should be reported. Avoid the NPE to let the original error show
up.

Change-Id: I9e1e2b0195bd61b7e531a09d0fc7bce109bd6515
2016-04-27 10:07:51 +02:00
Dave Borowitz adff322a69 Expose the ObjectInserter that created an ObjectReader
We've found in Gerrit Code Review that it is common to pass around
both an ObjectReader (or more commonly a RevWalk wrapping one) and an
ObjectInserter. These code paths often assume that the ObjectReader
can read back any objects created by the ObjectInserter without
flushing. However, we previously had no way to enforce that constraint
programmatically, leading to hard-to-spot problems.

Provide a solution by exposing the ObjectInserter that created an
ObjectReader, when known. Callers can either continue passing both
objects and check:
  reader.getCreatedFromInserter() == inserter
or they can just pass around ObjectReader and extract the inserter
when it's needed (checking that it's not null at usage time).

Change-Id: Ibbf5d1968b506f6b47030ab1b046ffccb47352ea
2016-04-26 17:21:37 -04:00
Dave Borowitz 8a26d0577f Add ObjectReader.Filter, like in ObjectInserter
Change-Id: If34ad8185dc8192435e622995fdca598b86add7e
2016-04-25 13:36:38 -04:00
Christian Halstrick 22d7ec2971 Fix calling of clean/smudge filters from Checkout,MergeCommands
When CheckoutCommand or MergeCommand is called then not in all situation
the treewalks have been prepared to support clean/smudge filters. Fix
this

Bug: 491505
Change-Id: Iab5608049221c46d06812552ab97299e44d59e64
2016-04-22 09:32:34 +02:00
Dave Borowitz a37d85ccd6 Support per-BatchRefUpdate atomic transactions
Repurpose RefDatabase#performsAtomicTransactions() slightly, to
indicate that the backend _supports_ atomic transactions, rather than
the current definition, which is that the backend always _uses_ atomic
transactions regardless of whether or not the caller actually wants
them. Allow BatchRefUpdate callers to turn off atomic transactions by
calling setAtomic(false). Defaulting to true means this is backwards
compatible.

Change-Id: I6df78d7df65ab147b4cce7764bd3101db985491c
2016-04-19 10:01:57 -04:00
Jon Schneider 80fa5f39f9 Fix ApplyCommand when result of patch is an empty file
Such hunks are identifiable by a zero value for "new start line". Prior
to the fix, JGit throws and ArrayIndexOutOfBoundsException on such
patches.

Change-Id: I4f3deb5e5f41a08af965fcc178d678c77270cddb
Signed-off-by: Jonathan Schneider <jkschneider@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-16 17:28:06 -05:00
Mike Edgar 4812fdab61 Make UploadPack observe exceptions reading refs
Now if refs are unreadable when serving an upload pack the handler
will fail due to the actual underlying failure. Previously all wants
would be rejected as invalid because Repository.getAllRefs() returned
an empty map.

Testing this required a new subclass of InMemoryRepository so that
an IOException could be injected at the correct time.

Signed-off-by: Michael Edgar <adgar@google.com>
Change-Id: Iac708b1db9d0ccce08c4ef5ace599ea0b57afdc0
2016-04-14 20:41:53 -04:00
Ned Twigg 32a5993227 Expose conflicting files in CheckoutConflictException
Change-Id: I5b3b7b0633354d5ccf0c6c320c0df9c93fdf8eeb
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-11 22:55:16 +02:00
Matthias Sohn eeeca8042f Merge "Fix CommitCommand to be able to skip writing to RefLog" 2016-04-11 16:50:11 -04:00
Christian Halstrick 36a53d1a3c Fix CommitCommand to be able to skip writing to RefLog
CommitCommand already provided a method to set the comment which should
be written into the reflog. The underlying RefUpdate class supported to
skip writing a reflog entry. But through the CommitCommand API it was
not possible to prevent writing a reflog entry. Fix this and allow
creating commits which don't occur in the reflog.

Change-Id: I193c53de71fb5958ea749c4bfa8360a51acc9b58
2016-04-11 08:58:48 +02:00
Matthias Sohn ccf8363693 Prepare 4.4.0-SNAPSHOT builds
Change-Id: If3162f4cc4ae6319b9f1e3293549485b039cfe7f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 17:59:59 +02:00
Matthias Sohn ac5d1bc37b Prepare 4.3.1-SNAPSHOT versions
Change-Id: I985828fd379e4596ae7f1c30a87e55df06a7924e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 17:24:30 +02:00
Matthias Sohn 08fc807780 Merge "Prepare 4.3.1-SNAPSHOT versions" 2016-04-08 11:01:18 -04:00
Terry Parker 2708b11b6c Filter corrupt objects from DfsReader.selectObjectRepresentation()
PackWriter.writeObject() can get into an infinite loop when corrupt
packs are present. When it finds a pack file with an object that can be
reused it calls DfsPackFile.copyAsIs(). If that method sees an invalid
CRC, it adds the object to the DfsPackFile's corrupt object list and
throws a CorruptObjectException, which it later catches as an
IOException and wraps in a
StoredObjectRepresentationNotAvailableException.
PackWriter.writeObjectImpl() catches that SORNAE and retries the
operation by calling DfsReader.selectObjectRepresentation(). But
currently that method returns the same object which was just seen to
be corrupt.

Change DfsPackFile.isCorrupt() from private to package private, and use
that method in DfsReader.selectObjectRepresentation() to filter out
corrupt objects.

The stack traces that show the problem are:
  org.eclipse.jgit.errors.CorruptObjectException.<init>(CorruptObjectException.java:113)
  org.eclipse.jgit.internal.storage.dfs.DfsPackFile.copyAsIs(DfsPackFile.java:624)
  org.eclipse.jgit.internal.storage.dfs.DfsReader.copyObjectAsIs(DfsReader.java:491)
  org.eclipse.jgit.internal.storage.pack.PackWriter.writeObjectImpl(PackWriter.java:1478)
  org.eclipse.jgit.internal.storage.pack.PackWriter.writeObject(PackWriter.java:1455)

  org.eclipse.jgit.internal.storage.dfs.DfsPackFile.getPackIndex(DfsPackFile.java:228)
  org.eclipse.jgit.internal.storage.dfs.DfsReader.findAllFromPack(DfsReader.java:476)
  org.eclipse.jgit.internal.storage.dfs.DfsReader.selectObjectRepresentation(DfsReader.java:455)
  org.eclipse.jgit.internal.storage.pack.PackWriter.writeObjectImpl(PackWriter.java:1492)
  org.eclipse.jgit.internal.storage.pack.PackWriter.writeObject(PackWriter.java:1455)

Change-Id: Iad7bbcaed1f11a6aa3b4f5af911a73a34c0fabfd
Signed-off-by: Terry Parker <tparker@google.com>
2016-04-07 16:25:05 -07:00
Matthias Sohn f2ee537d55 Prepare 4.3.1-SNAPSHOT versions
Change-Id: I52c98ba8fb3a303269a1f9380af114b6dd8c5009
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 00:56:43 +02:00
Matthias Sohn 4f0daa3bb2 JGit v4.3.0.201604071810-r
Change-Id: I902cdf1ff92ce8c6e9d80c4965d8d5bd8b9ac6c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 00:10:10 +02:00
Matthias Sohn 57e6274bcf JGit v4.3.0.201604071045-r
Change-Id: Iafab78d6be34d31a13f979b7be67611135c0f8bd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-07 16:41:24 +02:00
Matthias Sohn 27d06cb8d5 Merge branch 'master' into stable-4.3
* master:
  Remove repository from cache when it's closed
  Fix RefDirectory not closing resources
  Fix repository cache never closing repository

Change-Id: I9dc9d017806cba25125f69b53812cc3e062ef975
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-07 13:06:23 +02:00
Hugo Arès 7d2b3b9e25 Remove repository from cache when it's closed
RepositoryCache has 2 methods to remove a repository from the cache but
they are never called when a repository is closed. Users of the cache
were expected to call one of those 2 methods but how could they have
called them at proper time without having visibility of the repository
usage count.

Ideally, I would have reworked the RepositoryCache to wrap any
repository it opens in a class that would be responsible to unregister
them from the cache when it's really closed, i.e. when usage counter
reaches 0. The problem preventing the wrapping solution is the
RepositoryCache.register method that allows to register an already
opened repository in the cache. Such repositories cannot be wrapped
because callers are still holding a reference on the unwrapped
repository.

Document that RepositoryCache.close method is removing the repository
from the cache as well as closing it and rework
RepositoryCache.unregister method to only remove the repository from the
cache. Use the latter to unregister repository when Repository.doClose
is getting executed.

Change-Id: Ia364816e4da8d7b6cfa72f10758ca31aa8a1f9db
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-07 13:00:50 +02:00
Hugo Arès 53ea86cd7b Fix RefDirectory not closing resources
When repositories are opened using the RepositoryCache, they are kept in
memory and when the repository usage counter reaches 0, the
Repository.close method is called which then calls close method on its
reference and object databases.

The problem is that RefDirectory.close method was a no-op and the
reference database was kept in memory. This problem is only happening
when opening a repository using the RepositoryCache because it never
evicts repositories, it's just calling the close method.

Change-Id: Iacb961de8e8b1f5b37824bf0d1a4caf4c6f1233f
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
2016-04-07 11:32:57 +02:00
Hugo Arès 83235432e7 Fix repository cache never closing repository
Repository has a usage counter that is initialized to 1 at
instantiation and this counter is decremented when Repository.close
method is called. There is also a Repository.incrementOpen method that
RepositoryCache uses to increment the usage count when it's returning a
repository that is already opened.

The problem was that RepositoryCache was incrementing the usage count
for repositories that it just opened or registered. The usage count was
2 when it should have been 1.

Incrementing usage count is now only be done for repository that are
served from the cache.

This bug is causing slow memory increase of our Gerrit server until the
server become slow. Even if the RepositoryCache is using SoftReference,
it seems that the JVM is not garbage collecting the repositories because
it's not yet on the edge of being out of memory.

To test this change, I replicated all repositories(11k) from Gerrit
master to one slave. The Gerrit master used memory after this test was
10GB without this change and 3.5GB with.

Change-Id: I86c7b36174e384f106b51fe92f306018fd1dbdf0
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
2016-04-07 11:32:57 +02:00
Matthias Sohn f56b9001fc Merge branch 'master' into stable-4.3
* master:
  Add config parameter gc.prunePackExpire for packfile expiration
  In TestRepository, use a consistent clock

Change-Id: I7ac568e650fbd191e48a8f1a4068af72deb242e8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-06 16:05:14 +02:00
Christian Halstrick 47dd98e362 Enable calling of smudge filters when checking out paths
When checking out commits/branches JGit was triggering correctly
configured smudge filters. But when checking out paths (either from
index or from commits) JGit was not triggering smudge filters. Fix
CheckoutCommand to properly call filters.

Bug: 486560
Also-by: Pascal Krause <pascal.krausek@sap.com>
Change-Id: I5ff893054defe57ab12e201d901fe74e1376efea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-06 00:06:53 +02:00
Matthias Sohn 46bef2bcb6 Prepare 4.3-SNAPSHOT builds
Change-Id: Ib831f8870938113bd5338763f90a07d5c108b1de
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-05 23:10:06 +02:00
Preben Ingvaldsen 403f04d8dd Implement DIR_NO_GITLINKS
Implement the DIR_NO_GITLINKS setting with the same functionality
it provides in cGit.

Bug: 436200
Change-Id: I8304e42df2d7e8d7925f515805e075a92ff6ce28
Signed-off-by: Preben Ingvaldsen <preben@puppetlabs.com>
2016-04-05 10:43:40 +02:00
Christian Halstrick 74743bc547 Add config parameter gc.prunePackExpire for packfile expiration
JGit's Garbage Collector is repacking relevant objects into new
packfiles and is afterwards deleting the now obsolete packfiles. But to
prevent problems caused by race conditions JGit was not deleting
packfiles when they are too young. The same mechanism as for loose
objects and the config parameter gc.pruneExpire was used.
But JGit was reusing the parameter gc.pruneExpire also for packfiles
which may cause a lot of filesystem consumption if gc.pruneExpire was
set to the default of 2 weeks. Only two weeks after packfile creation gc
was allowed to delete this packfile.

This change introduces a new config paramter gc.prunePackExpire with a
default of "1.hour". This parameter is used when packfiles are deleted.
Only packfiles older than the specified time can be deleted.

For loose objects the behaviour is not changed and only the old
parameter gc.pruneExpire is relevant. 

Change-Id: I6209efb05678b15153bd22479dc13486907a44f8
2016-03-31 16:11:02 +02:00
Matthias Sohn bf32c9102f JGit v4.3.0.201603230630-rc1
Change-Id: I10835e5aa3618e5033424595942cc1649152cb24
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-23 11:27:04 +01:00
Jonathan Nieder b601e88738 Merge "Add missing @Deprecated annotation" 2016-03-11 17:58:13 -05:00
Jonathan Nieder 90cf265dd4 Add missing @Deprecated annotation
Noticed by error-prone (http://errorprone.info/bugpattern/DepAnn).

Change-Id: If8dcc2dd6d7cabffc95e10f8357ce31c12cf6b1c
2016-03-11 14:55:18 -08:00
Yuxuan 'fishy' Wang 7507d9e2a9 Fix wrong whitespaces.
This change replaced some tabs with spaces introduced in
change I8b3765713599e34f1411f9bbc7f575ec7c2384e0.

Change-Id: Ia5c23b38c9fbbb46f150e527347b61c64c8d9e87
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2016-03-11 14:37:16 -08:00
Yuxuan 'fishy' Wang 0ecb016d7d Add ignoreRemoteFailures option to RepoCommand
With ignoreRemoteFailures set to true, we can ignore remote failures
(e.g. the branch of a project described in the manifest file does not
exist), skip that project and continue to the next one, instead of fail
the whole operation.

Change-Id: I8b3765713599e34f1411f9bbc7f575ec7c2384e0
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2016-03-11 14:20:24 -08:00
Preben Ingvaldsen cff546b0cb Introduce FileModeStrategy to FileTreeIterator
This commit introduces a FileModeStrategy to
the FileTreeIterator class.  This provides a way to
allow different modes of traversing a file tree;
for example, to control whether or not a nested
.git directory should be treated as a gitlink.

Bug: 436200
Change-Id: Ibf85defee28cdeec1e1463e596d0dcd03090dddd
Signed-off-by: Preben Ingvaldsen <preben@puppetlabs.com>
2016-03-11 08:41:13 +01:00
Matthias Sohn 6bcd8c6e90 Merge "Change visibility of DirCacheTree#getObjectId to public" 2016-03-07 20:12:00 -05:00
Philipp Marx 011e5da3e0 Change visibility of DirCacheTree#getObjectId to public
Allow access to the ObjectId of a DirCacheTree if known for low level
integration code.

Change-Id: I6f05b10c9ac781f5e8b38af4a19e653313c91fa8
Signed-off-by: Philipp Marx <smigfu@googlemail.com>
2016-03-07 22:55:46 +01:00
Ivan Motsch b811e4399e Add EOL stream type detection to TreeWalk
TreeWalk provides the new method getEolStreamType. This new method can
be used with EolStreamTypeUtil in order to create a wrapped InputStream
or OutputStream when reading / writing files. The implementation
implements support for the git configuration options core.crlf, core.eol
and the .gitattributes "text", "eol" and "binary"

CQ: 10896
Bug: 486563
Change-Id: Ie4f6367afc2a6aec1de56faf95120fff0339a358
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-07 17:24:32 +01:00
XinTong Wang 770d36c8ba Fix apply patch which did not work with non-ascii characters
Bug: 483943
Change-Id: If28f64053d20ab1bee54245f223e952dc2fe392c
Signed-off-by: XinTong Wang <xintong@ca.ibm.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-03 01:43:02 +01:00
Ivan Motsch 2baa5f1d6d Tests on Windows with URIish fail
The reason is that URIish(URL) and URIish(String) make different parsing
of path / rawPath with regard to drive letters. /C:/... for URL and
C:/... for String. This patch fixes the issue.

Change-Id: I8e2013fff30b7bb198ff733c038e21366667b8a0
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
2016-02-25 15:39:41 +01:00
Shawn Pearce 68d8a51d08 Introduce specific WantNotValidException for servers
Capture the internal "want X not valid" state as a specific subclass
of PackProtocolException, allowing this to be more easily identified
in server stack traces and wrapper application code.

Change-Id: I4b1adb7497f396432da420b0f600ad25a261f912
2016-02-16 16:41:51 -08:00
Shawn Pearce e7a6e85b95 smart HTTP server: Pass along "want X not valid" to client
If the client sends a SHA-1 that the server does not recognize echo
this back to the client with an explicit error message instead of
the generic "internal server error".

This was always the intent of the implementation but it was being
dropped on smart HTTP due to the UploadPackServlet catching the
PackProtocolException, discarding the buffered message UploadPack
meant to send, and sending along a generic message instead.

Change-Id: I8d96b064ec655aef64ac2ef3e01853625af32cd1
2016-02-16 15:09:05 -08:00
Matthias Sohn ff5c756c79 Merge branch 'stable-4.2'
* stable-4.2:
  Revert "Fix warnings about unchecked conversion of MergeResult"

Change-Id: I31c2e0679ad4cff77190858ac9a570d04841c386
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-16 01:06:11 +01:00
Shawn Pearce 8c1d44e436 Merge "Revert "Fix warnings about unchecked conversion of MergeResult"" into stable-4.2 2016-02-15 19:00:50 -05:00
Shawn Pearce 591e39bf9d Revert "Fix warnings about unchecked conversion of MergeResult"
This reverts commit 979fa19110.
Breaks API.

Change-Id: I54af657898d49f64d6906fe3edfb6b08e996d901
2016-02-15 18:45:55 -05:00
Matthias Sohn 514b11ddcc Merge branch 'stable-4.2'
* stable-4.2:
  Don't use deprecated LockFile constructor
  Fix warnings about unchecked conversion of MergeResult
  MockServletConfig: Fix warning about unchecked conversion of Enumeration
  HugeFileTest: Make Git a class member and open in try-with-resource
  Suppress "unchecked cast" warnings related to UploadPackFactory.DISABLED
  DiffAlgorithms: Fix warnings about variable hiding
  DirCacheBasicTest: Open ObjectInserter.Formatter in try-with-resource
  DirCacheBuilderIteratorTest: Open TreeWalk in try-with-resource
  DirCacheCGitCompatabilityTest: Open TreeWalk in try-with-resource
  DirCacheCheckoutMaliciousPathTest: Open Git and RevWalk in t-w-r
  DirCacheIteratorTest: Open TreeWalk instances in try-with-resource
  ForPathTest: Open TreeWalk in try-with-resource
  GitConstructionTest: Open Git instance in try-with-resource
  IndexDiffTest: Open Git instances in try-with-resources
  ManifestParserTest: Don't use deprecated StringBufferInputStream
  InMemoryRepository: Remove unused RevWalk from batch method signature
  IndexModificationTimesTest: Open Git instances in try-with-resource
  InterIndexDiffFilterTest: Open TreeWalk in try-with-resource
  LockFileTest: Open Git instance in try-with-resource
  JGit v4.1.2.201602141800-r
  MergeCommandTest: Use JUnit's assume to check preconditions
  MergeCommandTest: Open Git instances in try-with-resource

Change-Id: Ie5dba6b9132a29e86958a04fa2b76465bcd2c6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-15 23:28:18 +01:00
Shawn Pearce 621b0740b3 Merge "Allow deletion of HEAD ref if the repository is bare." 2016-02-15 13:02:05 -05:00
David Pursehouse e96cb22a43 Don't use deprecated LockFile constructor
Change-Id: Ibc3e2f3372e1a65732dd6d3c71cec53fb1aa15e2
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 18:44:24 +09:00
David Pursehouse 979fa19110 Fix warnings about unchecked conversion of MergeResult
Change-Id: I1490b2209fa7b39676849c624adbc262a672f6df
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 18:22:53 +09:00
David Pursehouse f04bd20fc6 InMemoryRepository: Remove unused RevWalk from batch method signature
The RevWalk given in the arguments is not used. According to the
comment at the top of the method, a new RevWalk is intentionally
used in the implementation.

Remove the unused argument.

Change-Id: Iec81a1341d5bf377801475845b96a465753096ef
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 14:57:39 +09:00
Ivan Motsch 975aa88685 Add Attribute Macro Expansion
Attributes MacroExpander implements macros used in git attributes. This
is implemented inside the TreeWalk using a lazy created MacroExpander.
In addition, the macro expander caches the global and info attributes
node in order to provide fast merge of attributes.

Change-Id: I2e69c9fc84e9d7fb8df0a05817d688fc456d8f00
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
2016-02-11 17:08:49 +01:00
Matthias Sohn 28e2fed761 Merge branch 'stable-4.2'
* stable-4.2:
  RepoProject: Fix warnings about variable hiding
  RepoTest: Open Git in try-with-resources
  RepositoryResolveTest: Open Git in try-with-resource
  RepositoryTestCase: Open autocloseable types in try-with-resource
  ResetCommandTest: Use Git member in testHardResetAfterSquashMerge
  ResolveMergerTest: Open Git in try-with-resource
  RevCommitListTest: Open Git and RevWalk in try-with-resource
  RevCommitParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevObjectTest: Open RevWalk in try-with-resource
  RevTagParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevertCommandTest: Open Git in try-with-resource
  SquashMessageFormatterTest: Open git in try-with-resource
  StatusCommandTest: Open Git in try-with-resource
  SubmoduleAddTest: Open Git in try-with-resource
  SymlinksTest: Open git and TreeWalk in try-with-resource
  T0003_BasicTest: Open autocloseable types in try-with-resource
  TextHashFunctions: Fix warnings about variable hiding
  TreeFilterTest: Open TreeWalk in try-with-resource
  TreeWalkJava7Test: Open TreeWalk in try-with-resource
  Fix diff for added and removed submodule

Change-Id: If3ecc63f6dfac55474d3c1dd2f4105371f3d24fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-10 00:51:01 +01:00
Matthias Sohn 26012958a3 Merge branch 'stable-4.1' into stable-4.2
* stable-4.1:
  Fix diff for added and removed submodule

Change-Id: I37dd71ed19b06e9bbcffe37370081ab875c6d8d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-10 00:45:18 +01:00
David Pursehouse 4e5255c59a RepoProject: Fix warnings about variable hiding
Method parameter names were hiding class members of the same
name.

Change-Id: I182f2715894ac4259b09a371cb4e0eb24f52518a
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-05 18:22:58 +09:00
Saša Živkov a0e1374e22 Allow to reuse disableSslVerify method, move it to HttpSupport
The disableSslVerify method will be used in the follow up change.

Change-Id: Ie00b5e14244a9a036cbdef94768007f1c25aa8d3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-04 17:49:43 +01:00
Matthias Sohn 3bae524f6f Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.

Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.

At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:

  [lfs]
	  url = http://host:port/lfs
	  batch = true
  [lfs "http://host:port/lfs"]
	  access = none

the git-lfs client appends the "objects/batch" to the lfs.url.

Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.

The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].

- upload and download of large files is probably network bound hence use
  an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
  concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
  doesn't match id given in URL

The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.

[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api

Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
2016-02-04 17:49:43 +01:00
Matthias Sohn 536db18cc6 Implement SHA-256 abstraction
The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.

[1] https://git-lfs.github.com/

Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-04 17:49:42 +01:00
Hugo Arès df904a4227 Fix diff for added and removed submodule
Since If13f7b406, submodule difference are shown as a hunk. The issue
was that added and removed submodule were considered as Edit.REPLACE
instead of Edit.INSERT and Edit.DELETE in the DiffFormatter result.

Change-Id: I4330c2aa3f10e29d7d6b0b2e5286e59293a06239
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
2016-02-04 10:27:50 -05:00
Matthias Sohn 2de33d7678 Remove unused API problem filters
Change-Id: I8b1c99b108b4cea6356e0a62e6bfc0731df44cf5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-03 10:39:08 +01:00
Matthias Sohn b0facdc113 Merge branch 'stable-4.2'
* stable-4.2:
  DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
  CommitCommand: Remove declaration of unthrown exception
  Branch: Fix variable hiding warning
  ApplyCommandTest: Open Git in try-with-resource
  PackFileTest: Open ObjectInserter.Formatter in try-with-resource

Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-03 10:30:52 +01:00
David Pursehouse 521f513d6d CommitCommand: Remove declaration of unthrown exception
Change-Id: I1795b1e510ab170f062ee6c5988d7a7567abe260
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-02 18:27:47 +09:00
Matthias Sohn 9400d0cb17 Merge "Prevent ChainingCredentialsProvider from adding delegated providers twice" 2016-01-28 18:51:14 -05:00
Mike Williams 48ad6842fc Allow deletion of HEAD ref if the repository is bare.
Change-Id: I2281d818c9f76019e7e053e89b20214f2d663957
Signed-off-by: Mike Williams <miwilliams@google.com>
2016-01-28 11:43:59 -05:00
Yuxuan 'fishy' Wang 7960fa8735 [gitrepo] Support revision in remote tag.
Repo manifest file allows revision attribute in <remote> tag. This change
teaches JGit to read that information.

Change-Id: I1c878a2505b9d09fa09fbd404a119b71f2fb8fdb
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2016-01-27 16:01:06 -08:00
Rüdiger Herrmann bd67084d2b Prevent ChainingCredentialsProvider from adding delegated providers twice
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Change-Id: Icc0907af4fe26877d066db3c813dba37213a3ccc
2016-01-26 18:55:39 +01:00
Thomas Wolf 9441d61e37 Fix a @since tag
Ketch didn't make it into 4.2

Change-Id: Iab3ddfb926a2f8d31622e212c845c825369b09d5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2016-01-23 23:51:14 +01:00
Matthias Sohn fc4d117511 Prepare 4.2.1-SNAPSHOT builds
Change-Id: Ic9eac53b10ac97b2038b334d388ae3d21393f993
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-22 01:48:52 +01:00
Matthias Sohn 20c2079f70 JGit v4.2.0.201601211800-r
Change-Id: I89f33b0da81ff499315baa5311f95f31724fa510
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 23:47:12 +01:00
Terry Parker f321741cff Merge "Remove unused variable from DfsBlockCache" 2016-01-21 16:21:15 -05:00
Terry Parker 4d178be021 Remove unused variable from DfsBlockCache
Change-Id: I8ea167e44e9ebcac7ca08920e806d8bcb8188bf0
Signed-off-by: Terry Parker <tparker@google.com>
2016-01-21 13:02:39 -08:00
Matthias Sohn b30917ce21 Prepare 4.3.0-SNAPSHOT builds
Change-Id: Idcf0479529693b023042becd96698f9afd344bd4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 21:49:41 +01:00
Laurent Delaigue 4409751516 Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.

This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.

Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 17:08:04 +01:00
Chris Gavin 82f47ace3d Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.

Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.

Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7
Signed-off-by: Chris Gavin <chris@chrisgavin.me>
2016-01-21 17:07:57 +01:00
Laurent Delaigue 1e1fb9fc59 Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.

This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.

Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 11:37:26 +01:00
Shawn Pearce 08aac5904e Merge "Fix TransportException when reading bundle" 2016-01-21 01:15:12 -05:00
Chris Gavin d2bf41e7dd Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.

Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.

Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7
Signed-off-by: Chris Gavin <chris@chrisgavin.me>
2016-01-21 01:14:56 -05:00
Shawn Pearce 8fe9a8baf5 Merge changes from topic 'reftree'
* changes:
  debug-rebuild-ref-tree: Copy HEAD into RefTree
  debug-rebuild-ref-tree: Add --enable flag to turn the database on
  RefTreeDatabase: Allow ORIG_HEAD, etc. on non-bare repositories
  RefTreeDatabase: Expose bootstrap refs in getAdditionalRefs
2016-01-21 00:32:17 -05:00
Shawn Pearce 8d0551dd64 Merge changes from topic 'ketch'
* changes:
  daemon: Add --ketch=LEADER flag
  Ketch: Intercept push and route it through Ketch
  Ketch: Simple map of running leader instances
  Ketch: Basic replication system
2016-01-20 23:15:09 -05:00
Shawn Pearce 7b6122908b Merge "TreeWalk: Remove CorruptObjectException from addTree(AbstractTreeIterator)" 2016-01-20 11:03:47 -05:00
Shawn Pearce 2262a794b4 TreeWalk: Remove CorruptObjectException from addTree(AbstractTreeIterator)
This form of addTree() does not parse any objects and cannot throw the
declared checked exception.  Callers are being forced to try-catch
CorruptObjectException that cannot occur when the iterator instance
has already been constructed.

Change-Id: Id338035302903bab81569d1576eab063eee0885a
2016-01-20 07:41:08 -08:00
Christian Halstrick 7182cb2a26 Fix ResetCommand to return the resulting ref
ResetCommand was not returning the updated ref as a result of the call()
method. Since the ResetCommand is always updating the same ref (HEAD)
this should always be the HEAD ref.

Bug: 440750
Change-Id: I7974975c3ab05e68c208384e69cf0692ded6e8db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-20 11:27:31 +01:00
Christian Halstrick da43d8d798 Add option to allow empty commits to CommitCommand
CommitCommand should allow to specify whether empty commits (commits
having the same tree as the sole predecessor commit) are allowed or not.
Similar to native git's "--allow-empty" flag.

The defaults differ between JGit and native git even after this change.
When not specifying paths then by default JGit allows to create empty
commits while native git does not. It would be API breaking to change
this now.

Bug: 460301
Change-Id: I88feb0c3ffb2c686b1d0594e669729b065cda4cb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-20 11:14:19 +01:00
Shawn Pearce da5ef91ad0 daemon: Add --ketch=LEADER flag
Experimental flag to turn on the KetchLeader within this daemon JVM.
This is a manually elected leader process, set from the command line.
Remote followers for each repository are configured per-repository
using remote sections with ketch-type = FULL. For example:

Manually elected leader's $GIT_DIR/config:

  [ketch]
    name = A

  [remote "A"]
    ketch-type = FULL

  [remote "B"]
    url = git://127.0.0.1:9421/sample.git
    ketch-type = FULL

  [remote "C"]
    url = git://127.0.0.1:9422/sample.git
    ketch-type = FULL

Replica B and C daemons:

  git daemon \
    --export-all \
    --enable=receive-pack \
    --listen=127.0.0.1 --port=9421 \
    --base-path=$HOME/ketch_test/follower_one \
    $HOME/ketch_test/follower_one &

  git daemon \
    --export-all \
    --enable=receive-pack \
    --listen=127.0.0.1 --port=9422 \
    --base-path=$HOME/ketch_test/follower_two \
    $HOME/ketch_test/follower_two &

Change-Id: I165f85970a77e16b5263115290d685d8a00566f5
2016-01-19 23:05:28 -08:00
Shawn Pearce 1f9d205043 Ketch: Intercept push and route it through Ketch
Capture commands and pass to the in-process KetchLeader, allowing
it to replicate to followers.

Change-Id: I25dfeb2a93821af65354337f391480a72bae2210
2016-01-19 23:04:41 -08:00
Shawn Pearce b718e3d304 Ketch: Simple map of running leader instances
Supports use in basic daemons where RepositoryCache is also effective.

Change-Id: Ia736b32c2c710b5a802233badc0d30818de763cd
2016-01-19 23:03:35 -08:00
Shawn Pearce 9b33f4aeeb Ketch: Basic replication system
Git Ketch is a multi-master Git repository management system.  Writes
are successful only if a majority of participant servers agree.  Acked
writes are durable against server failures as a majority of the
participants store all required objects.

Git Ketch is modeled on the Raft Consensus Algorithm[1].  A ketch
sailing vessel is faster and more nimble than a raft.  It can also
carry more source codes.

Git Ketch front-loads replication costs, which vaguely resembles a
ketch sailing vessel's distinguishing feature of the main mast on the
front of the ship.

[1] https://raft.github.io/
Change-Id: Ib378dab068961fc7de624cd96030266660b64fb4
2016-01-19 23:03:32 -08:00
David Pursehouse aca07fac46 Merge branch 'stable-4.2'
* stable-4.2:
  CheckoutCommandTest: Create Git instances in try-with-resource
  BranchCommandTest: Create Git instances in try-with-resource
  CheckoutTest: Create Git instances in try-with-resource
  BranchTest: Create Git instances in try-with-resource
  URIishTest: Use @Test annotation's `expected` argument
  Suppress "The allocated object is never used" warning in tests
  Add $NON-NLS to suppress "Non-externalized string literal" warnings
  Don't use deprecated constructors of CmdLineException
  Prepare 4.2.0-SNAPSHOT builds
  Remove org.eclipse.jgit.updatesite project from tools/version.sh
  RevParse: Remove superfluous semicolon
  RefUpdateTest: Use try-with-resource for auto-closable types
  RefUpdateTest: Add null check to prevent potential NPE
  CommitCommand: Remove redundant null check
  JGit v4.2.0.201512141825-rc1

Change-Id: I2179859289b2f2e3d0b7c6d02ef7e7890c467f7b
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-20 09:45:48 +09:00
Dave Borowitz 2ccea7f05a ChangeIdUtil: Don't throw IOException
This could have only happened during the getBytes call. Instead, use
Constants.encode, which is a non-throwing implementation.

This change is binary compatible with existing code compiled against
older versions of JGit, although it might break compilation of
previously compiling code due to dead catch blocks.

Change-Id: I191fec5cac718657407230de141440e86d0151fb
2016-01-19 17:59:19 -05:00
Shawn Pearce a693d7b33c Merge "Transport: Implement AutoCloseable" 2016-01-19 17:36:39 -05:00
Shawn Pearce 6b662af76e Transport: Implement AutoCloseable
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.

Change-Id: I0c6ff3e39ebecdd7a028dbcae1856a818937b186
2016-01-19 14:27:11 -08:00
Shawn Pearce 68a6d0c0ab Merge changes from topic 'deadtree'
* changes:
  Ignore API errors about Tree, TreeEntry, FileTreeEntry and friends being removed
  Revert "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends""
2016-01-19 17:08:54 -05:00
Shawn Pearce 9d3b56a905 Merge "ReceiveCommand.abort(): Utility to mark batch of commands as failed" 2016-01-19 15:54:07 -05:00
Shawn Pearce 462017e02b Ignore API errors about Tree, TreeEntry, FileTreeEntry and friends being
removed

Bug: 486105
Change-Id: I04adcdb68bee7d5f608bb7ab959fe36a890f9ecd
2016-01-19 18:49:11 +01:00
Andrey Loskutov 7f31a9c9ad Revert "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and
friends""

This reverts commit 2cc80187d3.

Bug: 486105
Change-Id: Id4f9987c33d66cbed9de6e4d4d6784afdd01a3cf
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-19 18:48:48 +01:00
David Pursehouse 9ddc0ed83d Add $NON-NLS to suppress "Non-externalized string literal" warnings
Change-Id: I3e6f83ad2bc7d493e2c1ab5a8c60affa2b49c386
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-19 17:27:45 +01:00
Andrey Loskutov 2006e90abc Annotated to be removed Tree API with @noreference and @noextend
See https://wiki.eclipse.org/Eclipse/API_Central/API_Removal_Process.

Bug: 486105
Change-Id: I460e43da0d487279608729a2081c614e7065f56f
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-19 14:27:41 +01:00
Matthias Sohn 20f272a71c Prepare 4.2.0-SNAPSHOT builds
Change-Id: Ibe38a95bf36db0c0ed948280b28c416943ec0329
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-19 13:10:20 +01:00
Shawn Pearce eadfcd3ec1 ReceiveCommand.abort(): Utility to mark batch of commands as failed
If one or more commands is failing the entire group usually has to
also fail with "transaction aborted". Pull this loop into a helper
so the idiom can be easily reused in several places throughout JGit.

Change-Id: I3b9399b7e26ce2b0dc5f7baa85d585a433b4eaed
2016-01-18 11:39:09 -08:00
Shawn Pearce bbfca06d64 RefTreeDatabase: Allow ORIG_HEAD, etc. on non-bare repositories
Store these in the bootstrap layer where they are using $GIT_DIR
as the storage directory for any reference that does not contain '/'.

Change-Id: I5595bf514e4475b7c7e799c2c79446597a3abb4a
2016-01-15 12:46:33 -08:00
Shawn Pearce a290b822ab RefTreeDatabase: Expose bootstrap refs in getAdditionalRefs
By showing the bootstrap layer in getAdditionalRefs() garbage
collector code can be more RefDatabase agnostic and not care about
the special case of RefTree and RefTreeNames for the purposes of
building up the roots to GC. Instead they can combine getRefs(ALL)
and getAdditionalRefs() and have a clean set of roots.

Change-Id: I665cd2456e9316640215b6a08bc728d1356f36d8
2016-01-15 11:07:19 -08:00
Shawn Pearce f52581c6a5 Merge "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends"" 2016-01-15 10:45:43 -05:00
Shawn Pearce 2cc80187d3 Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends"
This reverts commit 0f8743d4d7.

JGit is unable to iterate its API.

Change-Id: Ie3d6a28e622a5c0cf54768a2299f1c44c0114c19
2016-01-15 10:45:36 -05:00
David Pursehouse 1ed5382b37 CommitCommand: Remove redundant null check
Repository.getWorkTree is annotated as @NonNull, so the check
for it returning null is redundant.

Change-Id: I597b0f774ff857b8900519f14a1a17a904cf7c6f
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-14 15:04:34 +09:00
Shawn Pearce 95bdc589b8 Merge "PackWriter: Declare preparePack object sets as @NonNull" 2016-01-12 19:45:40 -05:00
Shawn Pearce 4e650c0d76 PackWriter: Declare preparePack object sets as @NonNull
Require callers to pass in valid sets for both want and have
collections. Offer PackWriter.NONE as a handy constant for an
empty collection for the have part of preparePack instead of null.

Change-Id: Ifda4450f5e488cbfefd728382b7d30797e229217
2016-01-12 19:30:32 -05:00
Shawn Pearce 17580101f5 Merge "GC: Pack RefTrees in their own pack" 2016-01-12 18:49:02 -05:00
Shawn Pearce 40051505d7 GC: Pack RefTrees in their own pack
The RefTree graph needs to be quickly accessed to read references.
It is also distinct graph disconnected from the rest of the
repository. Store the commit and tree objects in their own pack.

Change-Id: Icbb735be8fa91ccbf0708ca3a219b364e11a6b83
2016-01-12 12:15:53 -08:00
Shawn Pearce 9e3d391953 Change to extensions.refsStorage
git-core just rerolled the extensible backends series with refsStorage
as the configuration key. Update JGit to match git-core.

Change-Id: If345a2403a996e358b29cfa2a2298f6e8d59d96b
2016-01-11 20:44:10 -08:00
Shawn Pearce 398d8e877f RefTree: Change peel suffix to " ^" (space carrot)
Using ^{} as the peel suffix has caused problems when projects used
tags like v2.1 and then v2.1.1, v2.2.2, etc.  The peeled value for
v2.1 was stored very far away in the tree relative to v2.1 itself as
^ sorts in the ASCII/UTF-8 encoding after all other common tag
characters like digits and dots.

Use " ^" instead as space is not valid in a reference name, sorts
before all other valid reference characters (thus forcing next entry
locality) and this looks like a peeled marker for the prior tag.

Change-Id: I26d2247a0428dfe26a9c319c02159502b3a67455
2016-01-11 15:27:14 -08:00
Shawn Pearce 088c2fc6e3 FileRepository: Support extensions.refsBackendType = RefTree
This experimental code can be enabled in $GIT_DIR/config:

  [core]
    repositoryformatversion = 1

  [extensions]
    refsBackendType = RefTree

When these are set the repository will read references from the
RefTree rooted by the $GIT_DIR/refs/txn/committed reference.

Update debug-rebuild-ref-tree to rebuild refs/txn/committed only from
the bootstrap layer.  This avoids misuse by rebuilding using packed-refs
and $GIT_DIR/refs tree.

Change-Id: Icf600e4a36b2f7867822a7ab1f1617d73c710a4b
2016-01-11 15:27:07 -08:00
Shawn Pearce 48e245fc60 RefTreeDatabase: Ref database using refs/txn/committed
Instead of storing references in the local filesystem rely on the
RefTree rooted at refs/txn/committed.  This avoids needing to store
references in the packed-refs file by keeping all data rooted under
a single refs/txn/committed ref.

Performance to scan all references from a well packed RefTree is very
close to reading the packed-refs file from local disk.

Storing a packed RefTree is smaller due to pack file compression,
about 49.39 bytes/ref (on average) compared to packed-refs using
~65.49 bytes/ref.

Change-Id: I75caa631162dc127a780095066195cbacc746d49
2016-01-11 15:26:42 -08:00
Shawn Pearce 8f31aa7c28 Merge "RevCommit: Better support invalid encoding headers" 2016-01-11 15:48:16 -05:00
Shawn Pearce 31d92ace5b RevCommit: Better support invalid encoding headers
With this support we no longer need the 'utf-8' alias. UTF-8 will be
automatically tried when the encoding header is not recognized and used
if the character sequence cleanly decodes as UTF-8.

Modernize some of the references to use StandardCharsets.

Change-Id: I4c0c88750475560e1f2263180c4a98eb8febeca0
2016-01-11 12:30:35 -08:00
Shawn Pearce 0f8743d4d7 Remove deprecated Tree, TreeEntry, FileTreeEntry and friends
These types were deprecated in 0.9.1 (aka 384a19eee0).
If anyone is still using them, its time to stop.

Change-Id: I3f73347ba78c639e0c6a504812bc1a0702f829b1
2016-01-09 09:45:33 -08:00
Shawn Pearce 1243e25aad Paths.pathCompare: Utility to sort paths from byte[]
Consolidate copies of this function into one location.

Add some unit tests to prevent bugs that were accidentally
introduced while trying to make this refactoring.

Change-Id: I82f64bbb8601ca2d8316ca57ae8119df32bb5c08
2016-01-09 09:44:57 -08:00
Shawn Pearce bace383507 Add Paths utility class
Simple container for some path related utility functions.

Change-Id: Ice2bec6ad12b1e2cea15988c01aa9dd4e016a849
2016-01-08 16:58:23 -08:00
Shawn Pearce 73eb32be82 RefTree: Store references in a Git tree
A group of updates can be applied by updating the tree in one step,
writing out a new root tree, and storing its SHA-1. If references
are stored in RefTrees, comparing two repositories is a matter of
checking if two SHA-1s are identical. Without RefTrees comparing two
repositories requires listing all references and comparing the sets.

Track the "refs/" directory as a root tree by storing references
that point directly at an object as a GITLINK entry in the tree.
For example "refs/heads/master" is written as "heads/master".

Annotated tags also store their peeled value with ^{} suffix, using
"tags/v1.0" and "tags/v1.0^{}" GITLINK entries.

Symbolic references are written as SYMLINK entries with the blob of
the symlink carrying the name of the symbolic reference target.

HEAD is outside of "refs/" namespace so it is stored as a special
"..HEAD" entry. This name is chosen because ".." is not valid in
a reference name and it almost looks like "../HEAD" which names
HEAD if the reader was inside of the "refs/" directory.

A new Command type is required to handle symbolic references and
peeled references.

Change-Id: Id47e5d4d32149a9e500854147edd7d93c1041a39
2016-01-07 15:40:26 -08:00
Shawn Pearce 5001038f52 Merge "InMemoryRepository: Abort BatchRefUpdate if a command previously failed" 2016-01-06 15:18:04 -05:00
Shawn Pearce 4c9eda17be InMemoryRepository: Abort BatchRefUpdate if a command previously failed
If any command has already been marked as failing, fail the entire batch.

Change-Id: I1692240841aa4f4cb252bdccbc6f11d9246929c1
2016-01-06 12:05:46 -08:00
Shawn Pearce f0d634eed7 DFS: Allow other RefDatabase implementations
Permit a DfsRepository implementation to use a different RefDatabase
than DfsRefDatabase.

Change-Id: Ia263285f547bde1943993cc994d0222185021a16
2016-01-06 11:54:45 -08:00
Matthias Sohn 91a3f17474 Merge changes Idaed9310,I62bd7c0b
* changes:
  ObjectChecker: honor some git-core fsck.* options
  ObjectChecker: allow some objects to skip errors
2016-01-03 10:27:37 -05:00
Shawn Pearce 163be57d0f Merge changes from topic 'add-df'
* changes:
  DirCache: Do not create duplicate tree entries
  DirCacheEditor: Replace file-with-tree and tree-with-file
  AddCommand: Use NameConflictTreeWalk to identify file-dir changes
2016-01-01 12:58:35 -05:00
Shawn Pearce 09500165a8 Fix "remote: Counting objects: ..." formatting
Trailing whitespace is usually removed in properties files so
JGitText did not supply a space between : and the remote message.
Ensure the space exists at runtime by reading the localized string
and appending a space if it is missing.

Messages should be dynamically fetched and not held in a static
class variable, as they can be changed using thread locals.

Change-Id: If6a3707d64094253b1a5304fbfafcf195db7497a
2015-12-31 17:37:16 -08:00
David Ostrovsky 13502fef8f Implement Buck driven build
Today there are plenty of modern build tool systems available in the
wild (in no particular order):

* http://bazel.io
* https://pantsbuild.github.io
* http://shakebuild.com
* https://ninja-build.org
* https://buckbuild.com

The attributes, that all these build tools have in common, are:

* reliable
* correct
* very fast
* reproducible

It must not always be the other build tool, this project is currently
using. Or, quoting Gerrit Code Review maintainer here:

  "Friends, don't let friends use <the other build tool system>!"

This change is non-complete implementation of JGit build in Buck,
needed by Gerrit Code Review to replace its dependency with standlone
JGit cell. This is very useful when a developer is working on both
projects and is trying to integrate changes made in JGit in Gerrit.

The supported workflow is:

  $ cd jgit
  $ emacs <hack>
  $ cd ../gerrit
  $ buck build --config repositories.jgit=../jgit gerrit

With --config repositories.jgit=../jgit jgit cell is routed through
JGit development tree.

To build jgit, issue:

  $ buck build //:jgit
  [-] PROCESSING BUCK FILES...FINISHED 0,0s

Yes, you can't measure no-op build time, given that Buck daemon is
used.

Change-Id: I301a71b19fba35a5093d8cc64d4ba970c2877a44
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2015-12-31 10:08:55 -08:00
Shawn Pearce 2fbd98dbe8 Merge "Fix encoding problem from curl repostory on github" 2015-12-30 23:47:28 -05:00
Shawn Pearce 80edcac06f Merge "Fix hanging fetch via SSH" 2015-12-30 23:27:09 -05:00
Eryk Szymanski 53c3bbe42f Fix encoding problem from curl repostory on github
Pushing curl repository to gerrit fails with a message:
remote: error: internal error while processing changes
java.nio.charset.IllegalCharsetNameException: 'utf8'

curl repository url: https://github.com/bagder/curl.git

To avoid this problem encodingAliases in RawParseUtils have
been extended to contain "'utf8'" (single quoted utf8) string.

Change-Id: I40f613cfdcabf0dc9455bee45116ab8d8c7dd6ee
Signed-off-by: Eryk Szymanski <eryksz@gmail.com>
2015-12-30 23:24:12 -05:00
Shawn Pearce ac41920a43 ObjectChecker: honor some git-core fsck.* options
Accept some of the same section keys that fsck does in git-core,
allowing repositories to skip over specific kinds of acceptable
broken objects, e.g.:

  [fsck]
    duplicateEntries = ignore
    zeroPaddedFilemode = ignore

The zeroPaddedFilemode = ignore is a synonym for the JGit specific
allowLeadingZeroFileMode = true. Only accept the JGit key if git-core
key was not specified.

Change-Id: Idaed9310e2a5ce5511670ead1aaea2b30aac903c
2015-12-30 15:19:09 -08:00
Shawn Pearce fa7ce0e0f3 ObjectChecker: allow some objects to skip errors
Some ancient objects may be broken, but in a relatively harmless way.
Allow the ObjectChecker caller to whitelist specific objects that are
going to fail checks, but that have been reviewed by a human and decided
the objects are OK enough to permit continued use of.

This avoids needing to rewrite history to scrub the broken objects out.

Honor the git-core fsck.skipList configuration setting when receiving a
push or fetching from a remote repository.

Change-Id: I62bd7c0b0848981f73dd7c752860fd02794233a6
2015-12-30 15:18:48 -08:00
Matthias Sohn 3fc93f8a56 Rename files using NIO2 atomic rename
Bug: 319233
Change-Id: I5137212f5cd3195a52f90ed5e4ce3cf194a13efd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-30 17:22:33 -05:00
Shawn Pearce e3acf01748 ObjectChecker: use java.text.Normalizer directly
Base Java version for JGit is now Java 7. The java.text.Normalizer
class was available in Java 6. Reflection is no longer required to
normalize strings for Mac OS X.

Change-Id: I98e14b72629a7a729a2d40a3aa275932841274e8
2015-12-30 15:45:13 -05:00
Shawn Pearce 7177b506b4 Merge changes I3db332bf,I614e7110,I37a4ccd6,I86b81e7f,Ic8b362e0, ...
* changes:
  Sort "eager" path-like options to the end of the help
  reset command: provide convenient and meaningful options help
  commit command: allow to specify path(s) argument(s)
  status command: consume more then one argument after --
  repo command: properly name the required 'path' argument
  Un-ignored existing CLI tests which run just fine on Java 7+
  Don't treat command termination due '-h' option as a fatal error
2015-12-30 15:44:30 -05:00
Shawn Pearce 018a91b715 Unify fetch and receive ObjectChecker setup
This avoids duplication of code between receive-pack and fetch-pack paths.
Separate methods are still required to check use of receive.fsckobjects vs.
fetch.fsckobjects, both of which default to transfer.fsckobjects.

Change-Id: I41193e093e981a79fc2f63914e273aaa44b82162
2015-12-29 16:55:41 -08:00
Shawn Pearce 29aa444760 PackWriter: use lib.ObjectIdSet to avoid wrapper
Hoist ObjectIdSet up to lib as part of the public API and add
the interface to some common types like PackIndex and JGit custom
ObjectId map types.  This cleans up wrapper code in a number of
places by allowing direct use of the types as an ObjectIdSet.

Future commits can now rely on ObjectIdSet as a simple read-only
type to check a set of objects from a number of storage options.

Change-Id: Ib62b062421d475bd52abd6c84a73916ef36e084b
2015-12-29 15:32:24 -08:00
Shawn Pearce 683c41af92 DirCache: Do not create duplicate tree entries
If a file (e.g.  "A") and a subtree file (e.g. "A/foo.c") both appear
in the DirCache this cache should not be written out as a tree object.
The "A" file and "A" subtree conflict with each other in the same tree
and will fail fsck.

Detect this condition during DirCacheBuilder and DirCacheEditor
finish() so the application can be halted early before it updates a
DirCache that might later write an invalid tree structure.

Change-Id: I95660787e88df336297949b383f4c5fda52e75f5
2015-12-29 11:33:39 -08:00
Shawn Pearce b71ba69410 DirCacheEditor: Replace file-with-tree and tree-with-file
If a PathEdit tries to store a file where a subtree was, or a subtree
where a file was, replace the entry in the DirCache with the new
name(s).  This supports switching between file and tree entry types
using a DirCacheEditor.

Add new unit tests to cover the conditions where these can happen.

Change-Id: Ie843d9388825f9e3d918a5666aa04e47cd6306e7
2015-12-29 11:33:39 -08:00
Shawn Pearce 3776b14ab4 AddCommand: Use NameConflictTreeWalk to identify file-dir changes
Adding a path that already exists but is changing type such as
from symlink to subdirectory requires a NameConflictTreeWalk to
match up the two different entry types that share the same name.

NameConflictTreeWalk needs a bug fix to pop conflicting entries
when PathFilterGroup aborts the walk early so that it does not
allow DirCacheBuilderIterator to copy conflicting entries into
the output cache.

Change-Id: I61b49cbe949ca8b4b98f9eb6dbe7b1f82eabb724
2015-12-29 11:33:35 -08:00
Andrey Loskutov 0505657d6a commit command: allow to specify path(s) argument(s)
This fixes the command below:

jgit commit a -m "added file a"

which currently fails with:

org.eclipse.jgit.api.errors.JGitInternalException: The combination of
arguments --all and --only is not allowed

Bug: 484973
Change-Id: I37a4ccd68101a66520ef99110f7aa0cbdcc8beba
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-29 15:00:09 +01:00
Andrey Loskutov 241b50be31 Simplify development of commands: added main() to CLIGitCommand
This will execute git commands (with arguments) specified on the command
line, handy for developing/debugging a sequence of arbitrary git
commands working on same repository.

The git working dir path can be specified via Java system property
"git_work_tree". If not specified, current directory will be used.

Change-Id: I621a9ec198c31e28a383818efeb4b3f835ba1d6f
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-28 22:59:59 +01:00
Shawn Pearce ef757a7e12 DirCacheEditor: Cleanup DeleteTree constructor
Neaten up formatting and avoid strings, which prevents the need for
NLS comment tags.  Instead check the last character using char
literal, and append a char literal instead of a string.

Change-Id: Ib68e017769a1f5c03200354a805769d585a48c8b
2015-12-28 11:43:31 -08:00
Shawn Pearce 110d3ca595 DirCacheEditor: Fix formatting to avoid , at start of line
Change-Id: I1b1d614470c67fe4736fdc9c26ae26fb38dd58b5
2015-12-28 11:43:30 -08:00
Shawn Pearce 21cec184b7 Merge "AddCommand: Avoid unnecessary string conversions" 2015-12-28 14:24:33 -05:00
Christian Halstrick 2fdce1ef8c Merge "AddCommand: Cleanup conditional logic" 2015-12-28 08:39:07 -05:00
Shawn Pearce a94e517940 AddCommand: Avoid unnecessary string conversions
Change-Id: I13634caeccd9f675a86adfdfa94099b6fb75463a
2015-12-23 22:26:50 -08:00
Shawn Pearce bb8627bd9b AddCommand: Cleanup conditional logic
Unnest and simplify conditional logic for handling entries.

Change-Id: I3093cab5f0edfaf3efbbd6c644e9c922edc67d38
2015-12-23 20:31:15 -08:00
Dmitry Neverov 9c71bf14b7 Close copy threads in case of errors
Bug: 484775
Change-Id: I3c7105188e615b6b994261f4ece0c8abc98eb444
Signed-off-by: Dmitry Neverov <dmitry.neverov@gmail.com>
2015-12-24 01:20:14 +01:00
Yuxuan 'fishy' Wang d88695e412 Skip nested copyfiles in RepoCommand.
Similar to nested directories, nested copyfiles won't work with git submodule
either.

Change-Id: Idbe965ec20a682fca0432802858162f8238f05de
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2015-12-22 14:05:35 -08:00
Andrey Loskutov 1aafa61958 Checkout should be able to override modified symbolic links
Handle existing symlink as a file, not as directory if deleting a file
before creating (overriding) a symlink.

Bug: 484491
Change-Id: I29dbf57d1daec2ba98454975b093e1d381d05196
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-17 10:42:44 +01:00
Shawn Pearce 8cfbbc9f54 Merge "ProgressSpinner: Simple busy wait entertainment" 2015-12-16 16:20:59 -05:00
Shawn Pearce a38531b21c ProgressSpinner: Simple busy wait entertainment
Keep a user amused while the server does work by spinning a
little ASCII-art object on a single line.

Change-Id: Ie8f181d1aa606d4ae69e5d3ca4db387cea739f38
2015-12-16 15:43:07 -05:00
James Kolb 3096a13340 Included cached deltas in delta packStatistics.
Previously, non-reuse deltas were only included in packStatistics if they
were not cached by the deltaWindow.

Change-Id: I7684d8214875f0a7569b34614f8a3ba341dbde9c
Signed-off-by: James Kolb <jkolb@google.com>
2015-12-16 14:06:16 -05:00
Jonathan Nieder cfa0b2fb3c Merge changes Ib4d53bdd,I55bd512c
* changes:
  Do not let PathFilter.create("a/b") match 'a' unless 'a' is a subtree
  Add tests for PathFilterGroup.Single
2015-12-16 12:09:49 -05:00
Shawn Pearce a3e9876d8f Merge "Fix InterruptTimer leak in BasePackConnection" 2015-12-16 10:29:02 -05:00
Matthias Sohn 6340c76d8b Fix InterruptTimer leak in BasePackConnection
When setting timeout on push, BasePackConnection creates a timer, which
will be terminated when push finishes. But, when using
SmartHttpPushConnection, it dropped the first timer created in the
constructor and then created another timer in doPush. If new threads are
created faster than the gc collects then this may stop the service if
it's hitting the max process limit. Hence don't create a new timer if it
already exists.

Bug: 474947
Change-Id: I6746ffe4584ad919369afd5bdbba66fe736be314
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-16 14:19:38 +01:00
Shawn Pearce 7c5b2761ed Merge "Refspec: loosen restrictions on wildcard "*"" 2015-12-15 23:22:12 -05:00
Shawn Pearce 9eda23e469 Merge "BaseRepositoryBuilder should trim CR from .git symref" 2015-12-15 23:18:14 -05:00
Shawn Pearce 8b749f7de5 Merge "Fix possible arithmetic overflow when setting a timeout" 2015-12-15 23:16:21 -05:00
Shawn Pearce fcd796a9a0 Merge "Accept UTF8 BOM with BlobBasedConfig" 2015-12-15 23:12:56 -05:00
Shawn Pearce 6297c2e92c Merge "Bug 484342: Support @ in username in SSH url." 2015-12-15 23:09:31 -05:00
Jonathan Nieder 75a0dcac18 Do not let PathFilter.create("a/b") match 'a' unless 'a' is a subtree
PathFilter and PathFilterGroup form JGit's implementation of git's
path-limiting feature in commands like log and diff.  To save time
when traversing trees, a path specification

	foo/bar/baz

tells the tree walker not to traverse unrelated trees like qux/.  It
does that by returning false from include when the tree walker is
visiting qux and true when it is visiting foo.

Unfortunately that test was implemented to be slightly over-eager: it
doesn't only return true when asked whether to visit a subtree "foo"
but when asked about a plain file "foo" as well.  As a result, diffs
and logs restricted to some-file/non-existing-suffix unexpectedly
match against some-file:

 $ jgit log -- LICENSE/no-such-file
 commit 629fd0d594
 Author: Shawn O. Pearce <spearce@spearce.org>
 Date:   Fri Jul 02 14:52:49 2010 -0700

     Clean up LICENSE file
[...]

Fix it by checking against the entry's mode.

Gitiles +log has the same bug and benefits from the same fix.

Callers know not to worry about what subtrees are included in the tree
walk because shouldBeRecursive() returns true in this case, so this
behavior change should be safe.  This also better matches the behavior
of C git:

 $ empty=$(git mktree </dev/null)
 $ git diff-tree --abbrev $empty HEAD -- LICENSE/no-such-file
 $ git diff-tree --abbrev $empty HEAD -- tools/no-such-file
 :000000 040000 0000000... b62648d... A  tools

Bug: 484266
Change-Id: Ib4d53bddd8413a9548622c7b25b338d287d8889d
2015-12-15 15:22:10 -08:00
Mike Gilbode 3e03907f65 Bug 484342: Support @ in username in SSH url.
Change-Id: I5795e925afff796488ba26c83694e806b76a374f
Signed-off-by: Mike Gilbode <gilbode@gmail.com>
2015-12-15 10:44:16 -05:00
Matthias Sohn 2647d024af Fix push with jgit pgm failing with "unauthorized"
Pushing with JGit commandline to e.g. Github failed with "unauthorized"
since HttpUrlConnection calls the configured authenticator implicitly.

The problem is that during a push two requests are sent to the server,
first a GET and then a POST (containing the pack data). The first GET
request sent anonymously is rejected with 401 (unauthorized). When an
Authenticator is installed the java.net classes will use the
Authenticator to ask the user for credentials and retry the request.
But this happens under the hood and JGit level code doesn't see that
this happens.

The next request is the POST but since JGit thinks the first GET request
went through anonymously it doesn't add authentication headers to the
POST request. This POST of course also fails with 401 but since this
request contains a lot of body-data streamed from JGit (the pack file!)
the java.net classes can't simply retry the request with authorization
headers. The whole process fails.

Fix this by using Apache httpclient which doesn't use Authenticator to
retrieve credentials. Instead initialize TransportCommand to use the
default credential provider if no other credentials provider was set
explicitly. org.eclipse.jgit.pgm.Main sets this default for the JGit
command line client.

Change-Id: Ic4e0f8b60d4bd6e69d91eae0c7e1b44cdf851b00
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-15 12:49:08 +01:00
Christian Halstrick 5c66ad5684 Merge "NetRCCredentialsProvider should return false if any item is missing" 2015-12-15 06:44:08 -05:00
Christian Halstrick 2f0b675a71 Merge "Fix ChainingCredentialsProvider" 2015-12-15 03:52:09 -05:00
Christian Halstrick 310e858f81 Fix possible arithmetic overflow when setting a timeout
BasePackPushConnection#readStringLongTimeout() was setting a timeout 10
times bigger than some other timeout or the pack transfer time. This
could lead to negative integer values when we hit an arithmetic
overflow. Add a check for this situation and set the timeout to
Integer.MAX_VALUE when overflow happens.

Bug: 484352
CC: Eugene Petrenko <eugene.petrenko@gmail.com>
Change-Id: Ie2a86312c1bcb1ec3e6388fa490ab3c845d41808
2015-12-15 09:44:09 +01:00
Andrey Loskutov 95b36b397b Null-annotated Ref class and fixed related compiler errors
This change fixes all compiler errors in JGit and replaces possible
NPE's with either appropriate exceptions, avoiding multiple "Nullable
return" method calls or early returning from the method.

Change-Id: I24c8a600ec962d61d5f40abf73eac4203e115240
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-15 09:07:05 +01:00
Matthias Sohn d2044aed4f JGit v4.2.0.201512141825-rc1
Change-Id: Id725ea888acd23793d37de2864ec0727160a2eaf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-15 00:22:13 +01:00
Shawn Pearce 8179185d11 Merge "push: Do not blindly overwrite peer" 2015-12-14 17:45:57 -05:00
Shawn Pearce 4f8c993c49 push: Do not blindly overwrite peer
If an application uses PushConnection directly on the native Git wire
protocols JGit should send along the application's expected oldId, not
the advertised value.  This allows the remote peer to compare-and-swap
since it was not tested inside JGit.

Discovered when I tried to use a PushConnection (bypassing the
standard PushProcess) and the client blindly overwrote the remote
reference, even though my app had supplied the wrong ObjectId for
the expectedOldObjectId. This was not expected and cost me over an
hour of debugging, plus "corruption" in the remote repository.

By passing along the exact expectedOldObjectId from the app the
remote side can do the check that the application skipped, and
avoid data loss.

Change-Id: Id3920837e6c47100376225bb4dd61fa3e88c64db
2015-12-14 09:18:27 -08:00
Christian Halstrick 5b55498b16 Fix FileTreeIterator.idSubmodule(Entry)
FileTreeIterator was calling by mistake
WorkingTreeIterator.idSubmodule(Entry). Instead it should always compute
idSubmodule on its own.

Change-Id: Id1b988aded06939b1d7edd2671e34bf756896c0e
2015-12-13 01:24:46 +01:00
Andrey Loskutov 85d09a9ec7 URIish: fixed full uri pattern not expecting end of line after host name
Bug: 483326
Change-Id: I8b6e3eb648c8ec2c38f73de22382537b1276b779
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-12 00:55:31 +01:00
Kaloyan Raev e23521f0d9 Add remote command to JGit CLI
Supported subcommands are:
 - <none> (lists available remotes)
 - add
 - remove
 - set-url
 - update
 
Supported options are:
 --verbose
 --push
 --prune

Bug: 481316
Change-Id: I57c34ed6daabb7d308bc383b17c1ef4af433e714
Signed-off-by: Kaloyan Raev <kaloyan.r@zend.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-11 22:54:48 +01:00
Doug Kelly 618b0c1ceb Accept UTF8 BOM with BlobBasedConfig
In I1f5dc07182dbf6bba2a9f4807fdd25b475da4ead, FileBasedConfig got
support for reading a configuration with UTF8 BOM.  Apply the same
support to BlobBasedConfig, to make SubmoduleWalk able to parse
.gitmodules configurations with BOM.

Change-Id: I25b5474779952fe2c076180b96fc2869eef190a8
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
2015-12-09 16:48:20 -06:00
Matthias Sohn afd167a1f2 NetRCCredentialsProvider should return false if any item is missing
Change-Id: I894d1621aaccd71dfe100fe83a1bd9d50a1e0808
2015-12-08 14:12:26 +01:00
Matthias Sohn 5786cc3c0f Fix ChainingCredentialsProvider
The ChainingCredentialsProvider gave up chaining to the next provider if
the first one returned no credentials items for the given URI.

Change-Id: I9539c50db35e564db9d43d8ebb71d7e9c6fdcc19
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-08 14:12:26 +01:00
Christian Halstrick 6dfad98d72 Merge "Fix NPE in HttpAuthMethod" 2015-12-08 02:25:35 -05:00
Jonathan Nieder e12482deed Use runtime retention for Nullable annotation
JGit's Nullable type was added[1] in the hope of being able to add
nullness annotations that (a) do not preclude building and running
with Java 7 and (b) could be shared by Gerrit, which uses a custom
Nullable type for other reasons[2].  Sharing a type is useful because
Eclipse's null analysis is only able to use one Nullable type at a
time in a given workspace (so for this analysis to function in a
workspace used to develop Gerrit, JGit and Gerrit would need to use
the same Nullable type).

The new Nullable type has CLASS instead of RUNTIME retention because
there wasn't any obvious use for the annotation at run time.

Gerrit uses the Nullable annotation to communicate with Guice.  Guice
injection happens at runtime, so it needs to be able to read the
@Nullable annotations at run time[3].  Otherwise Guice produces
provisioning errors, such as

 3) null returned by binding at com.google.gerrit.lucene.LuceneChangeIndex$Factory.create()
  but parameter 7 of com.google.gerrit.lucene.LuceneChangeIndex.<init>() is not @Nullable

Switch to RUNTIME retention to avoid this.

While at it, update the javadoc to explain more clearly how this
annotation relates to other Nullable types[4].  This should make it
clearer why JGit needed another Nullable type:

 A. Avoiding dependency on Java 8
 B. RUNTIME retention to allow Guice to read the annotation at run time
 C. Named Nullable so Guice can recognize the annotation
 D. Not an addition to Java EE's javax.annotation package, to avoid
    the split-package problem[2] that prevents the annotation from
    being readable at run time when loaded from an OSGi container
 E. Avoiding heavyweight dependencies, deprecated dependencies, and
    dependencies on package internals

 org.checkerframework.checker.nullness.qual.Nullable: A
 com.sun.istack.internal.Nullable: B, E
 *.CheckForNull, *.NullAllowed, etc: C
 edu.umd.cs.findbugs.annotations.Nullable: B, E
 javax.annotation.Nullable: D
 org.eclipse.jdt.annotation.Nullable: B
 org.jetbrains.annotations.Nullable: B
 org.jmlspecs.annotation.Nullable: E
 android.annotation.Nullable, android.support.annotation.Nullable: E

[1] https://git.eclipse.org/r/59993
[2] https://gerrit-review.googlesource.com/50112
[3] https://github.com/google/guice/blob/master/core/src/com/google/inject/internal/Nullability.java
[4] https://github.com/typetools/checker-framework/blob/5832a01f1/checker/src/org/checkerframework/checker/nullness/NullnessAnnotatedTypeFactory.java#L118
    http://types.cs.washington.edu/checker-framework/current/checker-framework-manual.html#nullness-related-work

Change-Id: I6c482653d2b53e3509abb11211b67fc29cf2949c
Signed-off-by: Jonathan Nieder <jrn@google.com>
2015-12-07 19:26:23 -05:00
Matthias Sohn ac89b47eeb Fix NPE in HttpAuthMethod
If the password char array is null constructing a new String from this
array fails with a NPE. Add a null check to fix this.

Change-Id: Ifae6eecca38d5f114861f44658a32521e6e96866
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-07 16:05:19 +01:00
Shawn Pearce 3d8e6b1e16 Support atomic push in JGit client
This should mirror the behavior of `git push --atomic` where the
client asks the server to apply all-or-nothing. Some JGit servers
already support this based on a custom DFS backend. InMemoryRepository
is extended to support atomic push for unit testing purposes.

Local disk server side support inside of JGit is a more complex animal
due to the excessive amount of file locking required to protect every
reference as a loose reference.

Change-Id: I15083fbe48447678e034afeffb4639572a32f50c
2015-12-02 22:06:06 -08:00
Matthias Sohn f109af47d0 Fix wrong @since tag In ReceiveCommand
Change-Id: I58fa657dd4783fed0ffca94020c87c49d99009c6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-02 15:24:26 +01:00
Shawn Pearce f89f30ffcd ReceiveCommand.filter: Accept Iterable
PreReceiveHook is given a Collection<ReceiveCommand> and it can be
very useful here to call ReceiveCommand.filter(cmds, NOT_ATTEMPTED).

Overload filter to accept both Iterable and List.
Keep backwards binary compatibility for List by upcasting to Iterable.

Change-Id: Ib1341876c703670945ef209edc8259715ee86c26
2015-12-01 19:59:03 -08:00
Shawn Pearce 8078021e8f Merge changes If1490ca2,Ife950253
* changes:
  DirCacheBuilder: Speed up reading from trees
  Delay locating .gitattributes until requested
2015-11-30 19:03:11 -05:00
Shawn Pearce 2d011cd648 DirCacheBuilder: Speed up reading from trees
Recursively copying a tree into a DirCache is a bottleneck for some
algorithms like the in memory merge code in Gerrit Code Review.  Drop
a layer down in the stack and use CanonicalTreeParser directly as the
addition logic only processes 1 tree at a time and does not need the
merge sorting feature (or overhead) of TreeWalk.

Combined with 761814fe9c ("DirCacheEntry: Speed up creation by
avoiding string cast") tree loading 38,900 entries nearly halves
in running time from 70ms to 36ms on some platforms.

Change-Id: If1490ca25de0679a71cf508f59b486f9cc816165
2015-11-30 15:46:40 -08:00
Shawn Pearce 93eca6dfe1 DirCache: Add helper to read from a tree
Application code sometimes wants to read a DirCache from an ObjectId,
but its confusing how to do this because its buried inside the
DirCacheBuilder.

Use this utility in a few places within JGit that also want to read
a DirCache from a tree's ObjectId.

Change-Id: I578b7e18e58753d154937f4ab835012b09e5adca
2015-11-29 22:05:24 -05:00
Shawn Pearce b0eb744604 Delay locating .gitattributes until requested
Instead of checking every entry for .gitattributes only look for the
entry on request by TreeWalk.  This avoids impacting uses like RevWalk
filtering history.

When the attrs is requested skip to the start of the tree and look for
.gitattributes until either it is found, or it is impossible to be
present.  Due to the sorting rules of tree entries .gitattributes
should be among the first or second entries in the tree so very few
entries will need to be considered.

Waiting to find the .gitattributes file by native ordering may miss
attrs for files like .config, which sorts before .gitattributes.
Starting from the front of the tree on demand ensures the attributes
are parsed as early as necessary to process any entry in the tree.

Due to TreeWalk recursively processing up the tree of iterators we
cannot just reset the current CanonicalTreeParser to the start as
parent parsers share the same path buffer as their children.
Resetting a parent to look for .gitattributes may overwrite path
buffer data used by a child iterator.  Work around this by building a
new temporary CanonicalTreeParser instance.

Change-Id: Ife950253b687be325340d27e9915c9a40df2641c
2015-11-29 18:56:49 -08:00
Shawn Pearce 45cc76524b DirCache: Fix bad code formatting
Line breaking before , is ugly to read.  Most formatters and humans
expect line break after , so update a few offensive locations.

Use String.format() for the construction of the error message when
a bad DirCachEntry is being failed on. This simplifies the code and
its not a performance critical section.

Change-Id: I5d990389e7ba24ef0861cf8ec0026ed030d4aeda
2015-11-28 09:28:09 -08:00
Shawn Pearce 761814fe9c DirCacheEntry: Speed up creation by avoiding string cast
The checkPath function is available as a byte[] form, in fact the
String form just converts to byte[] to run the algorithm.

Having DirCacheEntry take a byte[] -> String -> byte[] to check if
each path is valid is a huge waste of CPU time. On some systems it
can double the time required to read 38,999 files from trees to the
DirCache. This slows down any operation using a DirCache.

Expose the byte[] form and use it for DirCacheEntry creation.

Change-Id: I6db7bc793ece99ff3c356338d793c07c061aeac7
2015-11-28 09:01:03 -08:00
Shawn Pearce 885879ffe9 DirCache: Fix getEntriesWithin("") to not include null entries
The internal array may be longer than entryCnt, in this case the tail
of the array is padded with null entries. Do not return those to the
caller of getEntriesWithin().

Change-Id: I19efb05e103fab6b739ced407f6e28155a48dba6
2015-11-27 23:25:22 -08:00
Shawn Pearce 46e4992e92 Fix performance regression in CanonicalTreeParser
Change-Id: I14046559fddb9656d890d3099010117e84cd9439
2015-11-27 23:23:49 -08:00
Christian Halstrick bd31555ee5 Add support for smudge filters
If defined in .gitattributes call smudge filter during checkout.

To support checkout where current HEAD,index do not contain attributes
we need to also consider attributes from the tree we checkout. Therefore
CanonicalTreeParser has to learn how to provide attributes.

Change-Id: I168fdb81a8e1a9f991587b3e95a36550ea845f0a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-11-27 23:33:53 +01:00
Christian Halstrick 5d9f595eb8 Add support for clean filters
When filters are defined for certain paths in gitattributes make
sure that clean filters are processed when adding new content to the
object database.

Change-Id: Iffd72914cec5b434ba4d0de232e285b7492db868
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-11-27 23:23:09 +01:00
Ivan Motsch 75697adc5a Add the new class Attributes holding multiple Attribute(s)
Attributes represents a semantic collector of Attribute(s) and replaces
the anonymous Map<String,Attribute>. This class will be returned by
TreeWalk.getAttributes(). It offers convenient access to the attributes
wrapped in the Attributes object. Adds preparations for a future
Attribute Macro Expansion

Change-Id: I8348c8c457a2a7f1f0c48050e10399b0fa1cdbe1
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
2015-11-27 11:40:34 +01:00
Matthias Sohn 2cdc130dfc Add missing @since tags and missing javadoc
Change-Id: I8575797127fc96abea8af56f019ca39f5897486f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-11-25 19:25:53 -05:00
Christian Halstrick 55fb18feaa Fix FS.runProcess() to close the InputStream
When FS.runProcess was called and an InputStream was given the method
tried to pump the whole InputStream to the process. When the method
ended the InputStream was not giving any data anymore. Consequently
close the InputStream inside the method.

Change-Id: I0ed738a775e5c977b21447d195acee1ecf5e2cb9
2015-11-26 00:36:09 +01:00
Jonathan Nieder fe98218e0d repo: Do not use search path to find refs/remotes/origin/<branch>
When running from a non-bare repository, "jgit repo" checks whether
the rev passed in is a sha1 or branch name and in the latter case will
check out origin/<branch>.

We are expecting refs/remotes/origin/<branch>, but as a side effect of
using getRef we also end up looking for refs/origin/<branch>,
refs/heads/origin/<branch>, and so on.  Avoid that by using exactRef
instead.

Signed-off-by: Jonathan Nieder <jrn@google.com>
Change-Id: I670b2e48a88138a1f2104ea201baa958e9edbddb
2015-11-25 14:05:24 -08:00