The special characters <> and '\n' interfere with parsing of
identities. C git strips these special characters, so we should too.
Rather than allocating extra strings by calling String#trim(), add a
few lines to our sanitization method to perform the same trimming as
described in String's Javadoc.
Change-Id: I96edcb93a2fc194ee354d60566d352299742a52f
This might be somewhat surprising behavior to users who might
naturally assume the following invariant:
ident.equals(parseIdent(ident.toExternalString()))
This invariant does not hold since whitespace is only trimmed during
serialization. We don't want to mess with the strings during
initialization, as this is called during the highly-optimized commit
parsing codepath.
Change-Id: I081a603f0ac0e33167462244779b0ff3ad51e80c
We've found in Gerrit Code Review that it is common to pass around
both an ObjectReader (or more commonly a RevWalk wrapping one) and an
ObjectInserter. These code paths often assume that the ObjectReader
can read back any objects created by the ObjectInserter without
flushing. However, we previously had no way to enforce that constraint
programmatically, leading to hard-to-spot problems.
Provide a solution by exposing the ObjectInserter that created an
ObjectReader, when known. Callers can either continue passing both
objects and check:
reader.getCreatedFromInserter() == inserter
or they can just pass around ObjectReader and extract the inserter
when it's needed (checking that it's not null at usage time).
Change-Id: Ibbf5d1968b506f6b47030ab1b046ffccb47352ea
When CheckoutCommand or MergeCommand is called then not in all situation
the treewalks have been prepared to support clean/smudge filters. Fix
this
Bug: 491505
Change-Id: Iab5608049221c46d06812552ab97299e44d59e64
Such hunks are identifiable by a zero value for "new start line". Prior
to the fix, JGit throws and ArrayIndexOutOfBoundsException on such
patches.
Change-Id: I4f3deb5e5f41a08af965fcc178d678c77270cddb
Signed-off-by: Jonathan Schneider <jkschneider@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I5b3b7b0633354d5ccf0c6c320c0df9c93fdf8eeb
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
CommitCommand already provided a method to set the comment which should
be written into the reflog. The underlying RefUpdate class supported to
skip writing a reflog entry. But through the CommitCommand API it was
not possible to prevent writing a reflog entry. Fix this and allow
creating commits which don't occur in the reflog.
Change-Id: I193c53de71fb5958ea749c4bfa8360a51acc9b58
RepositoryCache has 2 methods to remove a repository from the cache but
they are never called when a repository is closed. Users of the cache
were expected to call one of those 2 methods but how could they have
called them at proper time without having visibility of the repository
usage count.
Ideally, I would have reworked the RepositoryCache to wrap any
repository it opens in a class that would be responsible to unregister
them from the cache when it's really closed, i.e. when usage counter
reaches 0. The problem preventing the wrapping solution is the
RepositoryCache.register method that allows to register an already
opened repository in the cache. Such repositories cannot be wrapped
because callers are still holding a reference on the unwrapped
repository.
Document that RepositoryCache.close method is removing the repository
from the cache as well as closing it and rework
RepositoryCache.unregister method to only remove the repository from the
cache. Use the latter to unregister repository when Repository.doClose
is getting executed.
Change-Id: Ia364816e4da8d7b6cfa72f10758ca31aa8a1f9db
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Repository has a usage counter that is initialized to 1 at
instantiation and this counter is decremented when Repository.close
method is called. There is also a Repository.incrementOpen method that
RepositoryCache uses to increment the usage count when it's returning a
repository that is already opened.
The problem was that RepositoryCache was incrementing the usage count
for repositories that it just opened or registered. The usage count was
2 when it should have been 1.
Incrementing usage count is now only be done for repository that are
served from the cache.
This bug is causing slow memory increase of our Gerrit server until the
server become slow. Even if the RepositoryCache is using SoftReference,
it seems that the JVM is not garbage collecting the repositories because
it's not yet on the edge of being out of memory.
To test this change, I replicated all repositories(11k) from Gerrit
master to one slave. The Gerrit master used memory after this test was
10GB without this change and 3.5GB with.
Change-Id: I86c7b36174e384f106b51fe92f306018fd1dbdf0
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
* master:
Add config parameter gc.prunePackExpire for packfile expiration
In TestRepository, use a consistent clock
Change-Id: I7ac568e650fbd191e48a8f1a4068af72deb242e8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When checking out commits/branches JGit was triggering correctly
configured smudge filters. But when checking out paths (either from
index or from commits) JGit was not triggering smudge filters. Fix
CheckoutCommand to properly call filters.
Bug: 486560
Also-by: Pascal Krause <pascal.krausek@sap.com>
Change-Id: I5ff893054defe57ab12e201d901fe74e1376efea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Implement the DIR_NO_GITLINKS setting with the same functionality
it provides in cGit.
Bug: 436200
Change-Id: I8304e42df2d7e8d7925f515805e075a92ff6ce28
Signed-off-by: Preben Ingvaldsen <preben@puppetlabs.com>
JGit's Garbage Collector is repacking relevant objects into new
packfiles and is afterwards deleting the now obsolete packfiles. But to
prevent problems caused by race conditions JGit was not deleting
packfiles when they are too young. The same mechanism as for loose
objects and the config parameter gc.pruneExpire was used.
But JGit was reusing the parameter gc.pruneExpire also for packfiles
which may cause a lot of filesystem consumption if gc.pruneExpire was
set to the default of 2 weeks. Only two weeks after packfile creation gc
was allowed to delete this packfile.
This change introduces a new config paramter gc.prunePackExpire with a
default of "1.hour". This parameter is used when packfiles are deleted.
Only packfiles older than the specified time can be deleted.
For loose objects the behaviour is not changed and only the old
parameter gc.pruneExpire is relevant.
Change-Id: I6209efb05678b15153bd22479dc13486907a44f8
This commit introduces a FileModeStrategy to
the FileTreeIterator class. This provides a way to
allow different modes of traversing a file tree;
for example, to control whether or not a nested
.git directory should be treated as a gitlink.
Bug: 436200
Change-Id: Ibf85defee28cdeec1e1463e596d0dcd03090dddd
Signed-off-by: Preben Ingvaldsen <preben@puppetlabs.com>
TreeWalk provides the new method getEolStreamType. This new method can
be used with EolStreamTypeUtil in order to create a wrapped InputStream
or OutputStream when reading / writing files. The implementation
implements support for the git configuration options core.crlf, core.eol
and the .gitattributes "text", "eol" and "binary"
CQ: 10896
Bug: 486563
Change-Id: Ie4f6367afc2a6aec1de56faf95120fff0339a358
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.2:
Don't use deprecated LockFile constructor
Fix warnings about unchecked conversion of MergeResult
MockServletConfig: Fix warning about unchecked conversion of Enumeration
HugeFileTest: Make Git a class member and open in try-with-resource
Suppress "unchecked cast" warnings related to UploadPackFactory.DISABLED
DiffAlgorithms: Fix warnings about variable hiding
DirCacheBasicTest: Open ObjectInserter.Formatter in try-with-resource
DirCacheBuilderIteratorTest: Open TreeWalk in try-with-resource
DirCacheCGitCompatabilityTest: Open TreeWalk in try-with-resource
DirCacheCheckoutMaliciousPathTest: Open Git and RevWalk in t-w-r
DirCacheIteratorTest: Open TreeWalk instances in try-with-resource
ForPathTest: Open TreeWalk in try-with-resource
GitConstructionTest: Open Git instance in try-with-resource
IndexDiffTest: Open Git instances in try-with-resources
ManifestParserTest: Don't use deprecated StringBufferInputStream
InMemoryRepository: Remove unused RevWalk from batch method signature
IndexModificationTimesTest: Open Git instances in try-with-resource
InterIndexDiffFilterTest: Open TreeWalk in try-with-resource
LockFileTest: Open Git instance in try-with-resource
JGit v4.1.2.201602141800-r
MergeCommandTest: Use JUnit's assume to check preconditions
MergeCommandTest: Open Git instances in try-with-resource
Change-Id: Ie5dba6b9132a29e86958a04fa2b76465bcd2c6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There's only one test method in this module and it's quite long, so
rather than using a try-with-resource and having to indent a huge
block of existing code, make the Git a member variable that gets
initialised and closed in @Before and @After annotated methods.
The methods are named 'before' and 'after' rather than the conventional
'setUp' and 'tearDown' so as not to conflict with the names of the
existing methods in LocalDiskRepositoryTestCase.
Change-Id: I5a4a9b59f244c450dbcae9fdde7d9e0f0cd24e6f
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Replace it with ByteArrayInputStream as suggested in [1].
[1] http://stackoverflow.com/a/2219543/381622
Change-Id: I5ca8d721a756a82ea5f5687a20555303eb1dfc18
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
* stable-4.2:
NoteMapTest: Open TreeWalk instances in try-with-resource
ObjectDirectoryTest: Fix warnings about variable hiding
PackWriterTest: Open RevWalk in try-with-resource
PatchIdDiffFormatterTest: Open Git and PatchIdDiffFormatter in try-with-resource
PathSuffixFilterTest: Open TreeWalk in try-with-resource
PostOrderTreeWalkTest: Open TreeWalk in try-with-resource
PullCommandWithRebaseTest: Open RevWalk in try-with-resource
PushCommandTest: Open Git instances in try-with-resource
RacyGitTests: Open NameConflictTreeWalk in try-with-resource
RecursiveMergerTest: Open TreeWalk and BufferedReader in try-with-resource
ReflogConfigTest: refactor commit method to avoid variable hiding
Update .mailmap
RefDirectoryTest: Fix warning about member variable hiding
ReflogResolveTest: Open Git instances in try-with-resource
ReflogTest: Open Git instances in try-with-resource
RepoCommandTest: Open Git instances in try-with-resource
Change-Id: I7964b699396629e31a9cc5600aedcf4be4e659a8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Using the assume method, instead of just returning, will cause the
test to be marked as skipped rather than passed on systems where
the precondition is not satisfied.
Change-Id: I13672371f6cd3c481a0a6247e0eaed3aac6d766e
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The variable and parameter named 'db' were hiding class members
with the same name.
Change-Id: I27017afdc5f49c38c6f5be494e7a21239ea601a7
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Also remove unnecessary nesting in test methods.
Change-Id: Id59f8403c0a7b38ebb6b3a24814257cd59ea575d
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The author and committer parameters were hiding class members of
the same name.
Since the passed in PersonIdent instances were being created from
the class members anyway, remove the parameters and instead create
the PersonIdent instances inline in the method.
Change-Id: I66b057df388835d57f332fdcbadb8a9f4e1094a4
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The parameter name 'totalWork' was hiding a class member variable
of the same name.
Change-Id: I646525e82900e23ffabfc756bcf5052ef873656a
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Attributes MacroExpander implements macros used in git attributes. This
is implemented inside the TreeWalk using a lazy created MacroExpander.
In addition, the macro expander caches the global and info attributes
node in order to provide fast merge of attributes.
Change-Id: I2e69c9fc84e9d7fb8df0a05817d688fc456d8f00
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
* stable-4.2:
RepoProject: Fix warnings about variable hiding
RepoTest: Open Git in try-with-resources
RepositoryResolveTest: Open Git in try-with-resource
RepositoryTestCase: Open autocloseable types in try-with-resource
ResetCommandTest: Use Git member in testHardResetAfterSquashMerge
ResolveMergerTest: Open Git in try-with-resource
RevCommitListTest: Open Git and RevWalk in try-with-resource
RevCommitParseTest: Open ObjectInserter.Formatter in try-with-resource
RevObjectTest: Open RevWalk in try-with-resource
RevTagParseTest: Open ObjectInserter.Formatter in try-with-resource
RevertCommandTest: Open Git in try-with-resource
SquashMessageFormatterTest: Open git in try-with-resource
StatusCommandTest: Open Git in try-with-resource
SubmoduleAddTest: Open Git in try-with-resource
SymlinksTest: Open git and TreeWalk in try-with-resource
T0003_BasicTest: Open autocloseable types in try-with-resource
TextHashFunctions: Fix warnings about variable hiding
TreeFilterTest: Open TreeWalk in try-with-resource
TreeWalkJava7Test: Open TreeWalk in try-with-resource
Fix diff for added and removed submodule
Change-Id: If3ecc63f6dfac55474d3c1dd2f4105371f3d24fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.1:
Fix diff for added and removed submodule
Change-Id: I37dd71ed19b06e9bbcffe37370081ab875c6d8d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use the class member instance of the Git rather than creating
a new one in the test. This makes it consistent with the other
tests in the module.
Change-Id: Ibe822ded76a80b94922e92bbc6f5d90bcc52f8a3
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
FileRepository and ObjectInserter.Formatter are autocloseable, so
use try-with-resource for these.
Remove suppression of unused variable warning that is no longer
necessary.
Change-Id: I270829f0a4030083c9599eb5785b0145dc590ed8
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.
[1] https://git-lfs.github.com/
Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since If13f7b406, submodule difference are shown as a hunk. The issue
was that added and removed submodule were considered as Edit.REPLACE
instead of Edit.INSERT and Edit.DELETE in the DiffFormatter result.
Change-Id: I4330c2aa3f10e29d7d6b0b2e5286e59293a06239
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
* stable-4.2:
DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
CommitCommand: Remove declaration of unthrown exception
Branch: Fix variable hiding warning
ApplyCommandTest: Open Git in try-with-resource
PackFileTest: Open ObjectInserter.Formatter in try-with-resource
Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.2:
FileTreeIteratorTest: Open Git and RevWalk in try-with-resource
RebaseCommandTest: Open RevWalk in try-with-resource
PullCommandTest: Open RevWalk in try-with-resource
BlameGeneratorTest: Create Git instances in try-with-resource
Remove unnecessary suppression of deprecation warnings in tests
DiffEntryTest: Open Git and TreeWalk in try-with-resource
DiffCommandTest: Open Git and RevWalk in try-with-resource
CommitCommandTest: Open Git and TreeWalk in try-with-resource
Change-Id: Ic886ec9b1a4b3b46f9fa14188b4df832ce36cfa6
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Repo manifest file allows revision attribute in <remote> tag. This change
teaches JGit to read that information.
Change-Id: I1c878a2505b9d09fa09fbd404a119b71f2fb8fdb
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
* stable-4.2:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
Remove java7 bundle from Maven central scripts
Prepare 4.2.1-SNAPSHOT builds
JGit v4.2.0.201601211800-r
Add progress monitor to Merger
Fix TransportException when reading bundle
Fix unused throws CorruptObjectException from addTree
Change-Id: I2325fb995561a6249b7b5e82fa413dfd34ef6007
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* changes:
ConcurrentRepackTest: Don't use deprecated WindowCache.reconfigure
ConcurrentRepackTest: Open RevWalk in try-with-resource
CommitOnlyTest: Open RevWalk in try-with-resource
UnpackedObjectTest: Create ObjectInserter.Formatter in try-with-resource
FileRepositoryBuilderTest: Use try-with-resource for auto-closeables
RepositorySetupWorkDirTest: Fix "resource leak" warnings
Replace with calls to WindowCacheConfig.install() as mentioned in
WindowCache.reconfigure's deprecation notice.
Change-Id: Ifdb33501a2209239029c815b1e4e844ea5b56075
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The ObjectInserter.Formatter instance is only used to call idFor.
Factor out a utility method to do that.
Change-Id: I4ef823110c2152ac7905681df3217eb8001f5bd9
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Use try-with-resource to create instances of FileRepository and
FileWriter.
"resource" and "unused" warnings no longer occur, so remove the
suppression annotations.
Change-Id: I3ad58d4cc2d4c019cd8edda7cb401e9d9f3fb790
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Use FileRepositoryBuilder to create the Repository, except in cases
where the creation was already in a try-block. Convert those to use
a try-with-resource.
Change-Id: I7d7adeee81bda6e80d91a119c7d690de3d00dc2b
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2262a794b4 removed throws CorruptObjectException, but was not
compiled under Eclipse and did not see errors generated by Eclipse.
Maven build silently ignored the unnecessary code.
Change-Id: I6e9014adcc604588e10e5963d13d0c6ef0e1c617
* stable-4.2:
BundleWriterTest: Open RevWalk in try-with-resource
DiffFormatterTest: Remove accidentally added trailing whitespace
CherryPickCommandTest: Create Git instances in try-with-resource
DiffFormatterTest: Create auto-closeable instances in try-with-resource
ConfigTest: Create Git instance in try-with-resource
CommitAndLogCommandTest: Use assumeFalse to skip test on Windows
CommitAndLogCommandTest: Create Git instances in try-with-resource
AddCommandTest: Create Git instances in try-with-resource
ArchiveCommandTest: Create Git instances in try-with-resource
TagCommandTest: Instantiate Git and RevWalk objects in try-with-resource
BlameCommandTest: Instantiate Git objects in try-with-resource
SideBandOutputStreamTest: Use try-with-resource
FileTreeIteratorJava7Test: Create Git instances in try-with-resource
Change-Id: Ib572e98e6117b70442aee9cd7e7b8c3cf65562a7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Git, ByteArrayOutputStream, and DiffFormatter are auto-closeable and
should be managed in try-with-resource.
Change-Id: I83395116acb4b4f7cd4300fd69564355bc07e4bb
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Use JUnit's assumeFalse method to cause the test to skip when
run on Windows.
Change-Id: I3f59440cfe62c37c127e381052b60471fbe8ec5e
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2262a794b4 removed throws CorruptObjectException, but was not
compiled under Eclipse and did not see errors generated by Eclipse.
Maven build silently ignored the unnecessary code.
Change-Id: I6e9014adcc604588e10e5963d13d0c6ef0e1c617
* changes:
debug-rebuild-ref-tree: Copy HEAD into RefTree
debug-rebuild-ref-tree: Add --enable flag to turn the database on
RefTreeDatabase: Allow ORIG_HEAD, etc. on non-bare repositories
RefTreeDatabase: Expose bootstrap refs in getAdditionalRefs
ResetCommand was not returning the updated ref as a result of the call()
method. Since the ResetCommand is always updating the same ref (HEAD)
this should always be the HEAD ref.
Bug: 440750
Change-Id: I7974975c3ab05e68c208384e69cf0692ded6e8db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
CommitCommand should allow to specify whether empty commits (commits
having the same tree as the sole predecessor commit) are allowed or not.
Similar to native git's "--allow-empty" flag.
The defaults differ between JGit and native git even after this change.
When not specifying paths then by default JGit allows to create empty
commits while native git does not. It would be API breaking to change
this now.
Bug: 460301
Change-Id: I88feb0c3ffb2c686b1d0594e669729b065cda4cb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Factor out the creation of the SideBandOutputStream objects into
a utility method that wraps it in a try-with-resource.
Remove the "unused" suppression that is now unnecessary, and add
declaration that the tests methods can throw Exception.
Change-Id: Iff02e4e3532bd6ab6e423f197e70d44c4f328d0b
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.
Change-Id: I0c6ff3e39ebecdd7a028dbcae1856a818937b186
Also rename a local variable in one of the tests that was hiding
a class variable of the same name.
Change-Id: Ia9398157b87a78df6eef0b64a833c16ca2e57ce3
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Also remove a local variable in one of the tests that was
hiding a member variable with the same name.
Change-Id: Ia4d94cdbf2d83d8be2645f0a93d8891d01606c59
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Specify the expected exception in the annotation, instead of
catching it and calling `fail()` when it wasn't raised.
Change-Id: I8a640c0e42353533e4e73b85b50c224dc060f2d7
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Store these in the bootstrap layer where they are using $GIT_DIR
as the storage directory for any reference that does not contain '/'.
Change-Id: I5595bf514e4475b7c7e799c2c79446597a3abb4a
By showing the bootstrap layer in getAdditionalRefs() garbage
collector code can be more RefDatabase agnostic and not care about
the special case of RefTree and RefTreeNames for the purposes of
building up the roots to GC. Instead they can combine getRefs(ALL)
and getAdditionalRefs() and have a clean set of roots.
Change-Id: I665cd2456e9316640215b6a08bc728d1356f36d8
Require callers to pass in valid sets for both want and have
collections. Offer PackWriter.NONE as a handy constant for an
empty collection for the have part of preparePack instead of null.
Change-Id: Ifda4450f5e488cbfefd728382b7d30797e229217
This experimental code can be enabled in $GIT_DIR/config:
[core]
repositoryformatversion = 1
[extensions]
refsBackendType = RefTree
When these are set the repository will read references from the
RefTree rooted by the $GIT_DIR/refs/txn/committed reference.
Update debug-rebuild-ref-tree to rebuild refs/txn/committed only from
the bootstrap layer. This avoids misuse by rebuilding using packed-refs
and $GIT_DIR/refs tree.
Change-Id: Icf600e4a36b2f7867822a7ab1f1617d73c710a4b
Instead of storing references in the local filesystem rely on the
RefTree rooted at refs/txn/committed. This avoids needing to store
references in the packed-refs file by keeping all data rooted under
a single refs/txn/committed ref.
Performance to scan all references from a well packed RefTree is very
close to reading the packed-refs file from local disk.
Storing a packed RefTree is smaller due to pack file compression,
about 49.39 bytes/ref (on average) compared to packed-refs using
~65.49 bytes/ref.
Change-Id: I75caa631162dc127a780095066195cbacc746d49
With this support we no longer need the 'utf-8' alias. UTF-8 will be
automatically tried when the encoding header is not recognized and used
if the character sequence cleanly decodes as UTF-8.
Modernize some of the references to use StandardCharsets.
Change-Id: I4c0c88750475560e1f2263180c4a98eb8febeca0
These types were deprecated in 0.9.1 (aka 384a19eee0).
If anyone is still using them, its time to stop.
Change-Id: I3f73347ba78c639e0c6a504812bc1a0702f829b1
Consolidate copies of this function into one location.
Add some unit tests to prevent bugs that were accidentally
introduced while trying to make this refactoring.
Change-Id: I82f64bbb8601ca2d8316ca57ae8119df32bb5c08
A group of updates can be applied by updating the tree in one step,
writing out a new root tree, and storing its SHA-1. If references
are stored in RefTrees, comparing two repositories is a matter of
checking if two SHA-1s are identical. Without RefTrees comparing two
repositories requires listing all references and comparing the sets.
Track the "refs/" directory as a root tree by storing references
that point directly at an object as a GITLINK entry in the tree.
For example "refs/heads/master" is written as "heads/master".
Annotated tags also store their peeled value with ^{} suffix, using
"tags/v1.0" and "tags/v1.0^{}" GITLINK entries.
Symbolic references are written as SYMLINK entries with the blob of
the symlink carrying the name of the symbolic reference target.
HEAD is outside of "refs/" namespace so it is stored as a special
"..HEAD" entry. This name is chosen because ".." is not valid in
a reference name and it almost looks like "../HEAD" which names
HEAD if the reader was inside of the "refs/" directory.
A new Command type is required to handle symbolic references and
peeled references.
Change-Id: Id47e5d4d32149a9e500854147edd7d93c1041a39
Maven pom files force the local encoding to UTF-8 to ensure there are
no differences between machines. They also set the JVM max heap to
256m. Match both in Buck so that results are consistent.
Change-Id: Ice5476dd09352a444a0c97aa0dc28806fddf2ab4
* changes:
DirCache: Do not create duplicate tree entries
DirCacheEditor: Replace file-with-tree and tree-with-file
AddCommand: Use NameConflictTreeWalk to identify file-dir changes
Compile each test in its own java_test() target so they can run in
parallel, reducing total time spent testing on large machines.
$ buck test --all
[-] PROCESSING BUCK FILES...FINISHED 0.3s [100%]
[-] BUILDING...FINISHED 2.9s [100%] (351/383 JOBS, 351 UPDATED, 0.0% CACHE MISS)
[-] TESTING...FINISHED 98.1s (3360 PASS/15 SKIP/0 FAIL)
Change-Id: I8d6541268315089299f933ed23d785b1b3431133
Accept some of the same section keys that fsck does in git-core,
allowing repositories to skip over specific kinds of acceptable
broken objects, e.g.:
[fsck]
duplicateEntries = ignore
zeroPaddedFilemode = ignore
The zeroPaddedFilemode = ignore is a synonym for the JGit specific
allowLeadingZeroFileMode = true. Only accept the JGit key if git-core
key was not specified.
Change-Id: Idaed9310e2a5ce5511670ead1aaea2b30aac903c
Some ancient objects may be broken, but in a relatively harmless way.
Allow the ObjectChecker caller to whitelist specific objects that are
going to fail checks, but that have been reviewed by a human and decided
the objects are OK enough to permit continued use of.
This avoids needing to rewrite history to scrub the broken objects out.
Honor the git-core fsck.skipList configuration setting when receiving a
push or fetching from a remote repository.
Change-Id: I62bd7c0b0848981f73dd7c752860fd02794233a6
Hoist ObjectIdSet up to lib as part of the public API and add
the interface to some common types like PackIndex and JGit custom
ObjectId map types. This cleans up wrapper code in a number of
places by allowing direct use of the types as an ObjectIdSet.
Future commits can now rely on ObjectIdSet as a simple read-only
type to check a set of objects from a number of storage options.
Change-Id: Ib62b062421d475bd52abd6c84a73916ef36e084b
If a file (e.g. "A") and a subtree file (e.g. "A/foo.c") both appear
in the DirCache this cache should not be written out as a tree object.
The "A" file and "A" subtree conflict with each other in the same tree
and will fail fsck.
Detect this condition during DirCacheBuilder and DirCacheEditor
finish() so the application can be halted early before it updates a
DirCache that might later write an invalid tree structure.
Change-Id: I95660787e88df336297949b383f4c5fda52e75f5
If a PathEdit tries to store a file where a subtree was, or a subtree
where a file was, replace the entry in the DirCache with the new
name(s). This supports switching between file and tree entry types
using a DirCacheEditor.
Add new unit tests to cover the conditions where these can happen.
Change-Id: Ie843d9388825f9e3d918a5666aa04e47cd6306e7
Adding a path that already exists but is changing type such as
from symlink to subdirectory requires a NameConflictTreeWalk to
match up the two different entry types that share the same name.
NameConflictTreeWalk needs a bug fix to pop conflicting entries
when PathFilterGroup aborts the walk early so that it does not
allow DirCacheBuilderIterator to copy conflicting entries into
the output cache.
Change-Id: I61b49cbe949ca8b4b98f9eb6dbe7b1f82eabb724
Similar to nested directories, nested copyfiles won't work with git submodule
either.
Change-Id: Idbe965ec20a682fca0432802858162f8238f05de
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
Handle existing symlink as a file, not as directory if deleting a file
before creating (overriding) a symlink.
Bug: 484491
Change-Id: I29dbf57d1daec2ba98454975b093e1d381d05196
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Previously, non-reuse deltas were only included in packStatistics if they
were not cached by the deltaWindow.
Change-Id: I7684d8214875f0a7569b34614f8a3ba341dbde9c
Signed-off-by: James Kolb <jkolb@google.com>
PathFilter and PathFilterGroup form JGit's implementation of git's
path-limiting feature in commands like log and diff. To save time
when traversing trees, a path specification
foo/bar/baz
tells the tree walker not to traverse unrelated trees like qux/. It
does that by returning false from include when the tree walker is
visiting qux and true when it is visiting foo.
Unfortunately that test was implemented to be slightly over-eager: it
doesn't only return true when asked whether to visit a subtree "foo"
but when asked about a plain file "foo" as well. As a result, diffs
and logs restricted to some-file/non-existing-suffix unexpectedly
match against some-file:
$ jgit log -- LICENSE/no-such-file
commit 629fd0d594
Author: Shawn O. Pearce <spearce@spearce.org>
Date: Fri Jul 02 14:52:49 2010 -0700
Clean up LICENSE file
[...]
Fix it by checking against the entry's mode.
Gitiles +log has the same bug and benefits from the same fix.
Callers know not to worry about what subtrees are included in the tree
walk because shouldBeRecursive() returns true in this case, so this
behavior change should be safe. This also better matches the behavior
of C git:
$ empty=$(git mktree </dev/null)
$ git diff-tree --abbrev $empty HEAD -- LICENSE/no-such-file
$ git diff-tree --abbrev $empty HEAD -- tools/no-such-file
:000000 040000 0000000... b62648d... A tools
Bug: 484266
Change-Id: Ib4d53bddd8413a9548622c7b25b338d287d8889d
Expand the existing PathFilterGroup tests to check which paths the
tree entry matches. This expands test coverage by ensuring that
PathFilterGroup's simpler code path to match against a single
PathFilter works correctly.
While at it, move the check on tree entry d/e/f/g.y into two separate
tests: one to check that it doesn't match any of the configured paths,
and another to check that it does not throw StopWalkException to end
the walk early.
Change-Id: I55bd512cd049fc2018659e2f86a4b8650f171fda
If an application uses PushConnection directly on the native Git wire
protocols JGit should send along the application's expected oldId, not
the advertised value. This allows the remote peer to compare-and-swap
since it was not tested inside JGit.
Discovered when I tried to use a PushConnection (bypassing the
standard PushProcess) and the client blindly overwrote the remote
reference, even though my app had supplied the wrong ObjectId for
the expectedOldObjectId. This was not expected and cost me over an
hour of debugging, plus "corruption" in the remote repository.
By passing along the exact expectedOldObjectId from the app the
remote side can do the check that the application skipped, and
avoid data loss.
Change-Id: Id3920837e6c47100376225bb4dd61fa3e88c64db
FileTreeIterator was calling by mistake
WorkingTreeIterator.idSubmodule(Entry). Instead it should always compute
idSubmodule on its own.
Change-Id: Id1b988aded06939b1d7edd2671e34bf756896c0e
This should mirror the behavior of `git push --atomic` where the
client asks the server to apply all-or-nothing. Some JGit servers
already support this based on a custom DFS backend. InMemoryRepository
is extended to support atomic push for unit testing purposes.
Local disk server side support inside of JGit is a more complex animal
due to the excessive amount of file locking required to protect every
reference as a loose reference.
Change-Id: I15083fbe48447678e034afeffb4639572a32f50c
Instead of checking every entry for .gitattributes only look for the
entry on request by TreeWalk. This avoids impacting uses like RevWalk
filtering history.
When the attrs is requested skip to the start of the tree and look for
.gitattributes until either it is found, or it is impossible to be
present. Due to the sorting rules of tree entries .gitattributes
should be among the first or second entries in the tree so very few
entries will need to be considered.
Waiting to find the .gitattributes file by native ordering may miss
attrs for files like .config, which sorts before .gitattributes.
Starting from the front of the tree on demand ensures the attributes
are parsed as early as necessary to process any entry in the tree.
Due to TreeWalk recursively processing up the tree of iterators we
cannot just reset the current CanonicalTreeParser to the start as
parent parsers share the same path buffer as their children.
Resetting a parent to look for .gitattributes may overwrite path
buffer data used by a child iterator. Work around this by building a
new temporary CanonicalTreeParser instance.
Change-Id: Ife950253b687be325340d27e9915c9a40df2641c
The checkPath function is available as a byte[] form, in fact the
String form just converts to byte[] to run the algorithm.
Having DirCacheEntry take a byte[] -> String -> byte[] to check if
each path is valid is a huge waste of CPU time. On some systems it
can double the time required to read 38,999 files from trees to the
DirCache. This slows down any operation using a DirCache.
Expose the byte[] form and use it for DirCacheEntry creation.
Change-Id: I6db7bc793ece99ff3c356338d793c07c061aeac7
If defined in .gitattributes call smudge filter during checkout.
To support checkout where current HEAD,index do not contain attributes
we need to also consider attributes from the tree we checkout. Therefore
CanonicalTreeParser has to learn how to provide attributes.
Change-Id: I168fdb81a8e1a9f991587b3e95a36550ea845f0a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When filters are defined for certain paths in gitattributes make
sure that clean filters are processed when adding new content to the
object database.
Change-Id: Iffd72914cec5b434ba4d0de232e285b7492db868
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Attributes represents a semantic collector of Attribute(s) and replaces
the anonymous Map<String,Attribute>. This class will be returned by
TreeWalk.getAttributes(). It offers convenient access to the attributes
wrapped in the Attributes object. Adds preparations for a future
Attribute Macro Expansion
Change-Id: I8348c8c457a2a7f1f0c48050e10399b0fa1cdbe1
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
PushCommandTest and RunExternalScriptTest didn't succeed on Windows.
Fix this by expecting a simple line-feed as line ending (instead of the
platform dependent line separator. Additionally correct the computation
of expected URLs in PushCommandTest.
Change-Id: Idcdc41cd7e535ff88df33ea0a249333ed8fc91b0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This test expected that the test scripts emit a platform-dependent
newline (crlf on windows, lf on linux). But that's not true. Expected
result should always be a trailing "\n" because the test scripts
explicitly echo a "\n" in the end.
Change-Id: I604e08cda8cebe276b5214ba0f618b6112c3441f
The Repository class provides only one method to look up a ref by
name, getRef. If I request refs/heads/master and that ref does not
exist, getRef will look further in the search path:
ref/refs/heads/master
refs/heads/refs/heads/master
refs/remotes/refs/heads/master
This behavior is counterintuitive, needlessly inexpensive, and usually
not what the caller expects.
Allow callers to specify whether to use the search path by providing
two separate methods:
- exactRef, which looks up a ref when its exact name is known
- findRef, which looks for a ref along the search path
For backward compatibility, keep getRef as a deprecated synonym for
findRef.
This change introduces findRef and exactRef but does not update
callers outside tests to use them yet.
Change-Id: I35375d942baeb3ded15520388f8ebb9c0cc86f8c
Signed-off-by: Jonathan Nieder <jrn@google.com>
The class FS_Win32 was always trying out to create a temporary symlink
in order to find out whether symlinks are supported. FS_Win32_Cygwin was
overwriting this method and always returned true. But when the user
running JGit does not have administrative rights then the creation of
symlinks is forbidden even if he is running on FS_Win32_Cygwin. A lot of
tests failed only on the Windows platform because of this. It was
correctly detected that FS_Win32_Cygwin is the filesystem abstraction to
be used but creation of symlinks always failed because of lacking
privileges of the user running the tests.
This fix teaches FS_Win32_Cygwin to behave like FS_Win32 and to test
whether symlinks can be created in order to find out whether symlinks
are supported.
Change-Id: Ie2394631ffc4c489bd37c3ec142ed44bbfcac726
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Remove references to the bundle org.eclipse.jgit.java7 which was removed
in 4.0.
Change-Id: I85527eb2a34bb94979fdab1311043ae77a2b5ecd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Prevent that WalkEncryptionTest fails when it can't determine the public
IP address using http://checkip.amazonws.com. Also set timeouts when
determining IP address in order to prevent long wait times during tests.
Change-Id: I1d2fe09f99df2a5f75f8077811a72fb2271cdddb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
getRef() loops over its search path to find a ref:
Ref ref = null;
for (String prefix : SEARCH_PATH) {
ref = readRef(prefix + needle, packed);
if (ref != null) {
ref = resolve(ref, 0, null, null, packed);
break;
}
}
fireRefsChanged();
return ref;
If readRef returns null (indicating that the ref does not exist), the
loop continues so we can find the ref later in the search path. And
resolve should never return null, so if we return null it should mean
we exhausted the entire search path and didn't find the ref.
... except that resolve can return null: it does so when it has
followed too many symrefs and concluded that there is a symref loop:
if (MAX_SYMBOLIC_REF_DEPTH <= depth)
return null; // claim it doesn't exist
Continue the loop instead of returning null immediately. This makes
the behavior more consistent.
Arguably getRef should throw an exception when a symref loop is
detected. That would be a more invasive change, so if it's a good
idea it will have to wait for another patch.
Change-Id: Icb1c7fafd4f1e34c9b43538e27ab5bbc17ad9eef
Signed-off-by: Jonathan Nieder <jrn@google.com>
Adds the getAttributes feature to the tree walk. The computation of
attributes needs to be done by the TreeWalk since it needs both a
WorkingTreeIterator and a DirCacheIterator.
Bug: 342372
CQ: 9120
Change-Id: I5e33257fd8c9895869a128bad3fd1e720409d361
Signed-off-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>