If the thread reading a pack file is interrupted don't invalidate that
pack file.
This could happen when Gerrit invoked JGit for computing a diff in one
thread and waited for the call to finish from another thread, with a
timeout. When the timeout was reached the "diff" thread was interrupted.
If it happened to be in an IO operation, reading a pack file, an
InterruptedIOException was thrown and the pack file was marked as
invalid and removed from the pack list.
Invalidating the pack in that case could cause the project disappearing in
Gerrit as discussed in [1] and [2].
[1] https://groups.google.com/forum/#!topic/repo-discuss/CYYoHfDxCfA
[2] https://groups.google.com/forum/#!topic/repo-discuss/ZeGWPyyJlrM
Change-Id: I2eb1f98370936b5be541d96d70c3973cbfc39238
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Now that d7a4473 removed the gitprefix property, we did not have a way to
specify the path to the Git system config file in case
discoverGitSystemConfig() fails. Fix that by introducing a member variable
that caches the result of discoverGitSystemConfig() as well as a setter
method to overwrite the content of that variable.
Change-Id: Icd965bffbe2f11b18c9505ee2ddd2afad5b64d70
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Change-Id: I31e788ee20ac3e8439559d9060d39e9792f6dc7d
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Commit d3348e introduced few errors in Eclipse.
This commit cleans up the new API:
- fixes API error in RepoCommand after moving IncludedFileReader type
- fixes unused imports in RepoCommand & RepoCommandTest
- fix javadoc errors in ManifestParser & RepoProject
- makes three (implicitly final) fields in ManifestParser final.
Change-Id: I4185f451d97039d155391f62815bb9d3afe39fa6
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
The repo xml manifest parser used in RepoCommand could also be useful for
others, so refactor to make it public.
Also this breaks backward compatibility slightly.
Change-Id: I5001bd2fe77541109fe32dbe2597a065e6ad585e
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
The only purpose of the gitprefix logic was to determine the path to the
system-wide config file. This is now done by discoverGitSystemConfig()
independent of the gitprefix, so get rid of this unused code.
Change-Id: Iaa88df9bd066dc1ed4067c18618af809e49876b3
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Change-Id: I969e26a5ab5f8ca3ab29024f405c1e34afdba493
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I4db7763826e4ada92074317d4d1c9a32299f3af8
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I700540eec06efb24eeb09bfcb40420820c32d156
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I67ec732ea2e5345a6946783f0c5ef60c07ce254e
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
- use try-with-resource where possible
- replace use of deprecated release() by close()
Change-Id: I0f139c3535679087b7fa09649166bca514750b81
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
git-core has supported this for a long time; allowing clients to
avoid progress messages from the server if they are dumping to a
pipe instead of a tty.
Avoid the two progress monitors going on side-band and expose
isQuiet() method to allow hooks to also reduce their output if
this is sensible for them.
Change-Id: I1df7e38d16765446b441366500b017a90b8ff958
In 77030a5e, AutoClosable was implemented on classes that use release().
This caused a resource leak because the ObjectReader.close method was
not calling the now deprecated release method, which is the method that
sub classes implements to release resources.
Change-Id: I247651ec8fd7ca9941d256ca46d14cc43cc35c6e
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Although the stable-4.0 branch already exists, 4.0 development is
still happening on master until IP logs are sent for review, which
will happen at the end of May.
Change-Id: I863ba85c6303f8ef2eb13bca5e2d30e5d3c58b29
Signed-off-by: Jonathan Nieder <jrn@google.com>
While trying to decide between "which matches every object" and "as it
matches every object", I became distracted and wrote both.
Change-Id: I867ce29664e661a81a9d441e59ffd0b72270dd98
Signed-off-by: Jonathan Nieder <jrn@google.com>
This will make it possible to declare a collection of objects as
ineligible for the walk en masse, for example if they are known to be
uninteresting via a bitmap.
Change-Id: I637008b25bf9fb57df60ebb2133a70214930546a
Signed-off-by: Jonathan Nieder <jrn@google.com>
The Java GC doesn't always clear these before running out of memory
and failing allocations. In practice OpenJDK 7 is leaving these live,
removing any advantage of the SoftReference to attempt to shed memory
when the GC is unable to continue allocating.
Instead follow the pattern of the DfsBlockCache and use hard refs
to the object data. Require applications to configure the cache
size more accurately given expected memory usage.
Change-Id: I87586b3e71b1cba0308a6a278d42e971be4bccd3
The LRU chain management code was broken leading to situations where
the chain was incomplete. This prevented the cache from removing
items when it exceeded its memory target, causing a leak.
One case was repeated hit on the head of the chain. moveToHead(e)
was invoked linking the head back to itself in a cycle orphaning
the rest of the table.
Add some unit tests to cover this and a few other paths.
Change-Id: Ib27486eaa1b1d2bf1c745a56d0a5832bfb029322
As Chris pointed out change I822721c76c64e614f87a080ced2457941f53adcd
slowed down merge since ANY_DIFF filter is much less efficient than the
manual detection of diffs done in ResolveMerger.processEntry() since it
avoids unnecessary filesystem calls using the git index. Hence only set
the ANY_DIFF filter on bare repositories which don't have a working tree
to scan.
To test performance I used the setup described in Chris' comment on
change I822721c76c64e614f87a080ced2457941f53adcd and modified
ResolveMerger.mergeTrees() to not add the working tree in order to
simulate merging in a bare repository.
At least on Mac I couldn't detect a speedup, with and without the
ANY_DIFF filter merge test takes an average 0.67sec.
Change-Id: I17b3a06f369cee009490f54ad1a2deb6c145c7cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Avoid always calling `sh -c umask` on startup, instead deferring
the invocation until the first time a working tree file needs to
use the execute bit. This allows servers using bare repos to avoid
a costly fork+exec for a value that is never used.
Store the umask as an int instead of two Boolean. This is slightly
smaller memory (one int vs. two references) and makes it easier for
an application to force setting the umask to a value that overrides
whatever the shell told JGit.
Simplify the code to bail by returning early when canExecute is
false, which is the common case for working tree files.
Change-Id: Ie713647615bc5bdf5d71b731a6748c28ea21c900
Applications that use a commit message once and do not
need it again can free the body to save memory. Expose
the disposeBody() methods to support this and use it in
pgm.Log which only visits each commit once.
Change-Id: I4142a0749c24f15386ee7fb119934a0432234de3
This was added a very long time ago to support the failed
DHT storage implementation. Since then no storage system
was able to make use of this API, but it pollutes internals
of the walkers.
Kill the API on ObjectReader and drop the invocations from
the walker code.
Change-Id: I36608afdac13a6c3084d7c7e0af5e0cb22900332
Previously using an ObjectWalk meant uninteresting commits may keep
their commit message buffers in memory just in case they were found to
be on the boundary and were output as UNINTERESTING for the caller.
This was incorrect inside StartGenerator. ObjectWalk hides these
internal UNINTERESTING cases from its caller unless RevSort.BOUNDARY
was explicitly set, and its false by default. Callers never see one
of these saved uninteresting commits.
Change the test to allow early dispose unless the application has
explicitly asked for RevSort.BOUNDARY. This allows uninteresting
commit buffers to be discarded and garbage collected in ObjectWalks
when the caller will never be given the RevCommit.
Change-Id: Ic1419cc1d9ee95f4d09386dd0730d54c12dcc157
Despite being the primary author of RevWalk and ObjectWalk I still
fail to remember to setRetainBody(false) in application code using
an ObjectWalk to examine the graph.
Document the default for RevWalk is setRetainBody(true), where the
application usually wants the commit bodies to display or inspect.
Change the default for ObjectWalk to setRetainBody(false), as nearly
all callers want only the graph shape and do not need the larger text
inside a commit body. This allows some code in JGit to be simplified.
Change-Id: I367e42209e805bd5e1f41b4072aeb2fa98ec9d99
That more or less defeats the purpose of using a StringBuilder.
Change-Id: I519f7bf1c9b6670e63c3714210f834ee845dc69f
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Change-Id: I3145f74ecee9f5b368e7f4b9fd7cb906f407eff5
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If UploadPack or ReceivePack has an exception record an identifier
associated with the repository as part of the log message. This can
help the HTTP admin track down the offending repository and take
action to repair the root cause.
Change-Id: I58f22b33cdb40994f044a26fba9fe965b45be51d
When DirCacheTree.contains() is called and 'aOff' is greater than 'aLen'
an ArrayIndexOutOfBoundsException was thrown. This fix makes
DirCacheTree.contains() more robust and allows parsing such index files
without throwing AIOOB.
I couldn't create a test case leading to this situation but I have seen
such situations while inspecting Bug: 465393. It seems that such
situations are created on Windows when there are invalid pathes in the
index. There may be a not yet known bug leading to such situations in
combination with invalid pathes.
Bug: 465393
Change-Id: I6535d924a22cba9a05df0ccd7e6dc2c9ddc42375
Since git-core ff5effd (v1.7.12.1) the native wire protocol transmits
the server and client implementation and version strings using
capability "agent=git/1.7.12.1" or similar.
Support this in JGit and hang the implementation data off UploadPack
and ReceivePack. On HTTP transports default to the User-Agent HTTP
header until the client overrides this with the optional capability
string in the first line.
Extract the user agent string into a UserAgent class under transport
where it can be specified to a different value if the application's
build process has broken the Implementation-Version header in the
JGit package.
Change-Id: Icfc6524d84a787386d1786310b421b2f92ae9e65
A larger than expected number of real-world repositories found on
the Internet contain invalid author, committer and tagger lines
in their history. Many of these seem to be caused by users misusing
the user.name and user.email fields, e.g.:
[user]
name = Au Thor <author@example.com>
email = author@example.com
that some version of Git (or a reimplementation thereof) copied
directly into the object header. These headers are not valid and
are rejected by a strict fsck, making it impossible to transfer
the repository with JGit/EGit.
Another form is an invalid committer line with double negative for
the time zone, e.g.
committer Au Thor <a@b> 1288373970 --700
The real world is messy. :(
Allow callers and users to weaken the fsck settings to accept these
sorts of breakages if they really want to work on a repo that has
broken history. Most routines will still function fine, however
commit timestamp sorting in RevWalk may become confused by a corrupt
committer line and sort commits out of order. This is mostly fine if
the corrupted chain is shorter than the slop window.
Change-Id: I6d529542c765c131de590f4f7ef8e7c1c8cb9db9
This error happens on nfs file system when you try to read a file that
was deleted or replaced.
When the error happens because the file was deleted, removing it from
the list is the proper way to handle the error, same use case as
FileNotFoundException. When the error happens because the file was
replaced, removing the file from the list will cause the file to be
re-read so it will get the latest version of the file.
Bug: 462868
Change-Id: I368af61a6cf73706601a3e4df4ef24f0aa0465c5
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Pack not found and pack corrupted/invalid are handled by the code (pack
is removed from the list) so logging an error and the stacktrace is
misleading because it implies that there is an action to take to fix the
error.
Lower the log level to warn and remove the stacktrace for those 2 types
of errors and keep the error log statement for any other.
Change-Id: I2400fe5fec07ac6d6c244b852cce615663774e6e
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-3.7:
Prepare 3.7.2-SNAPSHOT builds
JGit v3.7.1.201504261725-r
Revert "Let ObjectWalk.markUninteresting also mark the root tree as"
Change-Id: If1b62ff695e063d797c3d13c43e488ca56f29cbe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The Iff2de881 tried to fix missing tree ..." but introduced severe
performance degradation (>10x in some cases) when acting as server
(git push) and as client (replication). IOW cure is worse than the
disease.
This reverts commit c4797fe986.
Change-Id: I4e6056eb352d51277867f857a0cab380eca153ac
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Cached packs are only used when writing over the network or to
a bundle file and reuse validation is always disabled in these
two contexts. The client/consumer of the stream will be SHA-1
checksumming every object.
Reuse validation is most critical during local GC to avoid silently
ignoring corruption by stopping as soon as a problem is found and
leaving everything alone for the end-user to debug and salvage.
Cached packs are not supported during local GC as the bitmap rebuild
logic does not support including a cached pack in the result.
Strip out the validation and force PackWriter to always disable the
cached pack feature if reuseValidation is enabled.
Change-Id: If0d7baf2ae1bf1f7e71bf773151302c9f7887039
Sensible suggestion from Terry Parker as a late comment on
commit f2efcdc6f769d59722b17e9274932d585035cfb6.
Change-Id: I225775bfb6d3d91ae066ff00f9d80a9c02a422c2
This hint allows an underlying implementation to read more bytes when
possible and buffer them locally for future read calls to consume.
Change-Id: Ia986a1bb8640eecb91cfbd515c61fa1ff1574a6f
When a large pack (> 30% of the block cache) is being reused by
copying it pollutes the block cache with noise by storing blocks
that are never referenced again.
Avoid this by streaming the file directly from its channel onto
the output stream.
Change-Id: I2e53de27f3dcfb93de68b1fad45f75ab23e79fe7
The clone or fetch depth is a valuable bit of information
for access logging. Create a public getter to faciliate access.
A precondition check prevents unintentional misuse when the
data isn't valid yet.
Change-Id: I4603d5fd3bd4a767e3e2419b0f2da3664cfbd7f8
Signed-off-by: David Pletcher <dpletcher@google.com>
JGit hit IllegalArgumentException: invalid content length
when pushing large packs to S3.
Bug: 463015
Change-Id: Iddf50d90c7e3ccb15b9ff71233338c6b204b3648
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This was deprecated and should only be used by DirCacheCheckout and
friends. Other classes should use SystemReader.checkPath() instead.
Change-Id: I37cf753b1f081602dee9f0f47979eff39d735f92
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
- close RevWalk allocated in scan()
- replace use of deprecated ObjectReader.release() method
Change-Id: I41b2b10a1a44270a6ceaa1741e996c0921439852
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Replaces use of deprecated release() methods.
Change-Id: I0211bcf0a76a2fccc2c85fa74778e20c256984ba
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
- replaces use of deprecated ObjectInserter.release()
- auto-close TreeWalk
Change-Id: I540ee711b8c3430a71fdff07add506b7d9c039dc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Most callers/users of TemporaryBuffer are sizing the in-memory
portion large enough that most outputs fit into RAM. With this
assumption they don't pay close attention to the size of IOs
being written, as it "should" just be a copy from one byte array
to another.
Overflow sets up a local file handle, which is costly to write to
for small IO units. Wrap the local file in a BufferedOutputStream
to combine small writes together. Larger writes can still bypass the
buffer as BOS automatically avoids copying for larger writes.
Change-Id: I09f4136dd65c48830cfda86d9101bc647581018a
When reading back from an overflowed TemporaryBuffer the InputStream
must be closed to close the FileInputStream that is reading from
the backing file.
Change-Id: Id83d8f16f5b2c2618a9f841ec3508508455a6ae1
By writing the temporary overflow merge result to $GIT_DIR JGit
can ensure the same filesystem permissions apply to protect the
file contents.
If no directory is available from the repository (e.g. DfsRepository)
null will be passed and the system temporary directory will be used
instead.
Change-Id: I95532aa092676d18f1dc1e3fdbe6dcb1f91b782e
Formatting merge conflicts one byte at a time is going to be very
slow when the final OutputStream is a FileOutputStream and the JVM
is making system calls for each byte output.
When outputting a range of bytes from a byte[] the bol (beginning
of line) value only depends on the value of the last byte written.
Other bytes in the array can be passed directly to the lower stream
for more efficient output.
Change-Id: I3415f9a390ee215210a17bb5bf39164d197e1348
This should considerably speed up the treewalk on larger repositories.
Found by discussing new EGit API to support model merge in change
eda23bb556d342f421f03b93c7faa160998598aa
Change-Id: I822721c76c64e614f87a080ced2457941f53adcd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
cc: Laurent Delaigue <laurent.delaigue@obeo.fr>
This unintentionally was changed from severity debug to error which is
causing unexpected log entries.
Bug: 463349
Change-Id: I4b6d42a1420652ab6824e237bd231ba86896acbf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Bug: 390833
Change-Id: I29f7b79b241929877c93ac485c677487a91bb77b
Signed-off-by: André de Oliveira <andre.oliveira@liferay.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a non interactive rebase is launched, stopping after a conflict
should set the repository state to RepositoryState.REBASING_MERGE
instead of RepositoryState.REBASING_INTERACTIVE.
Bug: 452623
Change-Id: Ie885aab6d71dabd158a718af0d14fff643c9b850
Also-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When RecursiveMerger found that there are multiple base-commits for the
commits to be merged it tries to temporarily merge the base commits. But
if these base commits have no common predecessor there was a bug in JGit
leading to a NPE. This commit fixes this by enforcing that an empty tree
is used as base when merging two unrelated base commits.
This logic was already there when merging two commits which have no
common predecessor (ThreeWayMerger.mergeBase()). But the code which was
computing a new temporary base commit in case of criss-cross merges
didn't take care to pick an empty tree when no common predecessor can be
found.
Bug: 462671
Change-Id: Ibd96302f5f81383f36d3b1e3edcbf5822147b1a4
As we moved minimum Java version to 7 we don't need a separate bundle
and feature for JGit features depending on Java 7 anymore.
Change-Id: Ib5da61b0886ddbdea65298f1e8c6d65c9879ced1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This reverts commit 6bc48cdc62.
Until git v1.7.10.2~29^2~1 (builtin/merge.c: reduce parents early,
2012-04-17), C git merge would make merge commits with duplicate parents
when asked to with a series of commands like the following:
git checkout origin/master
git merge --no-ff origin/master
Nowadays "git merge" removes redundant parents more aggressively
(whenever one parent is an ancestor of another and not just when
duplicates exist) but merges with duplicate parents are still permitted
and can be created with git fast-import or git commit-tree and history
viewers need to be able to cope with them.
CommitBuilder is an interface analagous to commit-tree, so it should
allow duplicate parents. (That said, an option to automatically remove
redundant parents would be useful.)
Reported-by: Dave Borowitz <dborowitz@google.com>
Change-Id: Ia682238397eb1de8541802210fa875fdd50f62f0
Signed-off-by: Jonathan Nieder <jrn@google.com>
The block pointer list may have been relatively large, so no need to
make more garbage. Instead, just clear the list and null out all the
elements.
Another possible motivation: a caller may have provided an inaccurate
estimated size, so the list might have been resized several times. If
the list is reused later for a similarly underestimated workload, this
fix will prevent additional resizing on subsequent usages.
Change-Id: I511675035dcff1117381a46c294cc11aded10893
Callers may wish to use TemporaryBuffer as an essentially unbounded
buffer by passing Integer.MAX_VALUE as the size. (This makes it
behave like ByteArrayOutputStream, only without requiring contiguous
memory.) Unfortunately, it was always allocating an array in the
backing block pointer list to hold enough blocks to MAX_VALUE--all
262,016 of them. It wasn't allocating the blocks themselves, but this
array was still extremely wasteful, using about 2MiB of memory on a
64-bit system.
Tweak the interface to specify an estimated size, and only allocate
the block pointer list enough entries to hold that size. It's an
ArrayList, so if that estimate was wrong, it'll grow. We assume the
cost of finding enough contiguous memory to grow that array is
acceptable.
While we're in there, fix an off-by-one error: due to integer division
we were undercounting the number of blocks needed to store n bytes of
data as (n / SZ).
Change-Id: I794eca3ac4472bcc605b3641e177922aca92b9c0
Since 27ae8bc65 Git has implemented AutoCloseable, which means Eclipse
may warn if close() is never called on a Git instance. For example,
the following would result in a resource warning:
Repository repo = openRepository(foo);
Git git = new Git(repo);
try {
git.someCommand().call();
} finally {
repo.close();
}
(The same warning would occur if repo were created in a try-with-
resources block.)
The "obvious" fix is to open git in a try-with-resources block:
try (Repository repo = openRepository(foo);
Git git = new Git(repo)) {
git.someCommand().call();
}
Unfortunately, this construction was subtly broken: it would call both
git.close() and repo.close(), but git.close() would call repo.close()
again. Depending on the repository implementation, this might or might
not be ok. If it's not ok, it might not immediately cause an error, if
the reference count of repo was >2 at the time of closing.
Of course, explicitly calling git.close() followed by repo.close() in
two finally blocks has had the same double-closing problem since
forever. But the problem became worse when Git started implementing
AutoCloseable, because now Eclipse is _actively encouraging_
developers to change working code into broken code.
To work around this, keep track in Git's constructor of whether the
repository was passed in or opened at construction time, and only
close the repository if it was opened by Git.
Note that in the original example, there was not _actually_ a resource
leak, since repo was closed exactly once; git did not _need_ to be
closed in this case. But at least fixing this false-positive warning
no longer introduces a real bug.
Change-Id: Ie927a26ce3ae2bf8c3ef5cb963a60847067db95a
For some specific file, MyersDiff goes into an infinite loop[1]. Since
this problem is hard to reproduce and possibly harder to fix, this
change makes the MyersDiff interruptible so the diff can be aborted at
least when such infinite loop happens.
[1]https://bugs.eclipse.org/bugs/show_bug.cgi?id=444623
Change-Id: I6e006ccb122d1e68c9846a24d5399d94776c2858
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
When setting the parents of a commit with setParentIds() or
addParentId() it should be checked that we don't have duplicate parents.
An IllegalArgumentException should be thrown in this case.
Change-Id: I9fa9f31149b7732071b304bca232f037146de454
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This flushed out a number of bugs in the way DfsRefUpdate, or at least
the InMemoryRepository implementation, processes symrefs. These have
been fixed, to an extent, in InMemoryRepository, but other
implementations may still suffer from these bugs.
Change-Id: Ifd12115a0060b9ff45a88d305b72f91ca0472f9a
The TreeWalk(ObjectReader) constructor is explicitly to handle the case
where the caller is responsible for opening and closing the reader.
The reader should only be closed when it was created in the
TreeWalk(Repository) constructor.
Change-Id: I627681be80d69ea549f953255a64c7b3b68bcec9
The RevWalk(ObjectReader) constructor is explicitly to handle the case
where the caller is responsible for opening and closing the reader.
The reader should only be closed when it was created in the
RevWalk(Repository) constructor.
Change-Id: Ic0d595dc8d10de79e87549546c6c5ea2dc617e9b
PackFile is held by the block cache and cannot be auto closed in a
try-with-resources statement. Remove the interface as JGit does
explicit management of the instances.
ObjectDatabase and RefDatabase are internal details of Repository
and are managed with the Repository. Marking them AutoCloseable
provides no value to the library or an application using the API.
Change-Id: Ibee19eadd66233e6666b601583daa1834a7778f1
Reference equality (!= or ==) cannot be used to check for
String equality. String objects are not necessarily interned
to the same instance.
Use .isEmpty() since the function only cares about an empty
string and does not need to test a specific string value.
Change-Id: If530cb59666a8196d57d2348c893706a517ea541
The %x format specifier is not valid for a byte array.
This patch fixes a bug that would cause an IllegalFormatConversionException.
Change-Id: I025975eca7b2f10bbafa39f5519f8668e6536541
Signed-off-by: David Pletcher <dpletcher@google.com>
Native git supports "git describe --long". This will enforce returning a
long description of a commit even if a tag is directly pointing to the
commit (in contrast to just returning the tag name as it is now). This
commit teaches JGits DescribeCommand and the describe command in the pgm
package to support "--long".
Bug: 460991
Change-Id: I65e179b79e89049c6deced3c71cb3ebb08ed0a8f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The index provides access to a list of objects in a pack.
This will be helpful for repository integrity checking.
Change-Id: I435eeeb3fe1b1f5632d40528936416e97491d412
Signed-off-by: David Pletcher <dpletcher@google.com>
This hook uses the file .git/COMMIT_EDITMSG to receive and potentially
modify the commit message.
Change-Id: Ibe2faadfb5d3932a5a3da2252d8156c4c04856c7
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Hooks are now obtained via a convenient API like git commands, and
callers don't have to check for their existence.
The pre-commit hook has been updated accordingly.
Change-Id: I3383ffb10e2f3b588d7367b9139b606ec7f62758
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This allows for testing arbitrary sets of push/fetch hooks (e.g.
PreReceiveHook) without depending on either an external protocol (e.g.
HTTP) or the local filesystem.
Change-Id: I4ba2fff9c8a484f990dea05e14b0772deddb7411
TransportLocal knows how to spin up a thread to allow two repositories
in the same process to communicate using the wire protocol. However,
it is still tied to local on-disk filesystems, and needs to be able to
fork processes if not using the default git-{upload,receive}-pack
implementation.
Extract out the connection classes so they can be used by other
transport implementations.
Change-Id: I5db59086740735508c2e70a597c2d1a89014b072
* stable-3.7:
Prepare 3.7.1-SNAPSHOT builds
JGit v3.7.0.201502260915-r
Read user.name and email from environment first
Provide more details in exceptions thrown when packfile is invalid
Change-Id: I427f861c6bc94da5e3e05dbbebbf0ad15719a323
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This was missed in change I249869cadb2d55aef016371b9311b8583591b9cf
Change-Id: I19c9d4c04b6aa92b9e04c192dee70775d6985b58
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-3.7:
Add log4j and slf4j-log4j bridge to jgit feature
Use slf4j to log instead of printing to System.err
Use Target Platform Definition DSL to generate target platforms
Change-Id: Ic8779868150c910fa55fd20348e35723e6add0f1
According to [1] user name and email are taken first from the
environment variables:
GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL
GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL
In case (some of) these environment variables are not set, the
information is taken from the git configuration.
JGit doesn not yet support the environment variables GIT_AUTHOR_DATE and
GIT_COMMITTER_DATE.
[1] https://www.kernel.org/pub/software/scm/git/docs/git-commit-tree.html#_commit_information
Bug: 460586
Change-Id: I3ba582b4ae13674cf319652b5b13ebcbb96dd8ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Push certificates ("git push --signed") have been part of
git-core since version 2.2.0 (released Nov 26 2014). We also
want to support that feature.
This is not complete and is lacking the actual functionality
to validate the signature for now.
Change-Id: I249869cadb2d55aef016371b9311b8583591b9cf
Signed-off-by: Stefan Beller <sbeller@google.com>
Current ArchiveCommand design doesn't allow to pass in options to
underlying stream implementations. To overcome this, client has to
implement custom format implementation (it cannot be derived from
the existing one, because the classes are marked as final), and set
the options using ThreadLocal, before the method
ArchiveOutputStream createArchiveOutputStream(OutputStream s)
is get called.
This change extends the ArchiveCommand.Format by allowing to pass
option map during creation of ArchiveOutputStream.
ArchiveCommand is extended correspondingly. That way client can
easily pass options to the underlying streams:
Map<String, Object> level = ImmutableMap.<String, Object> of(
"level", new Integer(9));
new ArchiveCommand(repo)
.setFormat("zip")
.setFormatOptions(level)
.setTree(tree)
.setPaths(paths)
.setPrefix(prefix)
.setOutputStream(sidebandOut)
.call();
Change-Id: I1d92a1e5249117487da39d19c7593e4b812ad97a
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Mention packfile path in exceptions thrown when we detect that a
packfile is invalid and make excplicit that corrupt packs are removed
from the pack list.
Change-Id: I454ada5f8e69307d3f34d1c1b8f3cb87607ddf35
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Implement AutoClosable and deprecate the old release() method to give
JGit consumers some time to adapt.
Bug: 428039
Change-Id: Id664a91dc5a8cf2ac401e7d87ce2e3b89e221458
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-3.7:
Add option --orphan for checkout
Prepare post 3.7.0.201502031740-rc1 builds
JGit v3.7.0.201502031740-rc1
Support for the pre-commit hook
Fix FileUtils.testRelativize_mixedCase which failed on Mac OS X
Add a hook test
Introduce hook support into the FS implementations
If a pack isn't found on disk remove it from pack list
Conflicts:
org.eclipse.jgit.java7.test/META-INF/MANIFEST.MF
Change-Id: I936acd24d47b911fa30ab29856094e1b2c6ac3db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Introduce support for the pre-commit hook into JGit, along with the
--no-verify commit command option to bypass it when rebasing /
cherry-picking.
Change-Id: If86df98577fa56c5c03d783579c895a38bee9d18
Signed-off-by: Laurent Goubet <laurent.goubet@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This introduces the background plumbing necessary to run git hooks from
JGit. This implementation will be OS-dependent as it aims to be
compatible with existing hooks, mostly written in Shell. It is
compatible with unix systems and windows as long as an Unix emulator
such as Cygwin is in its PATH.
Change-Id: I1f82a5205138fd8032614dd5b52aef14e02238ed
Signed-off-by: Laurent Goubet <laurent.goubet@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If accessing a pack throws FileNotFoundException the pack was deleted
and we need to remove it from the pack list. This can be caused e.g. by
git gc.
Change-Id: I5d10f87f364dadbbdbfb61b6b2cbdee9c7457f3d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
ObjectInserter recently learned to read back inserted objects before
they have been flushed. It is in general unsafe to create refs to such
objects, but it is now much more possible to do so, by passing "new
RevWalk(inserter.newReader())" into RefUpdate#execute(RevWalk).
We can't change the RefUpdate interface to remove execute(RevWalk);
nor would we necessarily want to, for performance reasons. And in any
case, RefUpdate#safeParse explicitly ignores MissingObjectExceptions.
But we can enforce object existence in InMemoryRepository, which will
allow callers using this class in their tests to ensure they are using
the RefDatabase correctly.
Change-Id: I5c696ba23bcd2a536a0512fa7f5b6130961905c5
In case the index contains wrong tree extensions don't throw a
ArrayIndexOutOfBounds exception but revalidate the tree extension.
It happened that the git index written by Git for Windows contained valid
(means entryCount>0) tree extensions for pathes which are not existing
in the index. Native git handles this inconsistency silently but JGit
was crashing with a ArrayIndexOutOfBounds exception. Teach JGit to
better recognize such cases and revalidate such extensions.
It's hard to write a test because JGit doesn't write such extensions. It
only reads, validates and makes use of them. But the bug tells how to
create such situations.
Bug: 457152
Change-Id: Id3ffd7dc7ae1c55674d88bf1b43953234fe0b68d
* changes:
Document that repo returned by SubmoduleAddCommand needs to be closed
Document that Git instance returned by CloneCommand needs to be closed
This way we can ensure that the same command instance can't be used
concurrently in multiple threads.
Bug: 458023
Change-Id: I4884a1ef2f609f9fb24dda4bd5819dffb9f174b6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* origin/stable-3.6:
Prepare 3.6.3-SNAPSHOT builds
JGit v3.6.2.201501210735-r
Don't remove pack from pack list for problems which could be transient
Log reason for ignoring pack when IOException occurred
Change-Id: I61141b52839511d58e5a5b193bfde31e9f444a6c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If we hit a corrupt object or invalid pack remove the pack from the pack
list. Other IOException could be transient hence we should not remove
the pack from the list to avoid the problem reported on the Gerrit list
[1]. It looks like in the reported case the pack was removed from the
pack list causing MissingObjectExceptions which disappear when the
server is restarted.
[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU
Change-Id: I331626110d54b190e46cddc2c40f29ddeb9613cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This should help to identify the root cause of the problem discussed on
the Gerrit list [1].
[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU
Change-Id: I871f70e4bb1227952e1544b789013583b14e2b96
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This new git capability isn't yet in the wild in the git-core camp.
Hence we can silence this API warning.
Change-Id: I9e030959f658c58344b04ef2f9edd28060f0c964
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The atomic feature is now cooking in -next in git-core. Very rarely
features are ejected from the the next branch in git-core, so I consider
it reasonable to come up with this patch now to make the 2 implementations
interoperable.
Change-Id: I806a8ae3c045ca5936f69cb903baf9b99ee39181
Signed-off-by: Stefan Beller <sbeller@google.com>
Core classes to parse and process .gitattributes files including
support for reading attributes in WorkingTreeIterator and the
dirCacheIterator.
The implementation follows the git ignore implementation. It supports
lazy reading attributes while walking the working tree.
Bug: 342372
CQ: 9078
Change-Id: I05f3ce1861fbf9896b1bcb7816ba78af35f3ad3d
Also-by: Marc Strapetz <marc.strapetz@syntevo.com>
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Also-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Arthur Daussy <arthur.daussy@obeo.fr>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
* stable-3.6:
Prepare 3.6.2-SNAPSHOT builds
JGit v3.6.1.201501031845-r
Trim author/committer name and email in commit header
Rename detection should canonicalize line endings
PathMatcher should respect "assumeDirectory" flag
Change-Id: Idd48c6d94cf1ab09abc07f70d50890b1b78e1833
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
C Git trims name and email before inserting them into the commit header
so that " A U Thor " and " author@example.com " becomes
"A U Thor <author@example.com>" with a single separating space.
This changes PersonIdent#toExternalString() to trim name and email
before concatenating them.
Change-Id: Idd77b659d0db957626824f6632e2da38d7731625
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Native Git canonicalizes line endings when detecting
renames, more specifically it replaces CRLF by LF.
See: hash_chars in diffcore-delta.c
Bug: 449545
Change-Id: Iec2aab12ae9e67074cccb7fbd4d9defe176a0130
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The path matcher should not fail if the rule ends with trailing slash,
target pattern does not ends with the slash and the "assumeDirectory"
flag is set.
E.g. */bin/ should also match a/bin if this pattern is threated as
directory by WorkingTreeIterator (FileMode.TREE).
The old code/tests have never tested directory rules with patterns
*without* trailing slashes but with the "assumeDirectory" flag set.
Unfortunately this is exactly what WorkingTreeIterator does... The tests
are changed to test *both* cases now (with trailing slash and without)
if the target pattern has trailing slash (represents directory).
Bug: 454672
Change-Id: I621c1644d9e94df3eb9f6f09c6de0fe51f0950a4
Also-by: Markus Duft <markus.duft@salomon.at>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
* stable-3.6:
Prepare 3.6.1-SNAPSHOT builds
JGit v3.6.0.201412230720-r
[pgm] Add option --bare to clone command
[pgm] Implement clone using CloneCommand
Fix junit tests under windows when the platform is explicitly changed
Fix unit tests for windows by explicitly closing test repos
[pgm] Add option --tags for ls-remote
[pgm] Add option --heads for ls-remote
[pgm] Use LsRemoteCommand to implement ls-remote and add a test
Change-Id: I8f31e76cb7e9416919f37e02c7e51ab1d221df40
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
SystemReader used a chached ObjectChecker which was instantiated only
once. But in case of unit tests where we can change the platform
dynamically (e.g. MockSystemReader.setWindows()) this is wrong and
caused DirCacheCheckoutMaliciousPathTest.
testMaliciousAbsoluteCurDrivePathWindowsOnUnix() to fail. This change
allows
user of SystemReader to force the creation of a new ObjectChecker.
MockSystemReader.setWindows() and .setUnix() make use of this feature.
Change-Id: I87458d1dc63c1f5c18979f972b1c1f0d670a9ed8
* stable-3.6: (26 commits)
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Support the new repository layout for submodules
Allow explicit configuration of git directory in CloneCommand
Allow explicit configuration of git directory in InitCommand
Fix tests on windows by closing repos
RepoCommand should close opened repos
Fix LocalDiskRepositoryTestCase to create correct type of repos
Prevent NPE if ref can't be resolved when executing ReflogCommand
Fix DirCacheCheckout to set correct file length if core.autocrlf=true
CheckoutCommand: Fix checking out ours/theirs when no base stage exists
Make sure modifications to config-param trustFolderStat are detected
Apache HttpClientConnection: replace calls to deprecated LocalFile()
...
Change-Id: Ife6f21d64a148dbb0d0d9055356b1568188806fe
* stable-3.5:
JGit v3.5.3.201412180710-r
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: Iee8acbaa9d4d9047b550641db1b8845d64530785
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-3.4:
JGit v3.4.2.201412180340-r
ObjectChecker: Disallow names potentially mapping to ".git" on HFS+
ObjectChecker: Disallow Windows shortname "GIT~1"
ObjectChecker: Disallow ".git." and ".git<space>"
Always ignore case when forbidding .git in ObjectChecker
DirCache: Refuse to read files with invalid paths
DirCache: Replace isValidPath with DirCacheCheckout.checkValidPath
Replace "a." with "a-" in unit tests
Apache HttpClientConnection: replace calls to deprecated LocalFile()
Fix two nits about DirCacheEntry constructors
Detect buffering failures while writing rebase todo file
Deprecate TemporaryBuffer.LocalFile without parent directory
Switch FileHeader.extractFileLines to TemporaryBuffer.Heap
AmazonS3: Buffer pushed pack content under $GIT_DIR
DirCache: Buffer TREE extension to $GIT_DIR
Change-Id: I398cf40b006a05a6537788fc6eb1f84df1ed8814
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Mac's HFS+ folds concatentations of ".git" and ignorable Unicode
characters [1] to ".git" [2]. Hence we need to disallow all names which
could potentially be a shortname for ".git". Example: in an empty
directory create a folder ".g\U+200Cit". Now you can't create another
folder ".git".
The following characters are ignorable Unicode which are ignored on
HFS+:
unicode hex name
-------------------------------------------------
U+200C 0xe2808c ZERO WIDTH NON-JOINER
U+200D 0xe2808d ZERO WIDTH JOINER
U+200E 0xe2808e LEFT-TO-RIGHT MARK
U+200F 0xe2808f RIGHT-TO-LEFT MARK
U+202A 0xe280aa LEFT-TO-RIGHT EMBEDDING
U+202B 0xe280ab RIGHT-TO-LEFT EMBEDDING
U+202C 0xe280ac POP DIRECTIONAL FORMATTING
U+202D 0xe280ad LEFT-TO-RIGHT OVERRIDE
U+202E 0xe280ae RIGHT-TO-LEFT OVERRIDE
U+206A 0xe281aa INHIBIT SYMMETRIC SWAPPING
U+206B 0xe281ab ACTIVATE SYMMETRIC SWAPPING
U+206C 0xe281ac INHIBIT ARABIC FORM SHAPING
U+206D 0xe281ad ACTIVATE ARABIC FORM SHAPING
U+206E 0xe281ae NATIONAL DIGIT SHAPES
U+206F 0xe281af NOMINAL DIGIT SHAPES
U+FEFF 0xefbbbf ZERO WIDTH NO-BREAK SPACE
[1] http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf#G40025http://www.unicode.org/reports/tr31/#Layout_and_Format_Control_Characters
[2] http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html#UnicodeSubtleties
Change-Id: Ib6a1dd090b2649bdd8ec16387c994ed29de2860d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows creates shortnames for all non-8.3 files (see [1]). Hence we
need to disallow all names which could potentially be a shortname for
".git". Example: in an empty directory create a folder "GIT~1". Now you
can't create another folder ".git".
The path "GIT~1" may map to ".git" on Windows. A potential victim to
such an attack first has to initialize a git repository in order to
receive any git commits. Hence the .git folder created by init will get
the shortname "GIT~1". ".git" will only get a different shortname if the
user has created a file "GIT~1" before initialization of the git
repository.
[1] http://en.wikipedia.org/wiki/8.3_filename
Change-Id: I9978ab8f2d2951c46c1b9bbde57986d64d26b9b2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Windows treats "foo." and "foo " as "foo". The ".git" directory is
special, as it contains metadata for a local Git repository. Disallow
variations that Windows considers to be the same.
Change-Id: I28eb48859a95a89111b4987c91de97557e3bb539
The component name ".GIT" inside a tree entry could confuse a
case insensitive filesystem into looking at a submodule and
not a directory entry.
Disallow any case permutations of ".git" to prevent this
confusion from entering a repository and showing up at a
later date on a case insensitive system.
Change-Id: Iaa3f768931d0d5764bf07ac5f6f3ff2b1fdda01b
If the DirCache contains a path that is known to be invalid, refuse to
read the DirCache into memory. This avoids confusing errors later if
an invalid path read from the DirCache were to be passed into a new
DirCacheEntry constructor.
Change-Id: Ic033d81e23a5fbd554cc4dff80a232504562ffa8
isValidPath is an older simple form of the validation performed by
checkValidPath. Use the latter as it more consistently matches
git-core's validation rules.
By running the same validation as fsck, callers creating an entry
for the DirCache are more likely to learn early they are trying
to build trees that will fail fsck.
Change-Id: Ibf5ac116097156aa05c18e231bc65c0854932eb1
When updating a submodule (e.g. during recursive clone) the repository
for the submodule should be located at <gitdir>/modules/<submodule-path>
whereas the working tree of the submodule should be located at
<working-tree>/<submodule-path> (<gitdir> and <working-tree> are
associated to the containing repository). Since CloneCommand has learned
about specifying a separate gitdir this is easy to implement in
SubmoduleUpdateCommand.
Change-Id: I9b56a3dfa50f97f6975c2bb7c97b36296f331b64
This feature is needed to support the new submodule layout where the
.git folder of the submodules is under .git/modules/<submodule>.
Change-Id: If5f13426cfd09b7677e23478e9700c8c25a6dae5
Matthias Sohn