Exposing isEmpty, getLengthA, getLengthB make it easier to examine
the state of an edit and work with it from higher level code.
The before and after cut routines make it easy to split an edit
that contains another edit, such as to decompose a REPLACE that
contains a common sequence within it.
Change-Id: Id63d6476a7a6b23acb7ab237d414a0a1a7200290
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We shouldn't escape non-special ASCII characters such as '@' or '~'.
These are valid in a path name on POSIX systems, and may appear as
part of a path in a GNU or Git style patch script. Escaping them
into octal just obfuscates the user's intent, with no gain.
When parsing an escaped octal sequence, we must parse no more
than 3 digits. That is, "\1002" is actually "@2", not the Unicode
character \u0202.
Change-Id: I3a849a0d318e69b654f03fd559f5d7f99dd63e5c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we get an exception while indexing the incoming pack, its likely
a stream corruption. We already report an error to the client, but
we eat the stack trace, which makes debugging issues related to a
bug inside of JGit nearly impossible. Rethrow it under a new type
UnpackException, so embedding servers or applications can catch the
error and provide it to a human who might be able to forward such
traces onto a JGit developer for evaluation.
Change-Id: Icad41148bbc0c76f284c7033a195a6b51911beab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Applications just want a quick way to configure our diff
implementation, and then just want to use it without a lot of fuss.
Move all of the rename detection logic and path following logic
out of our pgm package and into DiffFormatter itself, making it
much easier for a GUI to take advantage of the features without
duplicating a lot of code.
Change-Id: I4b54e987bb6dc804fb270cbc495fe4cae26c7b0e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Create a new 'org.eclipse.jgit.api.errors' package to contain
exceptions related to using the Git porcelain API.
Change-Id: Iac1781bd74fbd520dffac9d347616c3334994470
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
When adding or deleting a file, we shouldn't ever prefix /dev/null
with the a/ or b/ prefixes. Doing so is a mistake and confuses a
patch parser which handles /dev/null magically, while a/dev/null is
a file called null in the dev directory of the project.
Also when adding or deleting the "diff --git" line has the "real"
path on both sides, so we should see the following when adding the
file called foo:
diff --git a/foo b/foo
--- /dev/null
+++ b/foo
The --- and +++ lines do not appear in a pure rename or copy delta,
C Git diff seems to omit these, so we now omit them as well. We also
omit the index line when the ObjectIds are exactly equal.
Change-Id: Ic46892dea935ee8bdee29088aab96307d7ec6d3d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Currently, if a branch is created that has special chars ('#' in the bug),
Config will surround the subsection name with double quotes during
it's toText method which will result in an invalid file after saving the
Config.
Bug: 318249
Change-Id: I0a642f52def42d936869e4aaaeb6999567901001
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Extended flags are processed and available via DirCacheEntry's
new isSkipWorkTree() and isIntentToAdd() methods. "resolve-undo"
information is completely ignored since its an optional extension.
Change-Id: Ie6e9c6784c9f265ca3c013c6dc0e6bd29d3b7233
The merge algorithm was reporting conflicts which where to big.
Example: The common base was "ABC", the "ours" version contained
"AB1C" (the addition of "1" after pos 2) and the "theirs" version also
contained "AB1C". We have two potentially conflicting edits in the
same region which happen to bring in exactly the same content. This
should not be a conflict - but was previously reported as
"AB<<<1===1>>>C".
This is fixed by checking every conflicting chunk whether the
conflicting regions have a common prefix or suffix and by removing
this regions from the conflict.
Change-Id: I4dc169b8ef7a66ec6b307e9a956feef906c9e15e
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This adds the first merge strategy to JGit which does real
content-merges if necessary. The new merge strategy "resolve" takes as
input three commits: a common base, ours and theirs. It will simply takeover
changes on files which are only touched in ours or theirs. For files
touched in ours and theirs it will try to merge the two contents
knowing taking into account the specified common base.
Rename detection has not been introduced for now.
Change-Id: I49a5ebcdcf4f540f606092c0f1dc66c965dc66ba
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
On Mac OS X MyerDiffPerformanceTest was failing since during the
first few tests the JIT compiler is running in parallel slowing down
the tests. When setting the JVM option -Xbatch forcing the JIT to do
its work prior to running the code this effect can be avoided. Instead
we chose to run some tests without recording prior to the recorded
tests since relying on -X JVM parameters isn't portable across JVMs.
Use 10k * powers of 2 as sample size instead of odd numbers used
before and also improve formatting of performance readings.
Bug: 323766
Change-Id: I9a46d73f81a785f399d3cf5a90c8c0516526e048
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Use 3 different types of LargeObjectException for the 3 major ways
that we can fail to load an object. For each of these use a unique
string translation which describes the root cause better than just
the ObjectId.name() does.
Change-Id: I810c98d5691b74af9fc6cbd46fc9879e35a7bdca
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add an explicit test case to check that we don't
overwrite dirty files in case Head & Index are
equal.
Change-Id: I6266d0a449e55369d2d0a048694dca5565c5fcf3
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This reverts commit db4c516f67 since
it breaks compatibility with Eclipse 3.5 which can no longer import
the projects
Bug: 323390
Change-Id: I3cc91364a6747cfcb4c611a9be5258f81562f726
A tag command is added to the Git porcelain API. Tests were
also added to stress test the tag command.
Change-Id: Iab282a918eb51b0e9c55f628a3396ff01c9eb9eb
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Implementation of a checkout (or 'git read-tree') operation which
works together with DirCache. This implementation does similar things
as WorkDirCheckout which main problem is that it works with deprecated
GitIndex. Since GitIndex doesn't support multiple stages of a file
which is required in merge situations this new implementation is
required to enable merge support.
Change-Id: I13f0f23ad60d98e5168118a7e7e7308e066ecf9c
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Otherwise PackFileTest.testDelta_LargeObjectChain() reproducibly
fails with OutOfMemoryError on Mac OS X 10.6.4.
Change-Id: I6a55ff9ba181102606a0d99ffd52392a1615a422
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Updates the project level settings to run the formatter
on save on only on the edited lines.
Change-Id: I26dd69d0c95e6d73f9fdf7031f3c1dbf3becbb79
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
We should use JUnit4 for tests. This patch updates
the MANIFEST.MF and respective launch configurations.
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
PersonIdent should be parsable for an invalid commit which
contains multiple authors, like "A <a@a.org>, B <b@b.org>".
PersonIdent(String) constructor now delegates to
RawParseUtils.parsePersonIdent().
Change-Id: Ie9798d36d9ecfcc0094ca795f5a44b003136eaf7
Because we are using the large stream size, we have to be
above the STREAM_THRESHOLD constant, which I just increased.
Change-Id: I6f10ec8558d9f751d4b547fcae05af94f1c8866b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We can now resolve expressions that reference a path within a
commit, designating a specific revision of a specific tree or
file in the project.
Change-Id: Ie6a8be629d264d72209db894bd680c5900035cc0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We now match on the -gABBREV style output created by git describe
when its describing a non-tagged commit, and resolve that back to
the full ObjectId using the abbreviation resolution feature that
we already support.
Change-Id: Ib3033f9483d9e1c66c8bb721ff48d4485bcdaef1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Calling it by the old numerical numbering system makes it really
hard to find the test that tests Repository.resolve(String).
Change-Id: I92d0ecbc8d66ce21bfed08888eeedf1300ffa594
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Its wrong to return null if we are resolving an abbreviation and we
have proven it matches more than one object. We know how to resolve
it if we had more nybbles, as there are two or more objects with the
same prefix. Declare that to the caller quite clearly by giving them
an AmbiguousObjectException.
Change-Id: I01bb48e587e6d001b93da8575c2c81af3eda5a32
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some algorithms are coded in a way that requires us to provide them
the entire object contents as a contiguous byte array. The parsers
in RevCommit and RevTag, or our RawText objects are really good
examples of these.
Instead of duplicating this logic everywhere, lets put it into the
base ObjectLoader type. That way the caller only needs to give us
their upper size bound, and we'll do the rest of the heavy work to
figure out if the object still fits within that bound, and get them
an array that has the complete contents.
Change-Id: Id95a7f79d2b97e39f6949370ccca2f2c9cfb1a0f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
If we can't resolve an abbreviation, it might be because there is
a new pack file we haven't picked up yet. Try scanning the packs
again and recheck each pack if there were differences from the last
scan we did.
Because of this, we don't have to open a pack during the test where
we generate a pack on the fly. We'll miss on the first loop during
which the PackList is the NO_PACKS magic initialization constant,
and pick up the newly created index during this retry logic.
Change-Id: I7b97efb29a695ee60c90818be380f7ea23ad13a3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ObjectReader implementations are now responsible for creating the
unique abbreviation of an ObjectId, or for resolving an abbreviation
back to its full form. In this latter case the reader can offer up
multiple candidates to the caller, who may be able to disambiguate
them based on context.
Repository.resolve() doesn't take multiple candidates into account
right now, but it could in the future by looking for a remaining
^0 or ^{commit} suffix and take an expansion if there is only one
commit that matches the input abbreviation. It could also use
the distance from an annotated tag to resolve "tag-NNN-gcommit"
style strings that are often output by `git describe`.
Change-Id: Icd3250adc8177ae05278b858933afdca0cbbdb56
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ObjectWriter is a deprecated API that people shouldn't be using.
So get rid of it in favor of the ObjectInserter API.
Change-Id: I6218bcb26b6b9ffb64e3e470dba5dca2e0a62fd4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These objects should be responsible for their own formatting,
rather than delegating it to some obtuse type called ObjectInserter.
While we are at it, simplify the way we insert these into a database.
Passing in the type and calling format in application code turned
out to be a huge mistake in terms of ease-of-use of the insert API.
Change-Id: Id5bb95ee56aa2a002243e9b7853b84ec8df1d7bf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since these types no longer support reading, calling them a Builder
is a better description of what they do. They help the caller to
build a commit or a tag object.
Change-Id: I53cae5a800a66ea1721b0fe5e702599df31da05d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
WorkingTreeIterator now optionally performs CRLF to LF conversion for
text files. A basic framework is left in place to support enabling
(or disabling) this feature based on gitattributes, and also to
support the more generic smudge/clean filter system. As there is
no gitattribute support yet in JGit this is left unimplemented,
but the mightNeedCleaning(), isBinary() and filterClean() methods
will provide reasonable places to plug that into in the future.
[sp: All bugs inside of WorkingTreeIterator are my fault, I wrote
most of it while cherry-picking this patch and building it on
top of Marc's original work.]
CQ: 4419
Bug: 301775
Change-Id: I0ca35cfbfe3f503729cbfc1d5034ad4abcd1097e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Callers might have a canonical tag encoding on hand that they
wish to convert into a clean structure for presentation purposes,
and the object may not be available in a repository. (E.g. maybe
its a "draft" tag being written in an editor.)
Change-Id: I387a462afb70754aa7ee20891e6c0262438fdf32
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Callers might have a canonical commit encoding on hand that they
wish to convert into a clean structure for presentation purposes,
and the object may not be available in a repository. (E.g. maybe
its a "draft" commit being written in an editor.)
Change-Id: I21759cff337cbbb34dbdde91aec5aa4448a1ef37
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The Tag class now only supports the creation of an annotated tag
object. To read an annotated tag, applictions should use RevTag.
This permits us to have exactly one implementation, and RevTag's
is faster and more bug-free.
Change-Id: Ib573f7e15f36855112815269385c21dea532e2cf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The Commit class now only supports the creation of a commit object.
To read a commit, applictions should use RevCommit. This permits
us to have exactly one implementation, and RevCommit's is faster
and more bug-free.
Change-Id: Ib573f7e15f36855112815269385c21dea532e2cf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ReadTreeTests relied on Repository.getIndex() which on
platforms which coarse FileSystemTimers failed to detect
index modifications. By explicitly reloading and writing
the index this problem is solved.
Change-Id: I0a98babfc2068a3b6b7d2257834988e1154f5b26
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When RevObject overrode equals() to provide only reference equality
we used to need to convert a RevObject into an ObjectId by copy()
just to use standard Java tools like JUnit assertEquals(), or to
use contains() or get() on standard java.util collection types.
Now that we have removed this override and made ObjectId's equals()
final (preventing any of this mess in the future), some copy()
calls are unnecessary. Anytime the value is being used as an input
to a lookup routine, or to an equals, we can avoid the copy().
However we still want to use copy() anytime we are given an ObjectId
that may exist long-term, where we don't want the high cost of the
additional storage from a RevCommit extension. So we can't remove
all uses of copy(), just some of them.
Change-Id: Ief275dace435c0ddfa362ac8e5d93558bc7e9fc3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Configuration change events were not being triggered, now they are
forwarded from the FileConfig up to the Repository's listeners.
Change-Id: Ida94a59f5a2b7fa8ae0126e33c13343275483ee5
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
There was a duplicated getter and setter for tagger in Tag.
There's no needed to have two getters and setters that represent
the same things. The appropriate tests were updated also.
Change-Id: If46dc00c4c0f31ea4234c6d3bda3c03e6ebbafac
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
indexState() encodes the complete state of the index
into one readable String. This helps to write tests
against the index. indexState() is enhanced to optionally
also contain the content of the files in the index.
Change-Id: Ie988f93768d864f4cbd55809a786bd5759fc24a5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Added a utility method to set the reset an index to match exactly
some content in the filesystem. This can be used by tests to prepare
commits in the working-tree and set the index in one shot.
[sp: Cleaned up formatting, added getEntryFile(), released inserter.]
Change-Id: If38b1f7cacaaf769f51b14541c5da0c1e24568a5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This patch fixes the problem that JGit tests run from Maven fail on
Mac OS X [1]. In Eclipse the tests succeed since we set Eclipse
workspace encoding to UTF-8 via "Preferences > General > Workspace
> Text file encoding", checked via JConsole that this setting changes
the JVM system property of the test run. This change copies this
setting to the Maven test environment so that we get consistent test
results on all platforms.
[1] http://kerneltrap.org/mailarchive/git/2009/2/6/4916414/thread
Change-Id: Ia657d4037c518411f7bc3c75626dbafb1f4706a2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If the exact rename matrix for a particular ObjectId isn't square we
crashed with an ArrayIndexOutOfBoundsException because the matrix
entries were encoded backwards. The encode function accepts the
source (aka deleted) index first, not second. Add a unit test to
cover this non-square case to ensure we don't have this regression
in the future.
Change-Id: I5b005e5093e1f00de2e3ec104e27ab6820203566
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* changes:
Enable configuration of non-standard pack settings
Pass PackConfig down to PackWriter when packing
Simplify UploadPack use of options during writing
Move PackWriter configuration to PackConfig
Allow PackWriter callers to manage the thread pool
It is useful to be able to replace an existing Change-Id
in the message, for example if the user decides not to
amend the previous commit.
Bug: 321188
Change-Id: I594e7f9efd0c57d794d2bd26d55ec45f4e6a47fd
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
TreeWalk calls this value "path", while "name" is the stuff after the
last slash. FileHeader should do the same thing to be consistent.
Rename getOldName to getOldPath and getNewName to getNewPath.
Bug: 318526
Change-Id: Ib2e372ad4426402d37939b48d8f233154cc637da
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A bug in rename detection would cause file scores to be wrong. The
bug was due to the way rename detection would judge the similarity
between files. If file A has three lines containing 'foo', and file
B has 5 lines containing 'foo', the rename detection phase should
record that A and B have three lines in common (the minimum of the
number of times that line appears in both files). Instead, it would
choose the the number of times the line appeared in the destination
file, in this case file B. I fixed the bug by having the
SimilarityIndex instead choose the minimum number, as it should. I
also added a test case to verify that the bug had been fixed.
Change-Id: Ic75272a2d6e512a361f88eec91e1b8a7c2298d6b
IndexDiff was re-implemented and now uses TreeWalk instead
of GitIndex. Additionally, gitignore support and retrieval of
untracked files was added.
Change-Id: Ie6a8e04833c61d44c668c906b161202b200bb509
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
When the add parameter is set all modified and deleted files
are staged prior to commit.
Change-Id: Id23bc25730fcdd151386cd495a7cdc0935cbc00b
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
This change is mainly done for a subsequent commit
which will introduce the "all" parameter to the Commit
command.
Bug: 318439
Change-Id: I85a8a76097d0197ef689a289288ba82addb92fc9
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
An utility method which was in RacyGitTests has been moved to
RepositoryTestCase. Also the javadoc has been improved.
This method allows to wait long enough until the
filesystem-timer has advanced. This is useful when it has to
be guaranteed that two files modifications have different
modification timestamps.
Change-Id: I2ebd7cd7818feba6acffb3f835101d8fd281bd5a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
The indexState() method was enhanced to be more configurable. A bitmask
controls which of the optional parts are reported. All data about
the worktree is not reported anymore by this method which makes the
interface more cleaner for users wanting to test only the state of the
index.
This was done because the previous version reported always so much
additional data that it was hard to write good assertions against it.
Change-Id: I9b481e97f8fcf3fcdbb785b801dc07bfa85dcc33
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
getEntryLastModified returned wrong values for
timestamps on boundaries.
Change-Id: I0457ff030946fda7ecc70671bc65db02a8c206ee
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Its only available in Java 6, and JGit still supports Java 5.
Change-Id: I6c8fac06106ffe20ac8b4b595f34e027f2d29cc2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In order to test racy git situations we have to be able to control
the last-modification timestamps of the filesystem. Since we already
access the modification timestamps of files through an abstraction
(the WorkingTreeIterator) I add a new implementation of this iterator
which allows to map timestamp-ranges to single constant timestamps.
For users of this iterator it looks like all files in that range
have been modified at exactly the same time.
With the help of this iterator a test has been writting which
checkes for racy git handling (smudging, unsmudging, dirty-detection).
Additionally add a method to RepositoryTestCase which encodes the
current index state in one String. This should include info about
pathes, file/index modtime, smudgeState, clean-state. Make
sure timestamps are presented in a way that it is easy to
write assertions against this strings (no concrete milliseconds
but t0,t1,...).
These two topics depend circulary on each other: thats why they have
been squashed in one commit.
Change-Id: I115c3f2f20fca9b481830bdc6b9d1ade2c3abdcf
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This refactoring permits applications to configure global per-process
settings for all packing and easily pass it through to per-request
PackWriters, ensuring that the process configuration overrides the
repository specific settings.
For example this might help in a daemon environment where the server
wants to cap the resources used to serve a dynamic upload pack
request, even though the repository's own pack.* settings might be
configured to be more aggressive. This allows fast but less bandwidth
efficient serving of clients, while still retaining good compression
through a cron managed `git gc`.
Change-Id: I58cc5e01b48924b1a99f79aa96c8150cdfc50846
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add a method isDirectoryFileConflict() to NameConflictTreeWalk which
tells whether the current path is part of a directory/file conflict.
Change-Id: Iffcc7090aaec743dd6f3fd1a333cac96c587ae5d
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
File pairs that are very dissimilar during a diff were not being
broken apart into their constituent ADD/DELETE pairs. The leads to
sub-optimal rename detection. Take, for example, this situation:
A file exists at src/a.txt containing "foo". A user renames src/a.txt
to src/b.txt, then adds a new src/a.txt containing "bar".
Even though the old a.txt and the new b.txt are identical, the
rename detection algorithm would not detect it as a rename since
it was already paired in a MODIFY. I added code to split all
MODIFYs below a certain score into their constituent ADD/DELETE
pairs. This allows situations like the one I described above to be
more correctly handled.
Change-Id: I22c04b70581f206bbc68c4cd1ee87a1f663b418e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add methods to the Repository class which write into MERGE_HEAD
and MERGE_MSG files. Since we have the read methods in the same
class this seems to be the right place.
Change-Id: I5dd65306ceb06e008fcc71b37ca3a649632ba462
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
LockFile.commit fails if another thread concurrently reads
the base file. The problem is fixed by retrying the rename
operation if it fails.
Change-Id: I6bb76ea7f2e6e90e3ddc45f9dd4d69bd1b6fa1eb
Bug: 308506
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
There were some broken links, incorrect uses of @value, an invalid
tag and an outdated comment.
Change-Id: I22886bcc869a4b62bd606ebed40669f7b4723664
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* delta: (103 commits)
Discard the uncompressed delta as soon as its compressed
Honor pack.windowlimit to cap memory usage during packing
Honor pack.threads and perform delta search in parallel
Cache small deltas during packing
Implement delta generation during packing
debug-show-packdelta: Dump a pack delta to the console
Initial pack format delta generator
Add debugging toString() method to ObjectToPack
Make ObjectToPack clearReuseAsIs signal available to subclasses
Correctly classify the compressing objects phase
Refactor ObjectToPack's delta depth setting
Configure core.bigFileThreshold into PackWriter
Add doNotDelta flag to ObjectToPack
Add more configuration options to PackWriter
Save object path hash codes during packing
Add path hash code to ObjectWalk
Add getObjectSize to ObjectReader
Allow TemporaryBuffer.Heap to allocate smaller than 8 KiB
Define a constant for 127 in DeltaEncoder
Cap delta copy instructions at 64k
...
Conflicts:
org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Diff.java
org.eclipse.jgit/resources/org/eclipse/jgit/JGitText.properties
org.eclipse.jgit/src/org/eclipse/jgit/JGitText.java
org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RewriteTreeFilter.java
Change-Id: I7c7a05e443a48d32c836173a409ee7d340c70796
The working tree iterator has perfect knowledge of the path structure
as well as immediate information about whether or not an ignore file
even exists at this level. We can exploit that to simplify the
logic and running time for testing ignored file status by pushing
all of the checks down into the iterator itself.
Change-Id: I22ff534853e8c5672cc5c2d9444aeb14e294070e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Charley Wang <chwang@redhat.com>
CC: Chris Aniszczyk <caniszczyk@gmail.com>
CC: Stefan Lay <stefan.lay@sap.com>
CC: Matthias Sohn <matthias.sohn@sap.com>
If we have two adds of the same object but no deletes the detector
threw an NPE because the entry that came back from the deleted map
was null (no matching objects). In this case we need to put the
adds all back onto the list of left over additions since they did
not match a delete.
Change-Id: Ie68fbe7426b4dc0cb571a08911c7adbffff755d5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Jeffrey Schumacher" <jeffschu@google.com>
* js/rename:
Implemented file path based tie breaking to exact rename detection
Added more test cases for RenameDetector
Added very small optimization to exact rename detection
Fixed Misleading Javadoc
Added file path similarity to scoring metric in rename detection
Fixed potential div by zero bug
Added file size based rename detection optimization
Create FileHeader from DiffEntry
log: Implement --follow
Cache the diff configuration section
log: Add whitespace ignore options
Format submodule links during differences
Redo DiffFormatter API to be easier to use
log, diff: Add rename detection support
Implement similarity based rename detection
Added a preliminary version of rename detection
Refactored code out of FileHeader to facilitate rename detection
During the exact rename detection phase in RenameDetector, ties were
resolved on a first-found basis. I added support for file path based
tie breaking during that phase. Basically, there are four situations
that have to be handled:
One add matching one delete:
In this simple case, we pair them as a rename.
One add matching many deletes:
Find the delete whos path matches the add the closest, and
pair them as a rename.
Many adds matching one delete:
Similar to the above case, we find the add that matches the
delete the closest, and pair them as a rename. The other adds
are marked as copies of the delete.
Many adds matching many deletes:
Build a scoring matrix similar to the one used for content-
based matching, scoring instead by file path. Some of the
utility functions in SimilarityRenameDetector are used in
this case, as we use the same encoding scheme. Once the
matrix is built, scan it for the best matches, marking them
as renames. The rest are marked as copies.
I don't particularly like the idea of using utility functions right
out of SimilarityRenameDetector, but it works for the moment. A later
commit will likely refactor this into a common utility class, as well
as bringing exact rename detection out of RenameDetector and into a
separate class, much like SimilarityRenameDetector.
Change-Id: I1fb08390aebdcbf20d049aecf402a36506e55611
When the path is prefixed with ~ the URI parser thought about this
as /~. Strip the / if the next character is the tilde.
Bug: 307017
Change-Id: I58203e5617956b46d83e8987d1f8042beddffac3
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
The new Add command adds files to the Git Index.
It uses the DirCache to access the git index. It
works also in case of an existing conflict.
Fileglobs (e.g. *.c) are not yet supported.
The new Add command does add ignored files because
there is no gitignore support in jgit yet.
Bug: 318440
Change-Id: If16fdd4443e46b27361c2a18ed8f51668af5d9ff
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
After refactoring ReadTreeTest the tests failed for filesystems
with coarse modification time granularity. This is fixed by
explicitly telling the repo to reread the index after we build
a new index.
Additionally the test testDirectoryFileSimple was simplified
by using buildTree() instead of misusing GitIndex to construct
trees.
Change-Id: I20d2f097491e4cc8c657a696beabc7026b485017
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This patch adds ignore compatibility to jgit. It encompasses
exclude files as well as .gitignore. Uses TreeWalk and
FileTreeIterator to find nodes and parses .gitignore
files when required. The patch includes a simple cache that
can be used to save results and avoid excessive gitignore
parsing.
CQ: 4302
Bug: 303925
Change-Id: Iebd7e5bb534accca4bf00d25bbc1f561d7cad11b
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
I added test cases to cover the majority of the code. It's not 100%
coverage yet, but the remaining bits are small.
Change-Id: Ib534c8e94b13358b8b22cf54e2ff84132bae6d14
The scoring method was not taking into account the similarity of
the file paths and file names. I changed the metric so that it is 99%
based on content (which used to be 100% of the old metric), and 1%
based on path similarity. Of that 1%, half (.5% of the total final
score) is based on the actual file names (e.g. "foo.java"), and half
on the directory (e.g. "src/com/foo/bar/").
Change-Id: I94f0c23bf6413c491b10d5625f6ad7d2ecfb4def
DeltaIndex is a simple pack style delta generator. The function works
by creating a compact index of a source buffer's blocks, and then
walking a sliding window along a desired result buffer, searching for
the window in the index. When a match is found, the window is
stretched to the longest possible length that is common with the
source buffer, and a copy instruction is created.
Rabin's polynomial hash function is used to compute the hash for a
block, permitting efficient sliding of the window in single byte
increments. The update function to slide one byte originated from
David Mazieres' work in LBFS, and our implementation of the update
step was certainly inspired by the initial work Geert Bosch proposed
for C Git in http://marc.info/?l=git&m=114565424620771&w=2.
To ensure the encoder runs in linear time with respect to the size of
the two input buffers (source and result), the maximum number of
blocks that can share the same position in the index's hashtable is
capped at a constant number. This prevents bad inputs from causing
the encoder to run in quadratic time, but comes with a penalty of
creating a longer delta due to fewer considered copy positions.
Strange hackery is used to cap the amount of memory used by the index
to be no more than 12 bytes for every 16 bytes of source buffer, no
matter what the JVM per-object overhead is. This permits an index to
always be no larger than 1.75x the source buffer length, which is an
important feature to support large windows of candidates to match
against while packing. Here the strange hackery is nothing more than
a manually managed chained hashtable, where pointers are array indexes
into storage arrays rather than object references.
Computation of the hash function for a single fixed sized block is
done through an unrolled loop, where the first 4 iterations have been
manually reduced down to eliminate unnecessary instructions. The
pattern is derived from ObjectId.equals(byte[], int, byte[], int),
where we have unrolled the loop required to compare two 20 byte
arrays. Hours of testing with the Sun 1.6 JRE concluded that the
non-obvious "foo[idx + 1]" style of reference is faster than
"foo[idx++]", and so that is what we use here during hashing.
Change-Id: If9fb2a1524361bc701405920560d8ae752221768
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the heap limit was set to something smaller than 8 KiB, we were
still allocating the full 8 KiB block size, and accepting up to
the amount we allocated by. Instead actually put a hard cap on
the limit.
Change-Id: Id1da26fde2102e76510b1da4ede8493928a981cc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Added support for converting DiffEntrys to FileHeaders. FileHeaders
are DiffEntrys with a buffer containing the diff output as well as
a list of HunkHeaders. The HunkHeaders contain EditLists. The
createFileHeader(DiffEntry) method in DiffFormatter performs a Myers
Diff on the files refered to by the DiffEntry, then puts the returned
EditList into a single HunkHeader, which is then put into the
FileHeader to be returned. It also generates the appropriate diff
header an puts it into the FileHeader's buffer. The rest of the diff
output, which would normally be parsed to generate the HunkHeaders,
is not generated. In fact, the purpose of this method is to avoid
the costly diff output generation and parsing normally required to
create a FileHeader.
Change-Id: I7d8b18c0f6c85e3d02ad58995d3d231e69af5887
ReadTreeTest was hardcoded to test WorkDirCheckout. Since we want
alternative checkout implementations (especially DirCacheCheckout)
this class has been refactored so that the tests can be reused
to test other implementations
The following changes have been done:
- abstract methods for checkout and prescanTwoTrees have been
introduced. Parameters are only the two trees. As index we
will implicitly use the current index of the repo.
- whenever tests needed a manipulated index before checkout
and prescanTwoTrees it was ensured that the correct index was
persisted (before we could use not-persisted instantiations of GitIndex
passed as parameters to checkout, prescanTwoTrees
- abstract methods for getting updated, conflicting, removed entries
resulting from the last checkout, prescanTwoTrees have been introduced
- an implementation for all these abstract methods using WorkDirCheckout
has been added
- method to assert a certain state of the index and the working tree has
been added
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Change-Id: Icf177cf8043487169a32ddd72b6f8f9246a433f7
We want to get rid of these APIs, because they don't perform as well
as DirCache/TreeWalk, or don't offer nearly as many features.
Bug: 319145
Change-Id: I2b28f9cddc36482e1ad42d53e86e9d6461ba3bfc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We need to validate the stream state after the InflaterInputStream
thinks the stream is done. Git expects a higher level of service from
the Inflater than the InflaterInputStream usually gives, we need to
ensure the embedded CRC is valid, and that there isn't trailing
garbage at the end of the file.
Change-Id: I1c9642a82dbd76b69e607dceccf8b85dc869a3c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Passing around the OutputStream and the Repository is crazy. Instead
put the stream in the constructor, since this formatter exists only to
output to the stream, and put the repository as a member variable that
can be optionally set.
Change-Id: I2bad012fee7f40dc1346700ebd19f1e048982878
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Content similarity based rename detection is performed only after
a linear time detection is performed using exact content match on
the ObjectIds. Any names which were paired up during that exact
match phase are excluded from the inexact similarity based rename,
which reduces the space that must be considered.
During rename detection two entries cannot be marked as a rename
if they are different types of files. This prevents a symlink from
being renamed to a regular file, even if their blob content appears
to be similar, or is identical.
Efficiently comparing two files is performed by building up two
hash indexes and hashing lines or short blocks from each file,
counting the number of bytes that each line or block represents.
Instead of using a standard java.util.HashMap, we use a custom
open hashing scheme similiar to what we use in ObjecIdSubclassMap.
This permits us to have a very light-weight hash, with very little
memory overhead per cell stored.
As we only need two ints per record in the map (line/block key and
number of bytes), we collapse them into a single long inside of
a long array, making very efficient use of available memory when
we create the index table. We only need object headers for the
index structure itself, and the index table, but not per-cell.
This offers a massive space savings over using java.util.HashMap.
The score calculation is done by approximating how many bytes are
the same between the two inputs (which for a delta would be how much
is copied from the base into the result). The score is derived by
dividing the approximate number of bytes in common into the length
of the larger of the two input files.
Right now the SimilarityIndex table should average about 1/2 full,
which means we waste about 50% of our memory on empty entries
after we are done indexing a file and sort the table's contents.
If memory becomes an issue we could discard the table and copy all
records over to a new array that is properly sized.
Building the index requires O(M + N log N) time, where M is the
size of the input file in bytes, and N is the number of unique
lines/blocks in the file. The N log N time constraint comes
from the sort of the index table that is necessary to perform
linear time matching against another SimilarityIndex created for
a different file.
To actually perform the rename detection, a SxD matrix is created,
placing the sources (aka deletions) along one dimension and the
destinations (aka additions) along the other. A simple O(S x D)
loop examines every cell in this matrix.
A SimilarityIndex is built along the row and reused for each
column compare along that row, avoiding the costly index rebuild
at the row level. A future improvement would be to load a smaller
square matrix into SimilarityIndexes and process everything in that
sub-matrix before discarding the column dimension and moving down
to the next sub-matrix block along that same grid of rows.
An optional ProgressMonitor is permitted to be passed in, allowing
applications to see the progress of the detector as it works through
the matrix cells. This provides some indication of current status
for very long running renames.
The default line/block hash function used by the SimilarityIndex
may not be optimal, and may produce too many collisions. It is
borrowed from RawText's hash, which is used to quickly skip out of
a longer equality test if two lines have different hash functions.
We may need to refine this hash in the future, in order to minimize
the number of collisions we get on common source files.
Based on a handful of test commits in JGit (especially my own
recent rename repository refactoring series), this rename detector
produces output that is very close to C Git. The content similarity
scores are sometimes off by 1%, which is most probably caused by
our SimilarityIndex type using a different hash function than C
Git uses when it computes the delta size between any two objects
in the rename matrix.
Bug: 318504
Change-Id: I11dff969e8a2e4cf252636d857d2113053bdd9dc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
During code review, Alex raised a few comments about commit
532421d989 ("Refactor repository construction to builder class").
Due to the size of the related series we aren't going to go back
and rebase in something this minor, so resolve them as a follow-up
commit instead.
Change-Id: Ied52f7a8f7252743353c58d20bfc3ec498933e00
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We default this to 1 MiB for now, but we allow users to modify
it through the Repository's configuration file to be a different
value. A new repository listener is used to identify when the
setting has been updated and trigger a reconfiguration of any
active ObjectReaders.
To prevent a horrible explosion we cap core.streamFileThreshold
at no more than 1/4 of the maximum JVM heap size. We do this
because we need at least 2 byte arrays equal in size to the
stream threshold for the worst case delta inflation scenario,
and our host application probably also needs some amount of the
heap for their working set size.
Change-Id: I103b3a541dc970bbf1a6d92917a12c5a1ee34d6c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Very large delta instruction streams, or deltas which use very large
base objects, are now streamed through as large objects rather than
being inflated into a byte array.
This isn't the most efficient way to access delta encoded content, as
we may need to rewind and reprocess the base object when there was a
block moved within the file, but it will at least prevent the JVM from
having its heap explode.
When streaming a delta we have an inflater open for each level in the
delta chain, to inflate the instruction set of the delta, as well as
an inflater for the base level object. The base object is buffered,
as is the top level delta requested by the application, but we do not
buffer the intermediate delta streams. This keeps memory usage lower,
so its closer to 1024 bytes per level in the chain, without having an
adverse impact on raw throughput as the top-level buffer gets pushed
down to the lowest stream that has the next region.
Delta instructions transparently collapse here, if the top level does
not copy a region from its base, the base won't materialize that part
from its own base, etc. This allows us to avoid copying around a lot
of segments which have been deleted from the final version.
Change-Id: I724d45245cebb4bad2deeae7b896fc55b2dd49b3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Similar to the loose object support, whole packed objects can
now be streamed back to the caller. The streaming is less
efficient as we copy the data from the cached window array
into the InflaterInputStream's internal buffer, then inflate
it there before returning to the application.
Like with unpacked objects, there is plenty of room for some
optimization, especially for the copyTo method, where we don't
necessarily need so much buffering to exist.
Change-Id: Ie23be81289e37e24b91d17b0891e47b9da988008
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The class is identical, but ObjectLoader.SmallObject is part of our
public API for storage implementations to build on top of.
Change-Id: I381a3953b14870b6d3d74a9c295769ace78869dc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Big loose objects can now be streamed if they are over the large
object size threshold. This prevents the JVM heap from exploding
with a very large byte array to hold the slurped file, and then
again with its uncompressed copy.
We may have slightly slowed down the simple case for small
loose objects, as the loader no longer slurps the entire thing
and decompresses in memory. To try and keep good performance
for the very common small objects that are below 8 KiB in size,
buffers are set to 8 KiB, causing the reader to slurp most of the
file anyway. However the data has to be copied at least once,
from the BufferedInputStream into the InflaterInputStream.
New unit tests are supplied to get nearly 100% code coverage on the
unpacked code paths, for both standard and pack style loose objects.
We tested a fair chunk of the code elsewhere, but these new tests
are better isolated to the specific branches in the code path.
Change-Id: I87b764ab1b84225e9b5619a2a55fd8eaa640e1fe
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
JGit does not currently do rename detection during diffs. I added
a class that, given a TreeWalk to iterate over, can output a list
of DiffEntry's for that TreeWalk, taking into account renames. This
class only detects renames by SHA1's. More complex rename detection,
along the lines of what C Git does will be added later.
Change-Id: I93606ce15da70df6660651ec322ea50718dd7c04
Instead of creating the DirCache from a static factory method, use
an instance method on Repository, permitting the implementation to
override the method with a completely different type of DirCache
reading and writing. This would better support a repository in the
cloud strategy, or even just an in-memory unit test environment.
Change-Id: I6399894b12d6480c4b3ac84d10775dfd1b8d13e7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Using a custom exception type makes it easire for an application
developer to understand why an exception was thrown out of a method
we declare. To remain compatiable with existing callers, we still
extend off IllegalStateException.
Change-Id: Ideeef2399b11ca460a2dbb3cd80eb76aa0a025ba
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We don't actually need a Repository object here, just an ObjectReader
that can load content for us. So change the API to depend on that.
However, this breaks the asCommit and asTag legacy translation methods
on RevCommit and RevTag, so we still have to keep the Repository
inside of RevWalk for those two types. Hopefully we can drop those in
the future, and then drop the Repository off the RevWalk.
Change-Id: Iba983e48b663790061c43ae9ffbb77dfe6f4818e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Added code to support ignoring leading, trailing, and changed
whitespace when performing a diff operation. I also added command
line options to Diff to enable the various whitespace ignoring
methods. These match the flags for git diff.
Change-Id: Ie56301aafad59ee3f0fe5de62719f5023cd702c8
We drop the "Object" suffix, because its pretty clear here that
we want to open an object, given that we pass in AnyObjectId as
the main parameter. We also fix the calling convention to throw
a MissingObjectException or IncorrectObjectTypeException, so that
callers don't have to do this error checking themselves.
Change-Id: I72c43353cea8372278b032f5086d52082c1eee39
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Rather than taking the ProgressMonitor objects in our constructor and
carrying them around as instance fields, take them as arguments to the
actual time consuming operations we need to run.
Change-Id: I2b230d07e277de029b1061c807e67de5428cc1c4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
JGit did not have support for skipping whitespace when comparing
lines in RawText objects. I added a subclass of RawText that skips
whitespace in its equals and hashCode methods. I used a subclass
rather than adding functionality into RawText so that performance
would not be impacted by extra logic.
This class only supports ignoring all whitespace. Others will follow
that allow other forms of whitespace ignoring.
Change-Id: Ic2f79e85215e48d3fd53ec1b4ad13373dd183a4a
The ObjectReader API demands that we release the reader when we are
done with it. PackWriter contains a reader, which it uses for the
entire packing session. Expose the release of the reader through
a release method on the writer.
This still doesn't address the RevWalk and TreeWalk users, who
don't correctly release their reader. But its a small step in the
right direction.
Change-Id: I5cb0b5c1b432434a799fceb21b86479e09b84a0a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Similar to what we did with the file code, move the pack writer
into its own package so the related classes and their package
private methods are hidden from the rest of the library.
Change-Id: Ic1b5c7c8c8d266e90c910d8d68dfc8e93586854f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We no longer need an ObjectLoader to be lazy and try to delay
the materialization of the object content. That was done only
to support PackWriter searching for a good reuse candidate.
Instead, simplify the code base by doing the materialization
immediately when the loader asks for it, because any caller
asking for the loader is going to need the content.
Change-Id: Id867b1004529744f234ab8f9cfab3d2c52ca3bd0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These were only used by PackWriter to help it filter object
representations. Their only user disappeared when we rewrote the
object selection code path to use the new representation type.
Change-Id: I9ed676bfe4f87fcf94aa21e53bda43115912e145
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This move isolates all of the local file specific implementation code
into a single package, where their package-private methods and support
classes are properly hidden away from the rest of the core library.
Because of the sheer number of files impacted, I have limited this
change to only the renames and the updated imports.
Change-Id: Icca4884e1a418f83f8b617d0c4c78b73d8a4bd17
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Only the ObjectDirectory type of database knows where to find the
objects directory on the local filesystem, so defer to it whenever
we need to know where the objects reside. Since this is the type
returned by FileRepository's getObjectDatabase() method, we mostly
don't have to do much other than use a slightly longer invocation.
Change-Id: Ie5f58132a6411b56c3acad73646ad169d78a0654
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This better matches with the name used in the environment
(GIT_WORK_TREE), in the configuration file (core.worktree),
and in our builder object.
Since we are already breaking a good chunk of other code
related to repository access, and this fairly easy to fix
in an application's code base, I'm not going to offer the
wrapper getWorkDir() method.
Change-Id: Ib698ba4bbc213c48114f342378cecfe377e37bb7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The new FileRepositoryBuilder class helps applications to construct
a properly configured FileRepository, with properties assumed based
upon the standard Git rules for the local filesystem.
To better support simple command line applications, environment
variable handling and repository searching was moved into this
builder class.
The change gets rid of the ever-growing FileRepository constructor
variants, and the multitude of java.io.File typed parameters,
by using simple named setter methods.
Change-Id: I17e8e0392ad1dbf6a90a7eb49a6d809388d27e4c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Not every type of Repository will be able to map an ObjectId into
a local file system path that stores that object's file contents.
Heck, its not even true for the FileRepository, as an object can
be stored in a pack file and not in its loose format.
Remove this from our public API, it was a mistake to publish it.
Change-Id: I20d1b8c39104023936e6d46a5b0d7ef39ff118e8
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The WindowCache is an implementation detail of PackFile and how its
used by ObjectDirectory. Lets start to hide it and replace the public
API with a more generic concept, ObjectReader.
Because PackedObjectLoader is also considered a private detail of
PackFile, we have to make PackWriter temporarily dependent upon the
WindowCursor and thus FileRepository and ObjectDirectory in order to
just start the refactoring. In later changes we will clean up the
APIs more, exposing sufficient support to PackWriter without needing
the file specific implementation details.
Change-Id: I676be12b57f3534f1285854ee5de1aa483895398
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some newer style APIs are updated to use the newer ObjectInserter
interface instead of the now deprecated ObjectWriter. In many of
the unit tests we don't bother to release the inserter, these are
typically using the file backend which doesn't need a release,
but in the future should use an in-memory HashMap based store,
which really wouldn't need it either.
Change-Id: I91a15e1dc42da68e6715397814e30fbd87fa2e73
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The ObjectInserter API permits ObjectDatabase implementations to
control their own object insertion behavior, rather than forcing
it to always be a new loose file created in the local filesystem.
Inserted objects can also be queued and written asynchronously to
the main application, such as by appending into a pack file that
is later closed and added to the repository.
This change also starts to open the door to non-file based object
storage, such as an in-memory HashMap for unit testing, or a more
complex system built on top of a distributed hash table.
To help existing application code port to the newer interface we
are keeping ObjectWriter as a delegation wrapper to the new API.
Each ObjectWriter instances holds a reference to an ObjectInserter
for the Repository's top-level ObjectDatabase, and it flushes and
releases that instance on each object processed.
Change-Id: I413224fb95563e7330c82748deb0aada4e0d6ace
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When the surrounding code is already heavily based upon the
assumption that we have a FileRepository (e.g. because it
created that type of repository) keep the type around and
use it directly. This permits us to continue to do things
like save the configuration file.
Change-Id: Ib783f0f6a11acd6aa305c16d61ccc368b46beecc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
To support other storage models other than just the local filesystem,
we split the Repository class into a nearly abstract interface and
then create a concrete subclass called FileRepository with the file
based IO implementation.
We are using an abstract class for Repository rather than the much
more generic interface, as implementers will want to inherit a large
array of utility functions, such as resolve(String). Having these in
a base class makes it easy to inherit them.
This isn't the final home for lib.FileRepository. Future changes
will rename it into storage.file.FileRepository, but to do that we
need to also move a number of other related class, which we aren't
quite ready to do.
Change-Id: I1bd54ea0500337799a8e792874c272eb14d555f7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Rather than relying on the helpers in RepositoryConfig to get
these objects, obtain them directly through the Config API.
Its only slightly more verbose, but permits us to work with the
base Config class, which is more flexible than the highly file
specific RepositoryConfig.
This is what I really meant to do when I added the section parser
and caching support to Config, we just failed to finish updating
all of the call sites.
Change-Id: I481cb365aa00bfa8c21e5ad0cd367ddd9c6c0edd
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We don't have to assume/depend on RepositoryConfig here, these
two tests can use higher level versions of the class and still
come up with the same test. That frees us up to do some changes
to the RepositoryConfig API.
Change-Id: Ia7b263c8c5efa3fae1054416d39c546867288132
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Long ago we stopped supporting the core.legacyHeaders variable,
as JGit (like C Git) stopped creating the new pack-style loose
objects, rendering this variable pointless. The test is still
valid, it proves we write the standard loose object format for
a commit, but the variable assignment has no impact on the test
so drop it from the code.
Change-Id: I051336ada23033c05e86bbff73ae5d78a37b1640
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This stream was used only to determine how many bytes had been
written thus far. Except we're always dumping it into a simple
ByteArrayOutputStream, which also knows that. Drop the dependency
on the pack stream and use ByteArrayOutputStream directly.
This lets us later move this test into the new storage.file
package without dragging along the pack stream that is an internal
implementation detail of PackWriter, which is more general than
just the file storage layer.
Change-Id: I291689c0b1ed799270c213ee73b710b2637fb238
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Setting this value is pointless, because its automatically set
by the refs.newUpdate call that created the update operation.
The API is protected by default, because application level code,
including this test, should not be calling it.
Change-Id: I8867a4e8007892e2bd44a05d7dec619081081943
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The mapCommit API is being deprecated because it doesn't run very
fast. Leaving tests around to test how fast it is relative to C Git
isn't instructive. Remove them, which should help aid the transition
away from the mapCommit API.
Change-Id: I27e1c844610d7da5b2c44b33a00602706973c9cc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some sources had dos line endings. Also configure all projects to use
unix line endings and UTF-8 text encoding.
Change-Id: I8fc9a1dbb219ffa91d1b3011b3b11b7e48e74ca7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a repository is "bare", it currently still returns a working directory.
This conflicts with the specification of "bare"-ness.
Bug: 311902
Change-Id: Ib54b31ddc80b9032e6e7bf013948bb83e12cfd88
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Currently, there is no way to read the content
of the Git Configuration in a way that would
allow to list all configured values generically.
This change extends the Config class in such a
way as to being able to get a list of sections and
to get a list of names for any given section or
subsection.
This is required in able to implement proper
configuration handling in EGit (show all the
content of a given configuration similar to
"git config -l").
Change-Id: Idd4bc47be18ed0e36b11be8c23c9c707159dc830
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
On Windows, FS_Win32_Cygwin has been used if a Cygwin Git installation
is present in the PATH. Assuming that the user works with the Cygwin
Git installation may result in unnecessary overhead if he actually
does not.
Applications built on top of jgit may have more knowledge on the
actually used Git client (Cygwin or not) and hence should be able to
configure which FS to use accordingly.
Change-Id: Ifc4278078b298781d55cf5421e9647a21fa5db24
A Change-Id helps tools like Gerrit Code Review to keeps different
versions of a patch together. The Change-Id is computed as a SHA-1
hash of some of the same basic information as a commit id on the first
commit intended to solve a particular problem and then reused for
updated solutions.
Change-Id: I04334f84e76e83a4185283cb72ea0308b1cb4182
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
ReadTreeTest contains a lot of useful tests for "checkout"
implementations. But ReadTreeTest was hardcoded to test only
WorkDirCheckout. This change doesn't add/modify any tests semantically
but refactors ReadTreeTest so that a different implementations of
checkout can be tested. This was done to allow DirCacheCheckout to be
tested without rewriting all these tests.
Change-Id: I36e34264482b855ed22c9dde98824f573cf8ae22
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Merges the current head with one other commit.
In this first iteration the merge command supports
only fast forward and already up-to-date.
Change-Id: I0db480f061e01b343570cf7da02cac13a0cbdf8f
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
The CommitCommand should take care to create a merge commit if the file
$GIT_DIR/MERGE_HEAD exists. It should then read the parents for the merge
commit out of this file. It should also take care that when commiting
a merge and no commit message was specified to read the message from
$GIT_DIR/MERGE_MSG.
Finally the CommitCommand should remove these files if the commit
succeeded.
Change-Id: I4e292115085099d5b86546d2021680cb1454266c
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
The strings are externalized into the root resource bundles.
The resource bundles are stored under the new "resources" source
folder to get proper maven build.
Strings from tests are, in general, not externalized. Only in
cases where it was necessary to make the test pass the strings
were externalized. This was typically necessary in cases where
e.getMessage() was used in assert and the exception message was
slightly changed due to reuse of the externalized strings.
Change-Id: Ic0f29c80b9a54fcec8320d8539a3e112852a1f7b
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Rather than keep track of both the position of the object, and the
position of its data, just keep track of the number of bytes used
by the object's header in the pack. This shaves 4 bytes out of the
size of the PackedObjectLoader instances.
We also can defer the addition instruction to the materialize()
operation, avoiding it entirely if the caller never actually uses
the loader. This may be relevant for PackWriter invocations,
where only 1 loader gets chosen for a given object, even though
the object may appear on disk in more than one pack file.
Error reporting is now simplified, as we can rely on the object
offset rather than its data offset. This is the value displayed
by pack debugging tools like `git verify-pack -v`, so its better
to use that in our own errors.
Because nobody needs getDataOffset() now, we can drop that from
the public API.
Change-Id: Ic639c0d5a722315f4f5c8ffda6e26643d90e5f42
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Added a new package org.eclipse.jgit.api and a builder-style API for
jgit. Added also the first implementation for two git commands: Commit
and Log.
This API is intended to be used by external components when
functionalities of the standard git commands are required. It will also
help to ease writing JGit tests.
For internal usages this API may often not be optimal because the git
commands are doing much more than required or they expect parameters of
an unappropriate type.
Change-Id: I71ac4839ab9d2f848307eba9252090c586b4146b
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
The repository state tells in which state the repo is and also which actions
are currently allowed. The state MERGING is telling that a commit is not
possible. But this is only true in the case of unmerged paths in the index.
When we are merging but have resolved all conflicts then we are in a special
state: We are still merging (means the next commit should have multiple
parents) but a commit is now allowed.
Since the MERGING state "canCommit()" cannot be enhanced to return true/false
based on the index state (MERGING is an enum value which does not have a
reference to the repository its state it is representing) I had to introduce a new
state MERGING_RESOLVED. This new state will report that a commit is possible.
CAUTION: there might be the chance that users of jgit previously blindly did a
plain commit (with only one parent) when the RepositoryState allowed them to
do so. With this change these users will now be confronted with a RepositoryState
which says a commit is possible but before they can commit they'll have to
check the MERGE_MESSAGE and MERGE_HEAD files and use the info from these
files.
Change-Id: I0a885e2fe8c85049fb23722351ab89cf2c81a431
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If two keys are the same length, but don't share the same sequence
of characters, we were incorrectly claiming they still matched due
to a bug in the for loop condition. I used the wrong variable and
the loop never executed, resulting in equality anytime the two keys
being compared were the same length.
Use the proper local variable to loop through the arrays, and add
a JUnit test to verify equality works as expected.
Change-Id: I4a02400e65a9b2e0da925b05a2cc4b579e1dd33a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The pack files were left open after the test ended, which meant
we could not delete them automatically when the test was over.
Make sure we close the repositories (and thus their underlying packs)
before the tear down finishes.
Bug: 310367
Change-Id: I4d2703efa4b2e0c347ea4f4475777899cf71073e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This option was mis-named from day 1. Its not checking that the
objects provided by the client are reachable, its actually doing
a scan to prove that objects referenced by the client are already
reachable through another reference on the server, or were sent
as part of the pack from the client.
Rename it checkReferencedObjectsAreReachable, since we really are
trying to validate that objects referenced by the client's actions
are reachable to the client.
We also need to ensure we run checkConnectivity() anytime this is
enabled, even if the caller didn't turn on fsck for object formats.
Otherwise the check would be completely bypassed.
Change-Id: Ic352ddb0ca8464d407c6da5c83573093e018af19
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If ensureProvidedObjectsVisible is enabled we expected any trees or
blobs directly reachable from an advertised reference to be marked
with UNINTERESTING. Unfortunately ObjectWalk doesn't bother setting
this until the traversal is complete. Even then it won't necessarily
set it on every tree if the corresponding commit wasn't popped.
When we are going to check the base objects for the received pack,
ensure the UNINTERESTING flag gets carried into every immediately
reachable tree or blob, because these are the ones that the client
might try to use as delta bases in a thin pack.
Change-Id: I5d5fdcf07e25ac9fc360e79a25dff491925e4101
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we need to append less than 20 bytes in order to fix a thin pack
and make it complete, we need to set the length of our file back to
the actual number of bytes used because the original SHA-1 footer was
not completely overwritten. That extra data will confuse the header
and footer fixup logic when it tries to read to the end of the file.
This isn't a very common case to occur, which is why we've never
seen it before. Getting a delta that requires a whole object which
uses less than 20 bytes in pack representation is really hard.
Generally a delta generator won't make these, because the delta
would be bigger than simply deflating the whole object. I only
managed to do this with a hand-crafted pack file where a 1 byte
delta was pointed to a 1 byte whole object.
Normally we try really hard to avoid truncating, because its
typically not safe across network filesystems. But the odds of
this occurring are very low. This truncation is done on a file
we have open for writing, will append more content onto, and is
a temporary file that we won't move into position for others to
see until we've validated its SHA-1 is sane. I don't think the
truncate on NFS issue is something we need to worry about here.
Change-Id: I102b9637dfd048dc833c050890d142f43c1e75ae
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The JSch bundle in Eclipse 3.4 does not export its packages with
version numbers. Use Require-Bundle on version 0.1.37 that comes
with Eclipse 3.4
There is no 0.1.37 in the maven repositories so the pom still refers
to 0.1.41 so the build can get the compile time dependencies right.
Bug: 308031
CQ: 3904 jsch Version: 0.1.37 (using Orbit CQ2014)
Change-Id: I12eba86bfbe584560c213882ebba58bf1f9fa0c1
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Since the API is changing relative to 0.7.0, we'll call our next
release 0.8.1. But until that gets released, builds from master
will be 0.8.0.qualifier.
Change-Id: I921e984f51ce498610c09e0db21be72a533fee88
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* push-sideband:
Reuse the line buffer between strings in PacketLineIn
http.server: Use TemporaryBuffer and compress some responses
Reduce multi-level buffered streams in transport code
Fix smart HTTP client buffer alignment
Use "ERR message" for early ReceivePack problems
Catch and report "ERR message" during remote advertisements
Wait for EOF on stderr before finishing SSH channel
Capture non-progress side band #2 messages and put in result
ReceivePack: Enable side-band-64k capability for status reports
Use more restrictive patterns for sideband progress scraping
Prefix remote progress tasks with "remote: "
Decode side-band channel number as unsigned integer
Refactor SideBandInputStream construction
Refactor SideBandOutputStream to be buffered
Change-Id: Ic9689e64e8c87971f2fd402cb619082309d5587f
Some transports actually provide stream buffering on their own,
without needing to be wrapped up inside of a BufferedInputStream in
order to smooth out system calls to read or write. A great example
of this is the JSch SSH client, or the Apache MINA SSHD server.
Both use custom buffering to packetize the streams into the encrypted
SSH channel, and wrapping them up inside of a BufferedInputStream
or BufferedOutputStream is relatively pointless.
Our SideBandOutputStream implementation also provides some fairly
large buffering, equal to one complete side-band packet on the main
data channel. Wrapping that inside of a BufferedOutputStream just to
smooth out small writes from PackWriter causes extra data copies, and
provides no advantage. We can save some memory and some CPU cycles
by letting PackWriter dump directly into the SideBandOutputStream's
internal buffer array.
Instead we push the buffering streams down to be as close to the
network socket (or operating system pipe) as possible. This allows
us to smooth out the smaller reads/writes from pkt-line messages
during advertisement and negotation, but avoid copying altogether
when the stream switches to larger writes over a side band channel.
Change-Id: I2f6f16caee64783c77d3dd1b2a41b3cc0c64c159
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of relying on our callers to wrap us up inside of a
BufferedOutputStream and using the proper block sizing, do the
buffering directly inside of SideBandOutputStream. This ensures
we don't get large write-throughs from BufferedOutputStream that
might overflow the configured packet size.
The constructor of SideBandOutputStream is also beefed up to check
its arguments and ensure they are within acceptable ranges for the
current side-band protocol.
Change-Id: Ic14567327d03c9e972f9734b8228178bc448867d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The tests were using a Locale.ROOT constant which was introduced
in Java 6. However, we need to retain Java 5 support.
Change-Id: I75c5648fcfc728a9aea2e839d2ad0320f5cf742f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Sasa Zivkov <sasa.zivkov@sap.com>
The support for NLS relies on java.util API to load a standard
ResourceBundle and then uses java reflection API to inject localized
strings into public String fields of the corresponding instance
of TranslationBundle.
Locale setting is supported per thread to enable concurrent threads
to use different locales. This is useful when JGit runs in a server
context where (error) messages might need to differ per-request to
suit the user's preference.
Change-Id: Ie0e63a0d7bb74eaad495dbe8248595d8a3a76883
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
If the build server is really busy, we might wait longer than 250 ms
before being interrupted, simply because one of our threads couldn't
be scheduled onto a CPU. Don't make that cause a test failure.
Instead tolerate longer than expected waits, but not shorter waits.
Change-Id: I64511eec24b49e33928451e4c8b8c124eddaf0c2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A 0 file mode in a DirCacheEntry is not a valid mode. To C git
such a value indicates the record should not be present. We already
were catching this bad state and exceptioning out when writing tree
objects to disk, but we did not fail when writing the dircache back
to disk. This allowed JGit applications to create a dircache file
which C git would not like to read.
Instead of checking the mode during writes, we now check during
mutation. This allows application bugs to be detected sooner and
closer to the cause site. It also allows us to avoid checking most
of the records which we read in from disk, as we can assume these
are formatted correctly.
Some of our unit tests were not setting the FileMode on their test
entry, so they had to be updated to use REGULAR_FILE.
Change-Id: Ie412053c390b737c0ece57b8e063e4355ee32437
Originally: http://thread.gmane.org/gmane.comp.version-control.git/128214/focus=128213
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Adam W. Hawks <awhawks@writeme.com>
Actually set the range of versions we are willing to accept for
each package we import, lest we import something in the future
that isn't compatible with our needs.
Change-Id: I25dbbb9eaabe852631b677e0c608792b3ed97532
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The supplied test case comes out of the example tree identified by
Robert de Wilde and Ilari on #git:
$ git ls-tree -rt a54f1a85ebf6a7f53aa60a45a1be33f8b078fb7e
040000 tree bfe058ad536cdb12e127cde63b01472c960ea105 A
040000 tree 4b825dc642 A/A
040000 tree 4b825dc642 A/B
100644 blob abbbfafe3129f85747aba7bfac992af77134c607 B
In this tree, "B" was being skipped because "A/A" as an empty tree
was immediately followed by "A/B", also an empty tree, but the
ObjectWalk broke out too early and never visited "B".
Bug: 286653
Change-Id: I25bcb0bc99d0cbbbdd9c2bd625ad6a691a6d0335
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We didn't skip the correct number of bytes when we skipped over an
unrecognized but optional dircache extension. We missed skipping
the 8 byte header that makes up the extension's name and length.
We also didn't include the skipped extension's payload as part of
our index checksum, resuting in a checksum failure when the index
was done reading. So ensure we always scan through a skipped
section and include it in the checksum computation.
Add a test case for a currently unsupported index extension, 'ZZZZ',
to verify we can still read the DirCache object even though we
don't know what 'ZZZZ' is supposed to mean.
Bug: 301287
Change-Id: I4bdde94576fffe826d0782483fd98cab1ea628fa
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This test doesn't actually depend upon the large data set we have
in the RepositoryTestCase, so drop that from the dependency and
use the more simple LocalDiskRepositoryTestCase instead.
Change-Id: I0fd4affe1dd5ec86e8c3253db42df11d3b612e36
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The unsetSection method can be used to delete an entire configuration
block, such as a [branch ""] or [remote ""] section in a file.
Change-Id: I93390c9b2187eb1b0d51353518feaed83bed2aad
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Config was confusing the following two variables when writing the
file back to text format:
[my]
empty =
enabled
When parsed, we say that my.empty has 1 value, null, and my.enabled
is an empty string value that in boolean context should be evaluated
as true.
Saving this configuration file back to text format was ignoring the
null value for my.empty, producing a completely different file than
what Config read:
[my]
empty
enabled
Instead handle the writing differently to ensure the original format
is output. New tests cases cover the expected behavior and return
values from accessor methods.
Change-Id: Id37379ce20cb27e3330923cf989444dd9f2bdd96
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Annotated tags created with C Git versions before the introduction
of c818566 ([PATCH] Update tags to record who made them, 2005-07-14),
do not have a "tagger" line present in the object header. This line
did not appear in C Git until v0.99.1~9.
Ancient projects such as the Linux kernel contain such tags, for
example Linux 2.6.12 is older than when this feature first appeared
in C Git. Linux v2.6.13-rc4 in late July 2005 is the first kernel
version tag to actually contain a tagger line.
It is therefore acceptable for the header to be missing, and for
the RevTag.getTaggerIdent() method to return null.
Since the Javadoc for getTaggerIdent() already explained that the
identity may be null, we just need to test that this is true when
the header is missing, and allow the ObjectChecker to pass anyway.
Change-Id: I34ba82e0624a0d1a7edcf62ffba72260af6f7e5d
See: http://code.google.com/p/gerrit/issues/detail?id=399
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Technically our project name is "JGit", not "Java Git". In fact
there is already another project called "JavaGit" (no space) that we
don't want to become confused with. Ensure we always call ourselves
"JGit" in user visible assets, like the bundle name.
Other Eclipse products list their provider as "Eclipse.org",
not "eclipse.org". So list ourselves that way in all of our
plugin.properties files.
Change-Id: Ibcea1cd6dda2af757a8584099619fc23b7779a84
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* ref-abstract:
Optimize RefAdvertiser performance by avoiding sorting
branch: Add -m option to rename a branch
Replace writeSymref with RefUpdate.link
Rewrite reference handling to be abstract and accurate
Create new RefList and RefMap utility types
Change-Id: If43aacf5aa4013edbd0a6e84d84c4f9e94de5be0
By using RefUpdate for symbolic reference creation we can reuse
the logic related to updating the reflog with the event, without
needing to expose something such as the legacy ReflogWriter class
(which we no longer have).
Applications using writeSymref must update their code to use the
new pattern of changing the reference through the updateRef method:
String refName = "refs/heads/master";
RefUpdate u = repository.updateRef(Constants.HEAD);
u.setRefLogMessage("checkout: moving to " + refName, false);
switch (u.link(refName)) {
case NEW:
case FORCED:
case NO_CHANGE:
// A successful update of the reference
break;
default:
// Handle the failure, e.g. for older behavior
throw new IOException(u.getResult());
}
Change-Id: I1093e1ec2970147978a786cfdd0a75d0aebf8010
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This commit actually does three major changes to the way references
are handled within JGit. Unfortunately they were easier to do as
a single massive commit than to break them up into smaller units.
Disambiguate symbolic references:
---------------------------------
Reporting a symbolic reference such as HEAD as though it were
any other normal reference like refs/heads/master causes subtle
programming errors. We have been bitten by this error on several
occasions, as have some downstream applications written by myself.
Instead of reporting HEAD as a reference whose name differs from
its "original name", report it as an actual SymbolicRef object
that the application can test the type and examine the target of.
With this change, Ref is now an abstract type with different
subclasses for the different types.
In the classical example of "HEAD" being a symbolic reference to
branch "refs/heads/master", the Repository.getAllRefs() method
will now return:
Map<String, Ref> all = repository.getAllRefs();
SymbolicRef HEAD = (SymbolicRef) all.get("HEAD");
ObjectIdRef master = (ObjectIdRef) all.get("refs/heads/master");
assertSame(master, HEAD.getTarget());
assertSame(master.getObjectId(), HEAD.getObjectId());
assertEquals("HEAD", HEAD.getName());
assertEquals("refs/heads/master", master.getName());
A nice side-effect of this change is the storage type of the
symbolic reference is no longer ambiguous with the storge type
of the underlying reference it targets. In the above example,
if master was only available in the packed-refs file, then the
following is also true:
assertSame(Ref.Storage.LOOSE, HEAD.getStorage());
assertSame(Ref.Storage.PACKED, master.getStorage());
(Prior to this change we returned the ambiguous storage of
LOOSE_PACKED for HEAD, which was confusing since it wasn't
actually true on disk).
Another nice side-effect of this change is all intermediate
symbolic references are preserved, and are therefore visible
to the application when they walk the target chain. We can
now correctly inspect chains of symbolic references.
As a result of this change the Ref.getOrigName() method has been
removed from the API. Applications should identify a symbolic
reference by testing for isSymbolic() and not by using an arcane
string comparsion between properties.
Abstract the RefDatabase storage:
---------------------------------
RefDatabase is now abstract, similar to ObjectDatabase, and a
new concrete implementation called RefDirectory is used for the
traditional on-disk storage layout. In the future we plan to
support additional implementations, such as a pure in-memory
RefDatabase for unit testing purposes.
Optimize RefDirectory:
----------------------
The implementation of the in-memory reference cache, reading, and
update routines has been completely rewritten. Much of the code
was heavily borrowed or cribbed from the prior implementation,
so copyright notices have been left intact as much as possible.
The RefDirectory cache no longer confuses symbolic references
with normal references. This permits the cache to resolve the
value of a symbolic reference as late as possible, ensuring it
is always current, without needing to maintain reverse pointers.
The cache is now 2 sorted RefLists, rather than 3 HashMaps.
Using sorted lists allows the implementation to reduce the
in-memory footprint when storing many refs. Using specialized
types for the elements allows the code to avoid additional map
lookups for auxiliary stat information.
To improve scan time during getRefs(), the lists are returned via
a copy-on-write contract. Most callers of getRefs() do not modify
the returned collections, so the copy-on-write semantics improves
access on repositories with a large number of packed references.
Iterator traversals of the returned Map<String,Ref> are performed
using a simple merge-join of the two cache lists, ensuring we can
perform the entire traversal in linear time as a function of the
number of references: O(PackedRefs + LooseRefs).
Scans of the loose reference space to update the cache run in
O(LooseRefs log LooseRefs) time, as the directory contents
are sorted before being merged against the in-memory cache.
Since the majority of stable references are kept packed, there
typically are only a handful of reference names to be sorted,
so the sorting cost should not be very high.
Locking is reduced during getRefs() by taking advantage of the
copy-on-write semantics of the improved cache data structure.
This permits concurrent readers to pull back references without
blocking each other. If there is contention updating the cache
during a scan, one or more updates are simply skipped and will
get picked up again in a future scan.
Writing to the $GIT_DIR/packed-refs during reference delete is
now fully atomic. The file is locked, reparsed fresh, and written
back out if a change is necessary. This avoids all race conditions
with concurrent external updates of the packed-refs file.
The RefLogWriter class has been fully folded into RefDirectory
and is therefore deleted. Maintaining the reference's log is
the responsiblity of the database implementation, and not all
implementations will use java.io for access.
Future work still remains to be done to abstract the ReflogReader
class away from local disk IO.
Change-Id: I26b9287c45a4b2d2be35ba2849daa316f5eec85d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These types can be used by RefDatabase implementations to manage
the collection.
A RefList stores items sorted by their name, and is an immutable
type using copy-on-write semantics to perform modifications to
the collection. Binary search is used to locate an existing item
by name, or to locate the proper insertion position if an item does
not exist.
A RefMap can merge up to 3 RefList collections at once during its
entry iteration, allowing items in the resolved or loose RefList
to override items by the same name in the packed RefList.
The RefMap's goal is O(log N) lookup time, and O(N) iteration time,
which is suitable for returning from a RefDatabase. By relying on
the immutable RefList we might be able to make map construction
nearly constant, making Repository.getAllRefs() an inexpensive
operation if the caches are current. Since modification is not
common, changes require up to O(N + log N) time to copy the internal
list and collapse or expand the list's array. As most changes
are made to the loose collection and not the packed collection,
in practice most changes would require less than the full O(N)
time, due to a significantly smaller N in the loose list.
Almost complete test coverage is included in the corresponding
unit tests. A handful of methods on RefMap are not tested in this
change, as writing the proper test depends on a future refactoring
of how the Ref class represents symbolic reference names.
Change-Id: Ic2095274000336556f719edd75a5c5dd6dd1d857
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Not all occurrences of ".git" are replaced by this constant, only
those where it actually refers to the directory with that name, i.e
not the ".git" directory suffix.
Asserts and comment are also excluded from replacement.
Change-Id: I65a9da89aedd53817f2ea3eaab4f9c2bed35d7ee
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Later we are going to add support for smart HTTP, which requires us to
buffer at least some of the request created by a client before we ship
it to the server. For many requests, we can fit it completely into a
1 MiB buffer, but if it doesn't we can drop back to using the chunked
transfer encoding to send an unknown stream length.
Rather than recoding the block based memory buffer, we refactor the
local file overflow strategy into a subclass, allowing the HTTP client
code to replace this portion of the logic with its own approach to
start the chunked encoding request.
Change-Id: Iac61ea1017b14e0ad3c4425efc3d75718b71bb8e
Signed-off-by: Shawn O. Pearce <sop@google.com>
The multi_ack_detailed extension breaks out the "ACK %s continue" status
code into "ACK %s common" and "ACK %s ready" states, making it easier to
discover which objects are truely common, and which objects are simply
on a chain the server doesn't care learning about.
Change-Id: Ie8e907424cfbbba84996ca205d49eacf339f9d04
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These routines create a fairly clean DSL for writing out the
structure of a repository in a test case. Abstract them into
a helper class that we can reuse in other test environments.
Change-Id: I55cce3d557e1a28afe2fdf37b3a5b67e2651c9f1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Other test suites may find this useful, especially when trying
to defeat the pack file compression with random data files.
Change-Id: Ic00a4ac626af7a1c94d18ee99305e295b267b1a3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since Robin reverted using the maven-bundle-plugin to produce the
OSGi manifest, there is no reason for us to reference it from our
build process anymore.
Also, when Robin reverted the to the Eclipse way of doing things,
we failed to update the ignore files to ignore our generated files
but not ignore our tracked .classpath.
Finally, we cannot delete the MANIFEST.MF file during a Maven build,
as this is once again a source file.
Change-Id: I53f77f2002cb4285f728968829560e835651e188
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This restores the ability to build using just Eclipse without
strange procedures, extra plugins and it is again possible to
work on both JGit and EGit in the same Eclipse workspace with
ease.
Change-Id: I0af08127d507fbce186f428f1cdeff280f0ddcda
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Per CQ 3559 "JGit - Eugene Myers O(ND) difference algorithm" we
have approval to check this into our master branch.
* cq-diff:
Add file content merge algorithm
Add performance tests for MyersDiff
Add javadoc comments, remove unused code, shift comments to correct place
Fixed MyersDiff to be able to handle more than 100k
Fix some warnings regarding unnecessary imports and accessing static methods
Add the "jgit diff" command
Prepare RawText for diff-index and diff-files
Add a test class for Myers' diff algorithm
Add Myers' algorithm to generate diff scripts
Add set to IntList
Conflicts:
org.eclipse.jgit/src/org/eclipse/jgit/diff/RawText.java
Change-Id: Ia8e98d81ba1ab52f84d0258a40e6ef5eece9a5b1
CC: Christian Halstrick <christian.halstrick@sap.com>
Adds the file content merge alorithm and tests for merge to jgit.
The merge algorithm:
- Gets as input parameters the common base, the two new contents
called "ours" and "theirs".
- Computes the Edits from base to ours and from base to theirs with
the help of MyersDiff.
- Iterates over the edits.
- Independent edits from ours or from theirs will just be applied
to the result.
- For conflicting edits we first harmonize the ranges of the edits
so that in the end we have exactly two edits starting and ending
at the same points in the common base. Then we write the two
conclicting contents into the result stream.
Change-Id: I411862393e7bf416b6f33ca55ec5af608ff4663
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
[sp: Fixed up two awkard comments in documentation.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The UnionInputStream utility class combines multiple sequential
InputStreams so they appear to the caller as a single stream with
no gaps. This can be used to concentate streams coming from multiple
independent HTTP connections (for example).
The companion unit test covers the class's full functionality.
Change-Id: I0676c7b5e082a5886bf0e8f43f9fd6c46a666228
Signed-off-by: Shawn O. Pearce <sop@google.com>
Tycho isn't production ready for projects like JGit to be using as
their primary build driver. Some problems we ran into with Tycho
0.6.0 that are preventing us from using it are:
* Tycho can't run offline
The P2 artifact resolver cannot perform its work offline. If the
build system has no network connection, it cannot compile a
project through Tycho. This is insane for a distributed version
control system where developers are used to being offline during
development and local testing.
* Magic state in ~/.m2/repository/.meta/p2-metadata.properties
Earlier iterations of this patch tried to use a hybrid build,
where Tycho was only used for the Eclipse specific feature and P2
update site, and maven-bundle-plugin was used for the other code.
This build seemed to work, but only due to magic Tycho specific
state held in my local home directory. This means builds are not
consistently repeatable across systems, and lead me to believe
I had a valid build, when in fact I did not.
* Manifest-first build produces incomplete POMs
The POM created by the manifest-first build format does not
contain the dependency chain, leading a downstream consumer to
not import the runtime dependencies necessary to execute the
bundle it has imported. In JGit's case, this means JSch isn't
included in our dependency chain.
* Manifest-first build produces POMs unreadable by Maven 2.x
JGit has existing application consumers who are relying on
Maven 2.x builds. Forcing them to step up to an alpha release
of Maven 3 is simply unacceptable.
* OSGi bundle export data management is tedious
Editing each of our pom.xml files to mark a new release is
difficult enough as it is. Editing every MANIFEST.MF file to
list our exported packages and their current version number is
something a machine should do, not a human. Yet the Tycho OSGi
way unfortunately demands that a human do this work.
* OSGi bundle import data management is tedious
There isn't a way in the MANIFEST.MF file format to reuse the
same version tags across all of our imports, but we want to have
a consistent view of our dependencies when we compile JGit.
After wasting more than 2 full days trying to get Tycho to work,
I've decided its a lost cause right now. We need to be chasing down
bugs and critical features, not trying to bridge the gap between
the stable Maven repository format and the undocumented P2 format
used only by Eclipse.
So, switch the build to use Apache Felix's maven-bundle-plugin.
This is the same plugin Jetty uses to produce their OSGi bundle
manifests, and is the same plugin used by the Apache Felix project,
which is an open-source OSGi runtime. It has a reasonable number
of folks using it for production builds, and is running on top of
the stable Maven 2.x code base.
With this switch we get automatically generated MANIFEST.MF files
based on reasonably sane default rules, which reduces the amount
of things we have to maintain by hand. When necessary, we can add
a few lines of XML to our POMs to tweak the output.
Our build artifacts are still fully compatible with Maven 2.x, so
any downstream consumers are still able to use our build products,
without stepping up to Maven 3.x. Our artifacts are also valid as
OSGi bundles, provided they are organized on disk into a repository
that the runtime can read.
With maven-bundle-plugin the build runs offline, as much as Maven
2.x is able to run offline anyway, so we're able to return to a
distributed development environment again.
By generating MANIFEST.MF at the top level of each project (and
therefore outside of the target directory), we're still compatible
with Eclipse's PDE tooling. Our projects can be imported as standard
Maven projects using the m2eclipse plugin, but the PDE will think
they are vaild plugins and make them available for plugin builds,
or while debugging another workbench.
This change also completely removes Tycho from the build.
Unfortunately, Tycho 0.6.0's pom-first dependency resolver is broken
when resolving a pom-first plugin bundle through a manifest-first
feature package, so bundle org.eclipse.jgit can't be resolved,
even though it might actually exist in the local Maven repository.
Rather than fight with Tycho any further, I'm just declaring it
plugina-non-grata and ripping it out of the build.
Since there are very few tools to build a P2 format repository, and
no documentation on how to create one without running the Eclipse
UI manually by poking buttons, I'm declaring that we are not going
to produce a P2 update site from our automated builds.
Change-Id: If7938a86fb0cc8e25099028d832dbd38110b9124
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This makes the jgit command line behave like the C Git implementation
in the respect.
These variables are not recognized in the core, though we add support
to do the overrides there. Hence other users of the JGit library, like
the Eclipse plugin and others, will not be affected.
GIT_DIR
The location of the ".git" directory.
GIT_WORK_TREE
The location of the work tree.
GIT_INDEX_FILE
The location of the index file.
GIT_CEILING_DIRECTORIES
A colon (semicolon on Windows) separated list of paths that
which JGit will not cross when looking for the .git directory.
GIT_OBJECT_DIRECTORY
The location of the objects directory under which objects are
stored.
GIT_ALTERNATE_OBJECT_DIRECTORIES
A colon (semicolon on Windows) separated list of object directories
to search for objects.
In addition to these we support the core.worktree config setting when
the git directory is set deliberately instead of being found.
Change-Id: I2b9bceb13c0f66b25e9e3cefd2e01534a286e04c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
An extra flag when creating a RefUpdate object allows the
caller to destroy the symref and replace it with an object
ref, a.k.a. detached HEAD.
Change-Id: Ia88d48eab1eb4861ebfa39e3be9258c3824a19db
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The test data is expected to have unix new lines by tests, but it
is converted to crlf on Windows platform (with msys git). As result
DiffFormatterReflowTest tests fail. To prevent this problem,
crlf conversion is disbled for test data related to that test.
Bug: 295077
Change-Id: I67d3ed543fcc38647041896146de12b1781ec6be
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Add some tests which make sure that the diff algorithm really behaves in the
promised O(N*D) manner. This tests compute diffs between multiple big chunks
of data, measure time for computing the diffs and fail if the measured times
are off O(N*D) by more than a factor 10
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Change-Id: I8e1e0be60299472828718371b231f1d8a9dc21a7
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Not all of our test cases really require the sample data packs,
and we are better off not using them because its hard to see exactly
what condition a test is testing when looking only at the Java code.
Clarify the dependency by only making the packs available when
there is a real need for it.
Change-Id: Id8a76ee7ee1f7efba585be4bed19a8fb5b3b3585
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This test depends upon the external git binary, and this isn't
really a pure Java test like our module tries to claim itself is.
So we move it out to exttst with other tests that require additional
external resources and/or executable code.
Change-Id: Ic9be0280c8bb50a5768336c64de794eb0a492b3d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Drop our simple and stupid jgit.sh and instead rely upon Maven
for the command line based build. Maven is relatively simple to
download and install, and doesn't require the entire Eclipse IDE.
To avoid too much refactoring of the current code we reuse the
existing src/ directory within each plugin, and treat each of
the existing OSGI bundles as one Maven artifact.
The command line wrapper jgit.sh no longer works in the uncompiled
state, as we don't know where to obtain our JSch or args4j from.
Developers will now need to compile it with `mvn package`, or run
our Main class from within an IDE which has the proper classpath.
Bug: 291265
Change-Id: I355e95fa92fa7502651091d2b651be6917a26805
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way we depend upon the MANIFEST.MF to define our classpath
and our build will act more like any other OSGI bundle build.
Change-Id: I9e1f1f5a0bccb0ab0e39e49b75fb400fea446619
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
According the javadoc, and implied by the name of the class, NB
is about network byte order. The purpose of moving the IO only,
and non-byte order related functions to another class is to
make it easier for new contributors to understand that they
can use these functions in general and it's also makes it easier
to understand where to put new IO related utility functions
Change-Id: I4a9f6b39d5564bc8a694b366e7ff3cc758c5181b
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Our project coding conventions do not want trailing whitespace at
the end of a source code line. Configure Eclipse to automatically
remove them when saving any Java source file.
Change-Id: I9701366b3b1240879761b30556e6ff416e969e1d
Reviewed-by: Mykola Nikishov <mn@mn.com.ua>
Reviewed-by: Alex Blewitt <alex.blewitt@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As discussed on the egit-dev mailing list, we prefer not to have
trailing whitespace in our source code. Correct all currently
offending lines by trimming them.
Change-Id: I002b1d1980071084c0bc53242c8f5900970e6845
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way dependencies are described by the MANIFEST.MF,
and the same build tools can be used to compile the tests.
Change-Id: I4dc926148410ecbadcf71b9474aeeb509691aa32
All 3rd party dependencies must come from orbit to comply
with Eclipse development process.
Change-Id: Ia43892ab6d0169f8335c1a41b37e8c12e94cafe2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When reading commits the system default encoding was used if no
encoding was specified in the commit. The patch modifies the test
to add a check that commit message was encoded correctly (the
test fails on old implementation if system encoding is not UTF-8)
and fixes Commit.decode() method to use UTF-8 if encoding is not
specified in the commit object.
Change-Id: I27101da3c2eb6edd0c4a9e4c0523e48b286e3cd5
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In the pre-historic commit 6d87484b4dee5671a38e64a8e4990dff40a4874f
two tests became identical. Remove one of them.
Change-Id: I6182ecd4db0162d87a5f4577005b2bf4d5e8c89f
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Some applications may wish to modify an int list.
Bug: 291083
Eclipse-CQ: 3559
Change-Id: Iea871443ec661230aec92397229f1eda6c74216f
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Only one test class actually needs this function, so instead of
us inheriting it down into every test, move it to that one class.
Change-Id: I5700ca48df4177153f2b3861dec7c538c621e775
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This test doesn't work because it requires a pack file which we have
lost to the ages. We couldn't include it because the pack was actually
a copy of the GPL'd C git.git project, and was there to test some sort
of corner case that the test never documented properly.
Change-Id: I282ee1c6a637a8654df93a3847507a6c60e4cfab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Honor the configuration parameter core.logAllRefUpdates when writing
reflogs. Instead of writing reflog entries always only write
reflogs if this parameter is set to true or if the corresponding
file in the <git-dir>/logs directory already exists. In other words:
if you are updating a ref and this parameter is set to false and
there is no file corresponding to your ref in the <git-dir>/logs
folder then no reflog will be written.
This is a fix for the issue http://code.google.com/p/egit/issues/detail?id=4
Change-Id: I908e4c77e3630dc3223b2d2a47cb4534dbe4ed42
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Per CQ 3448 this is the initial contribution of the JGit project
to eclipse.org. It is derived from the historical JGit repository
at commit 3a2dd9921c8a08740a9e02c421469e5b1a9e47cb.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Chris Aniszczyk