Add the ability to checkout a branch to the working tree.
Bug: 330860
Change-Id: Ie06b9e799a9e1be384da0b8996efa7209b32eac3
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
There was a bug introduced by commit 0e815fe. For non-versioned files
the merge algorithm detected an incoming deletion from THEIRS.
Consequently such files were deleted. That's a severe bug which was
fixed by more precisely detecting incoming deletions.
Change-Id: I4385d3c990db11d62e371a385dc8ee89841db84a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Philipp Thun <philipp.thun@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This is a first iteration to implement Rebase. At the moment, this
does not implement --continue and --skip, so if the first
conflict is found, the only option is to --abort the command.
Bug: 328217
Change-Id: I24d60c0214e71e5572955f8261e10a42e9e95298
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Instead of copying up to 4 fields from the parent iterator each time a
child iterator is initialized and used, construct a single state
object that contains the 4 fields, and pass that one state object
through to the child. This makes it easier to add additional state
fields that must be inherited, at the slight expense of an extra
object allocation per TreeWalk, and an extra level of field
indirection whenever the options, nameEncoder, or read buffer is
required by the iterator.
Change-Id: Ic4603c33b772d7a45f9c81140537d51945688fcb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Naming these inner classes ensures that stack traces which contain
them will give us useful information about which filter is involved in
the trace, rather than the generated names $1, $2, etc. This makes it
much easier to understand a stack trace at a glance.
Change-Id: Ia6a75fdb382ff6461e02054d94baf011bdeee5aa
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In the case where DirCacheCheckout was used to checkout a tree
without taking HEAD into account (e.g. during a clone or hard reset)
we didn't handle conflicts correctly. E.g. if there are conflicts
(entries with stage != 0) in the index and we tried to hard reset
we have been processing the conflicting pathes multiple times (once
for every stage). With this fix we will update the index with the
entry from the "merge" state (the state we want checkout) when we
detect existing conflicts.
Change-Id: Iffbddccaa588cf0d1460a5e44dabaf540d996e26
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
* rename-detection:
RenameDetector: Only scan deletes if adds exist
SimilarityRenameDetector: Initialize sizes to 0
SimilarityRenameDetector: Avoid allocating source index
SimilarityRenameDetector: Only attempt to index large files once
SimilarityIndex: Don't overflow internal counter fields
SimilarityIndex: Accept files larger than 8 MB
SimilarityIndex: Correct comment explaining the logic
* fs-fsync:
Remove unnecessary flush calls from LockFile
Remove unnecessary region locking from LockFile
Support core.fsyncRefFiles option
Support core.fsyncObjectFiles option
Simplify LockFile write(ObjectId) case
Rewrite the initialization of the encoding tables to be more clear,
but slightly slower to setup. We generally perfer a clear definition
of the data over a slightly slower class load time.
Change-Id: I0c7f89b6ab82dcf71525ffb69a388c312c195913
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since we have already modified this class to localize an error
message, we might as well strip it down to contain only the
functionality we need, or might ever use.
To keep this simple to review we don't adjust formatting right
away, so code that was buried inside of an if or else block whose
condition was removed might not have the correct indentation anymore.
We can fix this with a later reformatting change.
Change-Id: I2996aaa704e9d6182e5500c7a63240d5e9d722cc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When DirCacheCheckout should be used to checkout only one
tree (reset --hard, clone) then we had to use the standard
constructor and specify null as value for head. This change
adds explicit constructors not taking HEAD and documents
that.
Bug: 330021
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Fanout level notes trees are combined back together into a flat leaf
level tree if during a removal of a subtree there are less than 3/4 of
the fanout subtrees still existing, and the size of the combined leaf
is under the 256 split limit noted above.
This rule is used because deletes are less common than insertions, and
SHA-1's relatively uniform distribution suggests that with only 192
subtrees existing in the fanout, there should be approximately 192
names in the combined replacement leaf tree.
Change-Id: Ia9d145ffd5454982509fc40906bc4dbbf2b13952
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Leaf level notes trees are split into a new fan-out tree if an
insertion occurs and the tree already contains >= 256 notes in it.
The splitting may occur multiple times if all of the notes have the
same prefix; in the worst case this produces a tree path such as
"00/00/00/00/00/00/00/00/00/00/00/00/00/00/00/00/00/00/00/be" if all
of the notes begin with zeros.
Change-Id: I2d7d98f35108def9ec49936ddbdc34b13822a3c7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some algorithms need to be able to iterate through all notes within a
particular bucket, such as when splitting or combining a bucket.
Exposing an Iterator<Note> makes this traversal possible.
For a LeafBucket the iteration is simple, its over the sorted array of
elements. For FanoutBucket its a bit more complex as the iteration
needs to union the iterators of each fanout bucket, lazily loading any
buckets that aren't already in-memory.
Change-Id: I3d5279b11984f44dcf0ddb14a82a4b4e51d4632d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This is necessary to allow applications to wrap the note tree in
a commit and update the note branch with the new state.
Change-Id: Idbd7ead4a1b16ae2b64a30a4a01a29cfed548cdf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
NoteMap now supports editing in-memory, allowing applications to
modify the NoteMap once it has been loaded from the branch. The
ability to write the branch back to tree objects is not yet done,
so the edits are strictly transient.
Change-Id: I63448954abfca2a8e3e95369cd84c0d1176cdb79
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The lock file protocol relies on the atomic creation of a standardized
name in the parent directory of the file being updated. Since the
creation is atomic, at most one thread in any process can succeed on
this creation, and all others will fail. While the lock file exists,
that file is private to the thread that is writing it, and no others
will attempt to read or modify the file.
Consequently the use of the region level locks around the file are
unnecessary, and may actually reduce performance when using NFS, SMB,
or some other sort of remote filesystem that supports locking.
Change-Id: Ice312b6fb4fdf9d36c734c3624c6d0537903913b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If core.fsyncRefFiles is set to true, fsync is used whenever a
reference file is updated, ensuring the file contents are also
written to disk. This can help to prevent empty ref files after
a system crash when using a filesystem such as HFS+ where data
writes may be delayed.
Change-Id: Ie508a974da50f63b0409c38afe68772322dc19f1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some repositories may be on really unstable filesystems, but still
want to have good reliability when objects are written to disk. If
core.fsyncObjectFiles is set to true, request the JVM to ensure the
data is written before returning success to the caller of insert.
The option defaults to false because it should be useless on any
filesystem that orders writes and metadata, such as ext3 mounted with
data=ordered (or data=journal). But it may be useful on some systems
(especially HFS+) where file content may flush to the disk
independently of filesystem structure changes.
Because FileChannel.force(boolean) only claims to ensure data is
written if it was written using the write(ByteBuffer) method of
FileChannel, redirect all writes when using fsyncObjectFiles to go
through the FileChannel interface instead of through the older style
OutputStream interface. This may not be necessary on all JVMs, but
its more portable to follow the definition than the common behavior.
Change-Id: I57f6b6bb7e403c07fbae989dbf3758eaf5edbc78
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If there are only deletes, don't need perform rename or copy
detection. There are no adds (aka destinations) for the deletes
to match against.
Change-Id: I00fb90c509fa26a053de561dd8506cc1e0f5799a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Setting the array elements to -1 is more expensive than relying on
the allocator to zero the array for us first. Shifting the code to
always add 1 to the size (so an empty file is actually 1 byte long)
allows us to detect an unloaded size by comparing to 0, thus saving
the array fill calls.
Change-Id: Iad859e910655675b53ba70de8e6fceaef7cfcdd1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the only file added is really small, and all of the deleted
files are really big, none of the permutations will match up due
to the sizes being too far apart to fit the current rename score.
Avoid allocating the really big deleted SimilarityIndex by deferring
its construction until at least one add along that row has a
reasonable chance of matching it.
This avoids expending a lot of CPU time looking at big deleted
binary files when a small modified text file was broken due to a
high percentage of changed lines.
Change-Id: I11ae37edb80a7be1eef8cc01d79412017c2fc075
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If a file fails to index the first time the loop encounters it, the
file is likely to fail to index again on the next row. Rather than
wasting a huge amount of CPU to index it again and fail, remember
which destination files failed to index and skip over them on each
subsequent row.
Because this condition is very unlikely, avoid allocating the BitSet
until its actually needed. This keeps the memory usage unaffected
for the common case.
Change-Id: I93509b28b61a9bba8f681a7b4df4c6127bca2a09
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The counter portion of each pair is only 32 bits wide, but is part
of a larger 64 bit integer. If the file size was larger than 4 GB
the counter could overflow and impact the key, changing the hash,
and later resulting in an incorrect similarity score.
Guard against this overflow condition by capping the count for each
record at 2^32-1. If any record contains more than that many bytes
the table aborts hashing and throws TableFullException.
This permits the index to scan and work on files that exceed 4 GB
in size, but only if the file contains more than one unique block.
The index throws TableFullException on a 4 GB file containing all
zeros, but should succeed on a 6 GB file containing unique lines.
The index now uses a 64 bit accumulator during the common scoring
algorithm, possibly resulting in slower summations. However this
index is already heavily dependent upon 64 bit integer operations
being efficient, so increasing from 32 bits to 64 bits allows us
to correctly handle 6 GB files.
Change-Id: I14e6dbc88d54ead19336a4c0c25eae18e73e6ec2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Files bigger than 8 MB (2^23 bytes) tended to overflow the internal
hashtable, as the table was capped in size to 2^17 records. If a
file contained 2^17 unique data blocks/lines, the table insertion
got stuck in an infinite loop as the able couldn't grow, and there
was no open slot for the new item.
Remove the artifical 2^17 table limit and instead allow the table
to grow to be as big as 2^30. With a 64 byte block size, this
permits hashing inputs as large as 64 GB.
If the table reaches 2^30 (or cannot be allocated) hashing is
aborted. RenameDetector no longer tries to break a modify file pair,
and it does not try to match the file for rename or copy detection.
Change-Id: Ibb4d756844f4667e181e24a34a468dc3655863ac
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This comment was wrong, due to a copy-and-paste error. Here the
code is looking at records of dst that do not exist in src, and
are skipping past them to find another match.
Change-Id: I07c1fba7dee093a1eeffcf7e0c7ec85446777ffb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In order to safely edit a notes tree, NoteMap needs to retain any
non-note tree entries it read from the source tree and put them
back out into the modified tree when it commits a new version of
the note branch.
Remember any tree entries that didn't look like a note during
the parsing of the tree, so they can be put into a TreeFormatter
later when the tree writes to the repository.
Change-Id: Ia284af7e7866da35db35374c6c5869f00c857944
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of reading a note tree recursively up front when the NoteMap
is loaded, read only the root tree and load subtrees on demand when
they are accessed by the application. This gives a lower latency
to read a note for the recent commits on a branch, as only the paths
that are needed get read.
Given a 2/38 style fanout, the tree will fully load when 256 objects
have been accessed by the application. But unlike the prior version
of NoteMap, the NoteMap will load faster and answer lookups sooner,
as the loading time for all 256 levels is spread out across each of
the get() requests.
Given a 2/2/36 style fanout, the tree won't need to fully load until
about 65,536 objects are accessed.
To simplify the implementation we only support the flat layout (all
notes in the top level tree), or a 2/38, 2/2/36, 2/2/2/34, through
2/.../2 style fanout. Unlike C Git we don't support reading the old
experimental 4/36 fanout. This is sufficient because C Git won't
create the 4/36 style fanout when creating or updating a notes tree,
and there really aren't any in the wild today.
Change-Id: I6099b35916a8404762f31e9c11f632e43e0c1bfd
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The NoteMap makes it easy to read a small notes tree as created by
the `git notes` command in C Git. To make the initial implementation
simple a notes tree is read recursively into a map in memory.
This is reasonable if the application will need to access all notes,
or if there are less than 256 notes in the tree, but doesn't behave
well when the number of notes exceeds 256 and the application
doesn't need to access all of them.
We can later add support for lazily loading different subpaths,
thus fixing the large note tree problem described above.
Currently the implementation only supports reading. Writing notes
is more complex because trees need to be expanded or collapsed at
the exact 256 entry cut-off in order to retain the same tree SHA-1
that C Git would use for the same content. It also needs to retain
non-note tree entries such as ".gitignore" or ".gitattribute" files
that might randomly appear within a notes tree. We can also add
writing support later.
Change-Id: I93704bd84ebf650d51de34da3f1577ef0f7a9144
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
When setting up an SSH connection, use the caller supplied
CredentialsProvider, if one has been given to the Transport
or was defined as the default.
The CredentialsProvider is re-wrapped as a JSch UserInfo,
allowing the connection to use this for user interactive
prompts. This give a unified API for authentication on
any transport type.
Change-Id: Id3b4cf5bfd27a23207cdfb188bae3b78e71e02c0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This permits applications to set their preferred credentials UI
implementation once, rather than needing to define it on every
single Transport instance they open.
Change-Id: I010550de1a6becab27f7aa5a9901df5a1c7e74bd
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This change is based on http://egit.eclipse.org/r/#change,1652
by David Green. The change adds the concept of a CredentialsProvider
which can be registered for git transports and which is
responsible to return credential-related data like passwords and
usernames. Whenenver the transports detects that an authentication
with certain credentials has to be done it will ask the
CredentialsProvider for this data. Foreseen implementations for
such a Provider may be a EGitCredentialsProvider (caching
credential data entered e.g. in the Clone-Wizzard) or a NetRcProvider
(gathering data out of ~/.netrc file).
Bug: 296201
Change-Id: Ibe13e546b45eed3e193c09ecb414bbec2971d362
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: David Green <dgreen99@gmail.com>
The auth-scheme token (like "Basic" or "Digest") is not specified in a
case sensitive way. RFC2617 (http://tools.ietf.org/html/rfc2617) specifies
in section 1.2 the use of a "case-insensitive token to identify the
authentication scheme". Jetty, for example, uses "basic" as token.
Change-Id: I635a94eb0a741abcb3e68195da6913753bdbd889
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
The ObjectId (for a ref) can be easily reformatted into a temporary
byte[] and then passed off to write(byte[]), removing the duplicated
code that existed in both write methods.
Change-Id: I09740658e070d5f22682333a2e0d325fd1c4a6cb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We stopped handling URIs such as "example.com:/some/p ath", because
this was confused with the Windows absolute path syntax of "c:/path".
Support absolute style scp URIs again, but only when the host name
is more than 2 characters long.
Change-Id: I9ab049bc9aad2d8d42a78c7ab34fa317a28efc1a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This reverts commit 1e510ec20e.
Instead work around the warning by defining our constant by
constructing it through a StringBuilder.
Change-Id: If139509e769d649609c62eff359ebaea5dd286b2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Matthias Sohn <matthias.sohn@sap.com>
CC: Chris Aniszczyk <caniszczyk@gmail.com>
IndexDiff was extended to detect files which are both removed from the
index and untracked. Before this change these files were only added
to the removed collection.
Change-Id: I971d8261d2e8932039fce462b59c12e143f79f90
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The code analyzer can't know that passing a value known to be null is
not a problem. Hence better pass null explicitly instead of the
parameters being null.
Change-Id: I8db6f8014de6c00dd95974d60f61ecc66191e6d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There was a bug in ResolveMerger which is one reason for
bug 328841. If a merge was failing because of conflicts
deletions where not handled correctly. Files which have
to be deleted (because there was a non-conflicting deletion
coming in from THEIRS) are not deleted. In the
non-conflicting case we also forgot to delete the file but
in this case we explicitly checkout in the end these files
get deleted during that checkout.
This is fixed by handling incoming deletions explicitly.
Bug: 328841
Change-Id: I7f4c94ab54138e1b2f3fcdf34fb803d68e209ad0
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
The automatically generated commit message of a merge should have the
same structure as in C Git for consistency (as per git fmt-merge-msg).
Before this change:
merging refs/heads/a into refs/heads/master
After:
Merge branch 'a'
Plurals, "into" and joining by "," and "and" also work.
Change-Id: I9658ce2817adc90d2df1060e8ac508d7bd0571cb
This mirrors the getByte() API in ObjectId and allows the caller to
modify a single byte, which is useful when updating it as part of a
loop walking through 0x00..0xff inside of a range of objects.
Change-Id: I57fa8420011fe5ed5fc6bfeb26f87a02b3197dab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Processing git notes requires random access to part of the raw data
of each ObjectId... which isn't easy because ObjectIds are stored
with an internal representation of 5 ints. Expose random access
to the individual data bytes through new methods, avoiding the
need to convert first to a byte[20].
Change-Id: I99e64700b27fc0c95aa14ef8ad46a0e8832d4441
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of hiding this logic inside of DirCacheTree and the legacy
Tree type, pull it into a common place where we can reuse it by
creating tree records in a buffer that can be passed directly into
the ObjectInserter. This allows us to avoid some copying, as the
inserter can be given the internal buffer of the formatter.
Because we trust these two callers to feed us records in the proper
order, without '/' in the names, and without duplicate names in the
same tree, we don't do any validation inside of the formatter itself.
To protect themselves from making ordering errors, developers should
continue to use DirCache to process edits to source code trees.
Change-Id: Idf7f10e736d4a44ccdf8afe060535d7b0554a92f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
JGit merge algorithm behaved differently from C Git when
we had adjacent modifications. If line 9 was modified by
OURS and line 10 by theirs then C Git will return a
conflict while JGit was seeing this as independent
modifications. This change is not only there to achieve
compatibility, but there where also some really wrong
merge results produced by JGit in the area of adjacent
modifications.
Change-Id: I8d77cb59e82638214e45b3cf9ce3a1f1e9b35c70
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
When adding a new method near the end of the sequence we want to
show the full method inserted, and not tear the prior method due
to the common trailing curly brace being consumed as part of the
common end region of the sequences.
Bug: 328895
Change-Id: I233bc40445fb5452863f5fb082bc3097433a8da6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
HistogramDiff failed on cases where the initial element for the LCS
was actually very common (e.g. has 20 occurrences), and the first
element of the inserted region after the LCS was also common but
had fewer occurrences (e.g. 10), while the LCS also contained a
unique element (1 occurrence).
This happens often in Java source code. The initial element for
the LCS might be the empty line ("\n"), and the inserted but common
element might be "\t/**\n", with the LCS being a large span of
lines that contains unique method declarations. Even though "/**"
occurs less often than the empty line its not a better LCS if the
LCS we already have contains a unique element.
The logic in HistogramDiff would normally have worked fine, except I
tried to optimize scanning of B by making tryLongestCommonSequence
return the end of the region when there are matching elements
found in A. This allows us to skip over the current LCS region,
as it has already been examined, but caused us to fail to identify
an element that had a lower occurrence count within the region.
The solution used here is to trade space-for-time by keeping a
table of A positions to their occurrence counts. This allows the
matching logic to always use the smallest count for this region,
even if the smallest count doesn't appear on the initial element.
The new unit test testEdit_LcsContainsUnique() verifies this new
behavior works as expected.
Bug: 328895
Change-Id: Id170783b891f645b6a8cf6f133c6682b8de40aaf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Fixes the "Method ignores results of InputStream.read()" warning.
This is the only place where read() was used instead of readFully()
and the return value was not checked. So it was either an oversight
or should be documented. This change assumes it was an oversight.
Change-Id: I859404a7d80449c538a552427787f3e57d7c92b4
The value was accessed every time in the loop body with get(),
so use the more efficient entrySet().
Change-Id: I91d90cbd0b0d03ca4a3db986c58b8d80d80f40a4
This was already disabled in the Eclipse preferences for the project.
With this, Hudson should also ignore it.
Change-Id: I7a6b9a20451dc5ba9a61553248b5f4b6c6c7a78b
As described in Bug 328551 there was a bug that the merge algorithm
was not always reporting conflicts when the same line was deleted
and modified. This problem was introduced during commit
0c017188b4 when reported conflicts have
been checked for common pre- and suffixes.
This was fixed here by better determining whether after stripping
off common prefixes and suffixes from a conflicting region there
is still some conflicting part left.
I also added a unit test to test this situation.
Bug: 328551
Change-Id: Iec6c9055d00e5049938484a27ab98dda2577afc4
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
When creating a local branch based on another local branch, the
upstream configuration contains "." as origin and the source branch
as "merge". The PullCommand should support this by skipping the
fetch step altogether and use the base branch to merge with.
Change-Id: I260a1771aeeffca5b0161d1494fd63c672ecc2a6
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
AmbiguousObjectException contains an AbbreviatedObjectId and is
supposed to be serializable, so it should be serializable as well.
Change-Id: I8056e78aee20fdd3cb9600b52cd8ed988544293d
It's probably not possible that these numbers are negative in the
algorithm, but it's cleaner this way and gets rid of three more
FindBugs warnings.
Change-Id: Ifbce4e2c787fb9a7cd309c605e8d86211ef8a352
Don't permit transient worker threads to access the underlying output
stream of a ProgressMonitor, as they might get marked as the stream's
writer thread. Instead proxy update events from the workers back onto
the application's real work thread. This ensures that the stream only
sees a single thread, and its the thread that will remain alive for
the entire life cycle of the operation.
This fixes IOException("Write end dead") during local repository fetch
when threaded delta search is enabled. One of the transient delta
search threads became the designated writer for the pipe, and when it
terminated the reader end thought the writer was dead, even though the
main writer thread was still executing in PackWriter.
Bug: 326557
Change-Id: I01d1b20a3d7be1c0b480c7fb5c9773c161fe5c15
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
RefsDirectory fires a RefsChangedEvent when it detect that one
ref changed (new, modified, deleted). But there was a potential
of wrong events beeing fired leading to a endless loop in EGit.
Problem is that when calling getRefs(ALL) we don't want to report
additional refs and by that we remove the additional refs from
the list of "refs reported upwards last time". We fire an
RefsChangedEvent because we think that the special refs are not
there anymore.
I fixed this by removing eventing for the additional refs. Another
alternative would be to always scan also for additional refs and
put them in the list of refs. But getRefs(ALL) would then remove
the additional refs again. I didn't do that for performance reasons
and also because I am not sure whether we want evnting for
additional refs.
Change-Id: Icb9398b55a8c6bbf03e38f6670feb67754ce91e0
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
When checking out a tree, files that are identical to the file in
the current index and working directory don't need to be updated.
Change-Id: I9e025a53facd42410796eae821baaeff684a25c5
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
IndexDiff now allows to set an additional filter. This can be used
e.g. for restricting the tree walk to a given set of files.
Change-Id: I642de17e74b997fa0c5878c90631f6640ed70bdd
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
The RefDirectory class was not returning FETCH_HEAD and
MERGE_HEAD when trying to get all refs via getRefs(RefDatabase.ALL).
This fix adds constants for FETCH_HEAD and ORIG_HEAD and adds a
new getter getAdditionalRefs() to get these additional refs.
To be compatible with c git the getRefs(ALL) method will not return
FETCH_HEAD, MERGE_HEAD and ORIG_HEAD.
Change-Id: Ie114ca92e9d5e7d61d892f4413ade65acdc08c32
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
We need to use findbugs-maven-plugin:2.3.2-SNAPSHOT
since otherwise build fails with maven-3.0 [1], [2].
We should switch to the release version as soon
as this becomes available.
[1] http://www.sonatype.com/people/2010/10/maven-3-0-has-landed/
[2] http://jira.codehaus.org/browse/MFINDBUGS-122
Bug: 327799
Change-Id: I1c57f81cf6f0450e56411881488c4ee754e458e3
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If an object hasn't been accessed yet the pack list for a repository
may not have been scanned from disk. If an application (e.g. the dumb
transport servlet support code) asks for the pack list for an
ObjectDirectory, we should load it immediately.
Change-Id: I93d7b1bca422d905948e8e83b2afa83c8894a68b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If an ObjectInserter is created from a CachedObjectDirectory, we need
to ensure the cache is updated whenever a new loose object is actually
added to the loose objects directory, otherwise a future read from an
ObjectReader on the CachedObjectDirectory might not be able to open
the newly created object.
We mostly had the infrastructure in place to implement this due to the
injection of unpacked large deltas, but we didn't have a way to pass
the ObjectId from ObjectDirectoryInserter to CachedObjectDirectory,
because the inserter was using the underlying ObjectDirectory and not
the CachedObjectDirectory. Redirecting to CachedObjectDirectory
ensures the cache is updated.
Change-Id: I1f7bdfacc7ad77ebdb885f655e549cc570652225
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These messages may need to change depending on the current
thread's configured locale, and thus cannot be static.
Change-Id: I96751a63852ec9c4bf6c47edadcf8752700543df
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
There was a chance that we hit a NPE which doing a checkout
with DirCacheCheckout when there is no HEAD (e.g. initial
checkout). This is fixed here.
Change-Id: Ie3b8cae21dcd90ba8352823ea94a700f8ee9221a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Implemented the initial version of a cherry-pick command.
A correct error handling is missing (what happens if the
checkout fails, the cherry-pick leads to conflicts etc).
But straightforward cherry-picks works.
Change-Id: I235c0eb3a7a2d5bdfe40400f1deed06f29d746e1
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
These routines can be useful when debugging, because we can add an
expression to the Eclipse "Expressions" panel to show the text that
appears on a line. Gerrit Code Review also uses these in its own
subclass of RawText in order to format patch files, so pulling it up
to be part of core JGit may help other applications too.
Change-Id: I20a6b112e3403ecfc1c2715ae75dcecc1a85b167
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since the introduction of HashedSequence we no longer need to supply
the RawTextComparator at the time of constructing a RawText. Drop the
definition from the constructor, because it doesn't make sense as part
of our public API.
Change-Id: Iaab34611d60eee4a2036830142b089b2dae81842
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When an empty line was inserted at the beginning of the common end
part of a RawText the comparator incorrectly considered it to be
common, which meant the DiffAlgorithm would later not even have it be
part of the region it examines. This would cause JGit to skip a line
of insertion, which later confused Gerrit Code Review when it tried to
match up the pre and post RawText files for a difference that had this
type of insertion.
Define two new unit tests to check for this insertion of a blank line
condition and correct for it by removing the LF from the common region
when the condition is detected.
Change-Id: I2108570eb2929803b9a56f9fb9c400c758e7156b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The maven 2 build for jgit source bundle didn't create a correct
OSGi version string, instead of
org.eclipse.jgit.source_0.10.0.<timestamp>
the generated OSGi version was
org.eclipse.jgit.source_0.10.0.SNAPSHOT
This caused trouble when trying to install it from p2 repository.
Bug: 327616
Change-Id: Ic27c763ae9a4bcbb5bd6ed9562cd98bb4da3386b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It wrongly uses the full name of the branch to remove the
configuration entries but must use the shortened one.
Change-Id: Ie386a128a6c6beccc20bafd15c2e36254c5f560d
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
When creating a branch with CreateBranchCommand.call() without
specifying an explicit startPoint HEAD should be used as startPoint.
There was a bug leading to an NPE in such a case.
Change-Id: Ic0a5dc1f33a0987d66c09996c8012c45785500ff
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
There was a wrong javadoc comment telling that MergeCommand
only supports fast-forward merges. This has been fixed.
Change-Id: I7edea779a83528beee34a1753026288c384881ce
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
We wanted to wrap all LowLevel JGit excpetions into a
JGitInternalException so that users of this high-level interface
don't have to explicitly catch all of them. This
was forgotten on BranchCreateCommand.call() and I added
it.
Change-Id: Ie140e99574fb004137c66e80fb92eb6c6d0fa5e1
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
HistogramDiff outperforms it for any case where PatienceDiff needs to
fallback to another algorithm. Consequently it's not worth keeping
around, because we would always want a fallback enabled.
Change-Id: I39b99cb1db4b3be74a764dd3d68cd4c9ecd91481
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Its behavior is similar to PatienceDiff, and runs nearly as fast,
often beating the performance of MyersDiff.
Change-Id: I43c3faefa8109f1a68ef57522bec9cf27b5df252
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Large objects stored as deltas get unpacked by JGit into a loose
object, so they are cheaper to access later on. This unpacking was
broken because TeeInputStream copied the wrong length into the loose
object, sometimes copying too many bytes into the result. This
created a loose object that did not have the correct content, and
whose length did not match the length denoted in the object header.
Change-Id: I3ce1fd9f3dc5bd195249c7872b3bec49570424a2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When passing to a fallback algorithm, we can avoid creating a new copy
of the hash codes for each sequence by passing in the hashed sequences
directly. This makes it cheaper to switch from HistogramDiff down to
MyersDiff in a single pass.
Change-Id: Ibf2e81be57c083862eeb134279aed676653bf9b5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
There is a corner case where we get an EMPTY region during recursion,
but we didn't expect to receive that. Its harmless to ignore the
region since the region is empty and has no content, so do so rather
than throwing an exception
Change-Id: I50dcec81ecba763072bb739adfab5879fb48b23a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Certain inputs caused an infinite loop because the prior match data
couldn't be used as expected. Rather than incrementing the match
pointer before looking at an element, do it after, so the loop breaks
when we wrap around to the starting point.
Change-Id: Ieab28bb3485a914eeddc68aa38c256f255dd778c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The need for branching becomes more pressing with pull
support: we need to make sure the upstream configuration entries
are written correctly when creating and renaming branches
(and of course are cleaned up when deleting them).
This adds support for listing, adding, deleting and renaming
branches including the more common options.
Bug: 326938
Change-Id: I00bcc19476e835d6fd78fd188acde64946c1505c
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
In bug 323571 it is mentioned that if you call
'toURI().toURL().toString()' on a java.io.File you cannot pass
that string to jgit as an URIish. Problem is that the passed
URI looks like 'file:/C:/a/b.txt' and that we where expecting
double slashes after scheme':'. This fix adds support for this
single-slash file URLs.
Bug: 323571
Change-Id: I866a76a4fcd0c3b58e0d26a104fc4564e7ba5999
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This is the minimal implementation of a "Pull" command. It does not
have any parameters besides the generic progress monitor and timeout.
It works on the currently checked-out branch and assumes that the
configuration contains the keys "branch.<branch name>.remote" and
"branch.<branch name>.merge" to determine the remote configuration
for the fetch and the remote branch name for the merge.
Bug: 303404
Change-Id: I7fe09029996d0cfc09a7d8f097b5d6af1488fa93
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
There where quite some bugs regarding wrong URI parsing. In order
to solve them the parsing has to be refactored. We now have
specialized regexps for 'scheme://host/...', scp URIs and local
file names. Now we can detect problems while parsing 'git://host:/abc' which
was previously not possible.
Bug: 315571
Bug: 292897
Bug: 307017
Bug: 323571
Bug: 317388
Change-Id: If72576576ebb6b9d9dc8b7e51ddd87c9909e8b62
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The regular expression which should handle the
user/password part in an URI was potentially
processing too many chars. This led to problems
when user/pwd and port was specified
Change-Id: I87db02494c4b367283e1d00437b1c06d2c8fdd28
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The regular expressions used to parse URI's are constructed by
concatenating different segments to a big String. Introduce
String constants for these segements and document them.
Change-Id: If8b9dbaaf57ca333ac0b6c9610c3d3a515c540f9
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Some strings were not externalized. Also use them in HTTP tests to
ensure that they will also succeed when message bundles are
translated.
Change-Id: Id02717176557e7d57e676e1339cd89f2be88d330
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The strings used to construct the regex to parse
URIs are split differently. This makes it easier
to introduce meaningful String constants later on.
Change-Id: I9355fd42e57e0983204465c5d6fe5b6b93655074
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Projects like org.eclipse.mdt contain large XML files about 6 MiB
in size. So does the Android project platform/frameworks/base.
Doing a clone of either project with JGit takes forever to checkout
the files into the working directory, because delta decompression
tends to be very expensive as we need to constantly reposition the
base stream for each copy instruction. This can be made worse by
a very bad ordering of offsets, possibly due to an XML editor that
doesn't preserve the order of elements in the file very well.
Increasing the threshold to the same limit PackWriter uses when
doing delta compression (50 MiB) permits a default configured
JGit to decompress these XML file objects using the faster
random-access arrays, rather than re-seeking through an inflate
stream, significantly reducing checkout time after a clone.
Since this new limit may be dangerously close to the JVM maximum
heap size, every allocation attempt is now wrapped in a try/catch
so that JGit can degrade by switching to the large object stream
mode when the allocation is refused. It will run slower, but the
operation will still complete.
The large stream mode will run very well for big objects that aren't
delta compressed, and is acceptable for delta compressed objects that
are using only forward referencing copy instructions. Copies using
prior offsets are still going to be horrible, and there is nothing
we can do about it except increase core.streamFileThreshold.
We might in the future want to consider changing the way the delta
generators work in JGit and native C Git to avoid prior offsets once
an object reaches a certain size, even if that causes the delta
instruction stream to be slightly larger. Unfortunately native
C Git won't want to do that until its also able to stream objects
rather than malloc them as contiguous blocks.
Change-Id: Ief7a3896afce15073e80d3691bed90c6a3897307
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Natively support the HTTP basic and digest authentication methods
by setting the Authorization header without going through the JREs
java.net.Authenticator API. The Authenticator API is difficult to
work with in a multi-threaded server environment, where its using
a singleton for the entire JVM. Instead compute the Authorization
header from the URIish user and pass, if available.
Change-Id: Ibf83fea57cfb17964020d6aeb3363982be944f87
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
As findbugs pointed out, there was a small risk for creating multiple instances of
translation bundles. If that happens, drop the second instance.
Change-Id: I3aacda86251d511f6bbc2ed7481d561449ce3b6c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
HistogramDiff is an alternative implementation of patience diff,
performing a search over all matching locations and picking the
longest common subsequence that has the lowest occurrence count.
If there are unique common elements, its behavior is identical to
that of patience diff.
Actual performance on real-world source files usually beats
MyersDiff, sometimes by a factor of 3, especially for complex
comparators that ignore whitespace.
Change-Id: I1806cd708087e36d144fb824a0e5ab7cdd579d73
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Pass through the addAll request to our underlying ArrayList.
This way the underlying ArrayList grows no more than once during the
call, which may be important if the list was originally allocated
at the default size of 16, but 64 Edits are being added.
Change-Id: I31c3261e895766f82c3c832b251a09f6e37e8860
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
FetchCommand was missing the ability to set dry run and thin
preferences on the transport operation.
Change-Id: I0bef388a9b8f2e3a01ecc9e7782aaed7f9ac82ce
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Optional inCore parameter to Resolver/Strategy will
instruct it to perform all the operations in memory
and avoid modifying working folder even if there is one.
Change-Id: I5b873dead3682f79110f58d7806e43f50bcc5045
The hash code returned by RawTextComparator (or that is used
by the SimilarityIndex) play an important role in the speed of
any algorithm that is based upon them. The lower the number of
collisions produced by the hash function, the shorter the hash
chains within hash tables will be, and the less likely we are to
fall into O(N^2) runtime behaviors for algorithms like PatienceDiff.
Our prior hash function was absolutely horrid, so replace it with
the proper definition of the DJB hash that was originally published
by Professor Daniel J. Bernstein.
To support this assertion, below is a table listing the maximum
number of collisions that result when hashing the unique lines in
each source code file of 3 randomly chosen projects:
test_jgit: 931 files; 122 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 418
djb 5
sha1 6
string_hash31 11
test_linux26: 30198 files; 258 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 8675
djb 32
sha1 8
string_hash31 32
test_frameworks_base: 8381 files; 184 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 4615
djb 10
sha1 6
string_hash31 13
We can clearly see that prior_hash performed very poorly, resulting
in 8,675 collisions (elements in the same hash bucket) for at least
one file in the Linux kernel repository. This leads to some very
bad O(N) style insertion and lookup performance, even though the
hash table was sized to be the next power-of-2 larger than the
total number of unique lines in the file.
The djb hash we are replacing prior_hash with performs closer to
SHA-1 in terms of having very few collisions. This indicates it
provides a reasonably distributed output for this type of input,
despite being a much simpler algorithm (and therefore will be much
faster to execute).
The string_hash31 function is provided just to compare results with,
it is the algorithm commonly used by java.lang.String hashCode().
However, life isn't quite this simple.
djb produces a 32 bit hash code, but our hash tables are always
smaller than 2^32 buckets. Mashing the 32 bit code into an array
index used to be done by simply taking the lower bits of the hash
code by a bitwise and operator. This unfortuntely still produces
many collisions, e.g. 32 on the linux-2.6 repository files.
From [1] we can apply a final "cleanup" step to the hash code to
mix the bits together a little better, and give priority to the
higher order bits as they include data from more bytes of input:
test_jgit: 931 files; 122 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 418
djb 5
djb + cleanup 6
test_linux26: 30198 files; 258 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 8675
djb 32
djb + cleanup 7
test_frameworks_base: 8381 files; 184 avg. unique lines/file
Algorithm | Collisions
-------------+-----------
prior_hash 4615
djb 10
djb + cleanup 7
This is a massive improvement, as the number of collisions for
common inputs drops to acceptable levels, and we haven't really
made the hash functions any more complex than they were before.
[1] http://lkml.org/lkml/2009/10/27/404
Change-Id: Ia753b695de9526a157ddba265824240bd05dead1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As it turns out, every single diff algorithm we might try to
implement can benfit from using the SequenceComparator's native
concept of the simple reduceCommonStartEnd() step. For most inputs,
there can be a significant number of elements that can be removed
from the space the DiffAlgorithm needs to consider, which will
reduce the overall running time for the final solution.
Pool this logic inside of DiffAlgorithm itself as a default, but
permit a specific algorithm to override it when necessary.
Convert MyersDiff to use this reduction to reduce the space it
needs to search, making it perform slightly better on common inputs.
Change-Id: I14004d771117e4a4ab2a02cace8deaeda9814bc1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
PatienceDiff always uses a HashedSequence, which promises to provide
constant time access for hash codes during the equals method and
aborts fast if the hash codes don't match. Therefore we don't need
to cache the hash codes inside of the index, saving us memory.
Change-Id: I80bf1e95094b7670e6c0acc26546364a1012d60e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Most diff implementations really want to use cached hash codes for
elements, rather than element equality, as they need to perform many
compares and unique hash codes for elements can really speed that
process up.
To make it easier to define element hash functions, move the caching
of hash codes into a wrapper sequence type, so that individual
sequence types like RawText don't need to do this themselves. This
has a nice property of also allowing the sequence to no longer care
about the specific SequenceComparator that is going to be used, and
permits the caching to only examine the middle region that isn't
common to the two inputs.
Change-Id: If8623556da9419117b07c5073e8bce39de02570e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This is a faster exact match based form that tries to improve
performance for the common case of the header and trailer of
a text file not changing at all. After this fast path we use
the slower path based on the super class' using equals() to
allow for whitespace ignore modes to still work.
Some simple performance testing showed a major improvement over the
older implementation for a common edit we see in JGit. The test
compared blob 29a89bc and 372a978, which is the ObjectDirectory.java
file difference in commit 41dd9ed1c0.
The two text files are approximately 22 KiB in size.
DEFAULT old 203900 ns
DEFAULT new 100400 ns
This new version is 2x faster for the DEFAULT comparator, which does
not treat space specially. This is because we can now examine a
larger swath of text with fewer instructions per byte compared. The
older algorithm had to stop at each line break and recompute how to
examine the next line, while the new algorithm only stops when the
first difference is found.
WS_IGNORE_ALL old 298500 ns
WS_IGNORE_ALL new 63300 ns
Its 4.7x faster for the whitespace ignore comparator, as the common
header and footer do not have a whitespace difference. Avoiding the
special case handling for whitespace on each byte considered saves a
lot of time.
Since most edits to source code (and other text like files) appears in
the interior of the file, fast elimination of common header/footer
means faster diff throughput. In the less common case of an actual
header or footer edit, the common header/footer elimination is stopped
rather quickly either way, so there is very little downside to the
optimiation applied here.
Change-Id: I1d501b4c3ff80ed086b20bf12faf51ae62167db7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
DiffAlgorithm implementations may find it useful to construct an Edit
and use that to later subsequence the two base sequences, so define
two new utility methods a() and b() to construct the A and B ranges.
Once a subsequence has had Edits created for it the indexes are
within the space of the subsequence. These must be shifted back to
the original base sequence's indexes. Define toBase() as a utility
method to perform that shifting work in-place, so DiffAlgorithm
implementations have an efficient way to convert back to the caller's
original space.
Change-Id: I8d788e4d158b9f466fa9cb4a40865fb806376aee
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Adds API for performing git fetch operations.
Change-Id: Idd95664fd4e3bca03211e4ffda3e354849f92a35
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
If a thin pack has a large delta we need to be able to open
its cached copy from the loose object directory through the
CachedObjectDatabase handle. Unfortunately that did not support the
openObject2 method, which the LargePackedDeltaObject used directly
to bypass looking at the pack files.
Bug: 324868
Change-Id: I1d5886a6c3254c6dea2852d50b8614c31a93e615
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* stable-0.9:
Qualify post-0.9.3 builds
JGit 0.9.3
clone: Correct formatting of init message
Fix cloning of repositories with big objects
Qualify post-0.9.1 builds
JGit 0.9.1
Fix PlotCommitList to set lanes on child-less commits
When running IndexPack we use a CachedObjectDirectory, which
knows what objects are loose and tries to avoid stat(2) calls for
objects that do not exist in the repository, as stat(2) on Win32
is very slow.
However large delta objects found in a pack file are expanded into
a loose object, in order to avoid costly delta chain processing
when that object is used as a base for another delta.
If this expand occurs while working with the CachedObjectDirectory,
we need to update the cached directory data to include this new
object, otherwise it won't be available when we try to open it
during the object verify phase.
Bug: 324868
Change-Id: Idf0c76d4849d69aa415ead32e46a435622395d68
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When creating a new FileRepository, probe the capability of the
local filesystem and set core.filemode based on how it reacts.
We can't just rely on FS.supportsExecute() because a POSIX system
(which usually does support execute) might be storing the repository
on a partition that doesn't have execute support (e.g. plain FAT-32).
Creating a temporary file, setting both states, checking we get
the desired results will let us set the variable correctly on
all systems.
Change-Id: I551488ea8d352d2179c7b244f474d2e3d02567a2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In PlotCommitList.enter() commits are positioned on lanes for visual
presentation. This implementation was buggy: commits without
children (often the starting points for the RevWalk) are not positioned
on separate lanes.
The problem was that when handling commits with multiple children
(that's where branches fork out) it was not handled that some of the
children may not have been positioned on a lane yet. I fixed that and
added a number of tests which specifically test the layout of commits
on lanes.
Bug: 300282
Bug: 320263
Change-Id: I267b97ecccb5251cec54cec90207e075ab50503e
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A diff algorithm may find this type useful if it wants to delegate a
particular range of elements to another algorithm, without changing
the underlying sequence types.
Change-Id: I4544467781233e21ac8b35081304b2bad7db00f6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This makes it easier to parametrize DiffFormatter with a different
implementation, as we later plan to add PatienceDiff to JGit.
Change-Id: Id35ef478d5fa20fe10a1ba297f9436fd7adde9ce
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
git allows remotes to be relative paths, but the regex
validating urls wouldn't accept anything starting with "..".
Other functionality works fine with these paths.
Bug: 311300
Change-Id: Ib74de0450a1c602b22884e19d994ce2f52634c77
Instead of spooling large delta bases into temporary files and then
immediately deleting them afterwards, spool the large delta out to
a normal loose object. Later any requests for that large delta can
be answered by reading from the loose object, which is much easier
to stream efficiently for readers.
Since the object is now duplicated, once in the pack as a delta and
again as a loose object, any future prune-packed will automatically
delete the loose object variant, releasing the wasted disk space.
As prune-packed is run automatically during either repack or gc, and
gc --auto triggers automatically based on the number of loose objects,
we get automatic cache management for free. Large objects that were
unpacked will be periodically cleared out, and will simply be restored
later if they are needed again.
After a short offline discussion with Junio Hamano today, we may want
to propose a change to prune-packed to hold onto larger loose objects
which also exist in pack files as deltas, if the loose object was
recently accessed or modified in the last 2 days.
Change-Id: I3668a3967c807010f48cd69f994dcbaaf582337c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Recently created objects are usually what branches point to, and
are usually written out as loose objects. But due to the high cost
of asking the operating system if a file exists, these are the last
thing that ObjectDirectory examines when looking for an object by
its ObjectId.
Caching recently seen loose objects permits the opening code to
jump directly to the loose object, accelerating lookup for branch
heads that are accessed often.
To avoid exploding the cache its limited to approximately 2048
entries. When more ids are added, the table is simply cleared
and reset in size.
Change-Id: I18f483217412b102f754ffd496c87061d592e535
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This class is used only to cache the unpacked form of an object that
was used as a base for another object. The theory goes that if an
object is used as a delta base for A, it will probably also be a
delta base for B, C, D, E, etc. and therefore having an unpacked copy
of it on hand will make delta resolution for the others very fast.
However since objects are usually only accessed once, we don't want
to cache everything we unpack, just things that we are likely to
need again. The only things we need again are the delta bases.
Hence, its a delta base cache.
This gets us the class name UnpackedObjectCache back, so we can
use it to actually create a cache of unpacked object information.
Change-Id: I121f356cf4eca7b80126497264eac22bd5825a1d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The core.autocrlf variable can take on three values: false, true,
and input. Parsing it as a boolean is wrong, we instead need to
parse a tri-state enumeration.
Add support for parsing and setting enum values from Java from and
to the text based configuration file, and use that to handle the
autocrlf variable.
Bug: 301775
Change-Id: I81b9e33087a33d2ef2eac89ba93b9e83b7ecc223
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of making the sequence itself responsible for the equivalence
function, use an external function that is supplied by the caller.
This cleans up the code because we now say cmp.equals(a, ai, b, bi)
instead of a.equals(ai, b, bi).
This refactoring also removes the odd concept of creating different
types of sequences to have different behaviors for whitespace
ignoring. Instead DiffComparator now supports singleton functions
that apply a particular equivalence algorithm to a type of sequence.
Change-Id: I559f494d81cdc6f06bfb4208f60780c0ae251df9
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When checkReferencedIsReachable is set in ReceivePack we are trying
to prove that the push client is permitted to access an object that
it did not send to us, but that the received objects link to either
via a link inside of an object (e.g. commit parent pointer or tree
member) or by a delta base reference.
To do this check we are making a list of every potential delta base,
and then ensuring that every delta base used appears on this list.
If a delta base does not appear on this list, we abort with an error,
letting the client know we are missing a particular object.
Preventing spurious errors about missing delta base objects requires
us to use the exact same list of potential delta bases as the remote
push client used. This means we must use TOPO ordering, and we
need to enable BOUNDARY sorting so that ObjectWalk will correctly
include any trees found during the enumeration back to the common
merge base between the interesting and uninteresting heads.
To ensure JGit's own push client matches this same potential delta
base list, we need to undo 60aae90d4d ("Disable topological
sorting in PackWriter") and switch back to using the conventional
TOPO ordering for commits in a pack file. This ensures that our
own push client will use the same potential base object list as
checkReferencedIsReachable uses on the receiving side.
Change-Id: I14d0a326deb62a43f987b375cfe519711031e172
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since we are only checking the links between objects we don't need
to hold onto commit messages after their headers have been parsed
by the walker. Dropping them saves a bit of memory, which is always
good when accepting huge pack files.
Change-Id: I378920409b6acf04a35cdf24f81567b1ce030e36
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the copy instruction was larger than the input buffer given to us,
we copied the wrong part of the base stream during the next read().
This occurred on really big binary files where a copy instruction
of 64k wasn't unreasonable, but the caller's buffer was only 8192
bytes long. We copied the first 8192 bytes correctly, but then
reseeked the base stream back to the start of the copy region on
the second read of 8192 bytes. Instead of a sequence like ABCD
being read into the caller, we read AAAA.
Change-Id: I240a3f722a3eda1ce8ef5db93b380e3bceb1e201
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As ObjectStreams are supposed to be buffered, most implementors will
be wrapping their underlying stream inside of a BufferedInputStream
in order to satisfy this requirement. Because developers are by
nature lazy, they will use the default buffer size rather than
specify their own.
The OpenJDk JRE implementations use 8192 as the default buffer
size, and when the higher level reader uses the same buffer size
the buffers "stack" nicely by avoiding a copy to the internal
buffer array. As OpenJDK is a popular virtual machine, we should
try to benefit from this nice stacking property during copyTo().
Change-Id: I69d53f273b870b841ced2be2e9debdfd987d98f4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We can slightly optimize this method by removing some compares
based on knowledge of how the orderings have to work. This way
a getType() invocation requires at most 2 int compares for any
result, vs. the 6 required to find REPLACE before.
Change-Id: I62a04cc513a6d28c300d1c1496a8608d5df4efa6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Exposing isEmpty, getLengthA, getLengthB make it easier to examine
the state of an edit and work with it from higher level code.
The before and after cut routines make it easy to split an edit
that contains another edit, such as to decompose a REPLACE that
contains a common sequence within it.
Change-Id: Id63d6476a7a6b23acb7ab237d414a0a1a7200290
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We shouldn't escape non-special ASCII characters such as '@' or '~'.
These are valid in a path name on POSIX systems, and may appear as
part of a path in a GNU or Git style patch script. Escaping them
into octal just obfuscates the user's intent, with no gain.
When parsing an escaped octal sequence, we must parse no more
than 3 digits. That is, "\1002" is actually "@2", not the Unicode
character \u0202.
Change-Id: I3a849a0d318e69b654f03fd559f5d7f99dd63e5c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
QuotedString.GIT_PATH returns the input reference exactly if
the string does not require quoting, otherwise it returns a
copy that contains the quotes on either end, plus escapes in
the middle where necessary to meet conventions.
Testing the return against '"' + name + '"' is always false,
because GIT_PATH will never return it that way. The only way
we have quotes on either end is if there is an escape in the
middle, in which case the string isn't equal anyway.
Change-Id: I4d21d8e5c7da0d7df9792c01ce719548fa2df16b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we get an exception while indexing the incoming pack, its likely
a stream corruption. We already report an error to the client, but
we eat the stack trace, which makes debugging issues related to a
bug inside of JGit nearly impossible. Rethrow it under a new type
UnpackException, so embedding servers or applications can catch the
error and provide it to a human who might be able to forward such
traces onto a JGit developer for evaluation.
Change-Id: Icad41148bbc0c76f284c7033a195a6b51911beab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of getting the limit from CoreConfig, use the larger of the
reader's limit or 5 MiB, under the assumption that any annotated tag
or commit of interest should be under 5 MiB. But if a repository
was really insane and had bigger objects, the reader implementation
can set its streaming limit higher in order to allow RevWalk to
still process it.
Change-Id: If2c15235daa3e2d1f7167e781aa83fedb5af9a30
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
IDEs like Eclipse offer up the settings in WindowCacheConfig to the
user as a global set of options that are configured for the entire
JVM process, not per-repository, as the cache is shared across the
entire JVM. The limit on how much we are willing to allocate for
an object buffer is similar to the limit on how much we can use for
data caches, allocating that much space impacts the entire JVM and
not just a single repository, so it should be a global limit.
Change-Id: I22eafb3e223bf8dea57ece82cd5df8bfe5badebc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the iterators passed into a diff formatter are working tree
iterators, we should enable ignoring files that are ignored, as
well as actually pull up the current content from the working tree
rather than getting it from the repository.
Because we abstract away the working directory access logic,
we can now actually support rename detection between the working
directory and the local repository when using a DiffFormatter.
This means its possible for an application to show an unstaged
delete-add pair as a rename if the add path is not ignored.
(Because the ignored file wouldn't show up in our difference output.)
Even more interesting is we can now do rename detection between any
two working trees, if both input iterators are WorkingTreeIterators.
Unfortunately we don't (yet) optimize for comparing the working
tree with the index involved so we can take advantage of cached
stat data to rule out non-dirty paths.
Change-Id: I4c0598afe48d8f99257266bf447a0ecd23ca7f5e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When comparing a DirCache and a WorkingTree using ANY_DIFF we
sometimes didn't recursive into a subtree of both sides gave us
zeroId() back for the identity of a subtree. This happens when the
DirCache doesn't have a valid cache tree for the subtree, as then
it uses zeroId() for the ObjectId of the subtree, which then appears
to be equal to the zeroId() of the WorkingTreeIterator's subtree.
We work around this by adding a hasId() method that returns true
only if this iterator has a valid ObjectId. The idEquals method
on TreeWalk than only performs a compare between two iterators if
both iterators have a valid id.
Change-Id: I695f7fafbeb452e8c0703a05c02921fae0822d3f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Applications just want a quick way to configure our diff
implementation, and then just want to use it without a lot of fuss.
Move all of the rename detection logic and path following logic
out of our pgm package and into DiffFormatter itself, making it
much easier for a GUI to take advantage of the features without
duplicating a lot of code.
Change-Id: I4b54e987bb6dc804fb270cbc495fe4cae26c7b0e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
canResetHead now returns true.
Resetting mixed / hard works in EGit in merging state.
Change-Id: I1512145bbd831bb9734528ce8b71b1701e3e6aa9
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Sorting the array can be useful when its being used as a map of pairs
that are appended into the array and then later merge-joined against
another array of similar semantics.
Change-Id: I2e346ef5c99ed1347ec0345b44cda0bc29d03e90
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We know exactly how many lines we need by the time we compute our
per-line hashes, as we have already built the lines IntList to give
us the starting position of each line in the buffer. Using that
we can properly size the array, and don't need the dynamic growing
feature of IntList. So drop the indirection and just use a fixed
size array.
Change-Id: I5c8c592514692a8abff51e5928aedcf71e100365
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Create a new 'org.eclipse.jgit.api.errors' package to contain
exceptions related to using the Git porcelain API.
Change-Id: Iac1781bd74fbd520dffac9d347616c3334994470
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Today while debugging some TreeWalk related code I noticed this
filter did not have a toString(), making it harder to see what the
filter graph was at a glance in the debugger. Add a toString()
for debugging to match other TreeFilters, and clean up the Javadoc
slightly so its a bit more clear about the purpose of the filter.
While we are mucking about with some of this code, simplify
the logic of include so its shorter and thus faster to read.
The pattern now more closely matches that of SkipWorkTreeFilter.
Change-Id: Iad433a1fa6b395dc1acb455aca268b9ce2f1d41b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This allows callers to force the iterator back to its starting point,
so it can be traversed again. The default way to do this is to use
back(1) until first() is true, but this isn't very efficient for any
iterator. All current implementations have better ways to implement
reset without needing to seek backwards.
Change-Id: Ia26e6c852fdac8a0e9c80ac72c8cca9d897463f4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we are asked to create a difference between two files the caller
really wants to see that output. Instead of punting because a file
is too big to process, consider it to be binary. This reduces the
accuracy of our output display, but makes it a lot more likely that
the formatter can still generate something semi-useful.
We set our default binary threshold to 50 MiB, which is the same
threshold that PackWriter uses before punting and deciding a file
is too big to delta compress. Anything under this size we try to
load and process, anything over that size (or that won't allocate
in the heap) gets tagged as binary.
Change-Id: I69553c9ef96db7f2058c6210657f1181ce882335
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When adding or deleting a file, we shouldn't ever prefix /dev/null
with the a/ or b/ prefixes. Doing so is a mistake and confuses a
patch parser which handles /dev/null magically, while a/dev/null is
a file called null in the dev directory of the project.
Also when adding or deleting the "diff --git" line has the "real"
path on both sides, so we should see the following when adding the
file called foo:
diff --git a/foo b/foo
--- /dev/null
+++ b/foo
The --- and +++ lines do not appear in a pure rename or copy delta,
C Git diff seems to omit these, so we now omit them as well. We also
omit the index line when the ObjectIds are exactly equal.
Change-Id: Ic46892dea935ee8bdee29088aab96307d7ec6d3d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of trying to stream out the header, we can drop a redundant
code path by formatting the header into a temporary buffer and then
streaming out the actual line differences later.
Its a small amount of unnecessary work to buffer the file header,
but these are typically very tiny so the cost to format and reparse
is relatively low.
Change-Id: Id14a527a74ee0bd7e07f46fdec760c22b02d5bdf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Other fields in this class are initialized in their declaration, make
the code consistent with itself and use only one style.
Change-Id: I49a007e97ba52faa6b89f7e4b1eec85dccac0fa4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This class does a lot more than just reflow a patch script, it now is
the primary means of creating a diff output.
Change-Id: I74467c9a53dc270ef8c84e7c75f388414ec8ba8f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
To give the user more control on which file system abstraction
should be used on Windows, FS.detect() may be configured
to assume a Cygwin installation or nor.
Currently, if a branch is created that has special chars ('#' in the bug),
Config will surround the subsection name with double quotes during
it's toText method which will result in an invalid file after saving the
Config.
Bug: 318249
Change-Id: I0a642f52def42d936869e4aaaeb6999567901001
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Extended flags are processed and available via DirCacheEntry's
new isSkipWorkTree() and isIntentToAdd() methods. "resolve-undo"
information is completely ignored since its an optional extension.
Change-Id: Ie6e9c6784c9f265ca3c013c6dc0e6bd29d3b7233
The merge algorithm was reporting conflicts which where to big.
Example: The common base was "ABC", the "ours" version contained
"AB1C" (the addition of "1" after pos 2) and the "theirs" version also
contained "AB1C". We have two potentially conflicting edits in the
same region which happen to bring in exactly the same content. This
should not be a conflict - but was previously reported as
"AB<<<1===1>>>C".
This is fixed by checking every conflicting chunk whether the
conflicting regions have a common prefix or suffix and by removing
this regions from the conflict.
Change-Id: I4dc169b8ef7a66ec6b307e9a956feef906c9e15e
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
the merge command should use by default the "resolve" merge strategy.
Change-Id: I6c6973a3397cca12bd8a6bd950d04b1766a08b4c
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
This adds the first merge strategy to JGit which does real
content-merges if necessary. The new merge strategy "resolve" takes as
input three commits: a common base, ours and theirs. It will simply takeover
changes on files which are only touched in ours or theirs. For files
touched in ours and theirs it will try to merge the two contents
knowing taking into account the specified common base.
Rename detection has not been introduced for now.
Change-Id: I49a5ebcdcf4f540f606092c0f1dc66c965dc66ba
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Use 3 different types of LargeObjectException for the 3 major ways
that we can fail to load an object. For each of these use a unique
string translation which describes the root cause better than just
the ObjectId.name() does.
Change-Id: I810c98d5691b74af9fc6cbd46fc9879e35a7bdca
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Currently our algorithm requires that we have the delta base as
a contiguous byte array... but getCachedBytes() might not work
if the object is considered to be large by its underlying loader.
Use the limited form to obtain the object as a byte array instead.
Change-Id: I33f12a8811cb6a4a67396174733f209db8119b42
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This string is part of the network protocol, and isn't meant to
be translated into another language. Clients actually scan for
the string "unpack error " off the wire and react magically to
this information. If it were translated, they would instead have
a protocol exception, which isn't very useful when there is already
an error occurring.
Change-Id: Ia5dc8d36ba65ad2552f683bb637e80b77a7d92f0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This reverts commit db4c516f67 since
it breaks compatibility with Eclipse 3.5 which can no longer import
the projects
Bug: 323390
Change-Id: I3cc91364a6747cfcb4c611a9be5258f81562f726
Large delta streams are unpacked incrementally, but because a delta
can seek to a random position in the base to perform a copy we may
need to inflate the base repeatedly just to complete one delta.
So work around it by copying the base to a temporary file, and then
we can read from that temporary file using random seeks instead.
Its far more efficient because we now only need to inflate the
base once.
This is still really ugly because we have to dump to a temporary
file, but at least the code can successfully process a large
file without throwing OutOfMemoryError. If speed is an
issue, the user will need to increase the JVM heap and ensure
core.streamFileThreshold is set to a higher value, so we don't use
this code path as often.
Unfortunately we lose the "optimization" of skipping over portions
of a delta base that we don't actually need in the final result.
This is going to cause us to inflate and write to disk useless
regions that were deleted and do not appear in the final result.
We could later improve on our code by trying to flatten delta
instruction streams before we touch the bottom base object, and
then only store the portions of the base we really need for the
final result and that appear out-of-order. Since that is some
pretty complex code I'm punting on it for now and just doing this
simple whole-object buffering.
Because the process umask might be permitting other users to read
files we create, we put the temporary buffers into $GIT_DIR/objects.
We can reasonably assume that if a reader can read our temporary
buffer file in that directory, they can also read the base pack
file we are pulling it from and therefore its not a security breach
to expose the inflated content in a file. This requires a reader
to have write access to the repository, but only if the file is
really big. I'd rather err on the side of caution here and refuse
to read a very big file into /tmp than to possibly expose a secured
content because the Java 5 JVM won't let us create a protected
temporary file that only the current user can access.
Change-Id: I66fb80b08cbcaf0f65f2db0462c546a495a160dd
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A tag command is added to the Git porcelain API. Tests were
also added to stress test the tag command.
Change-Id: Iab282a918eb51b0e9c55f628a3396ff01c9eb9eb
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Implementation of a checkout (or 'git read-tree') operation which
works together with DirCache. This implementation does similar things
as WorkDirCheckout which main problem is that it works with deprecated
GitIndex. Since GitIndex doesn't support multiple stages of a file
which is required in merge situations this new implementation is
required to enable merge support.
Change-Id: I13f0f23ad60d98e5168118a7e7e7308e066ecf9c
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
The Merger was was only exposing the merge base as an
AbstractTreeIterator. Since we need the merge base as
RevCommit to generate the merge result I expose it here.
Change-Id: Ibe846370a35ac9bdb0c97ce2e36b2287577fbcad
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Updates the project level settings to run the formatter
on save on only on the edited lines.
Change-Id: I26dd69d0c95e6d73f9fdf7031f3c1dbf3becbb79
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
PersonIdent should be parsable for an invalid commit which
contains multiple authors, like "A <a@a.org>, B <b@b.org>".
PersonIdent(String) constructor now delegates to
RawParseUtils.parsePersonIdent().
Change-Id: Ie9798d36d9ecfcc0094ca795f5a44b003136eaf7
Applying deltas in the large streaming mode is horrifically slow.
Trying to pack icu4c is impossible because a single 11 MiB file
sits on top of a 15 MiB file though a 10 deep delta chain, which
results in this very slow inflate process.
Upping the default limit to 15 MiB lets us process this large in a
reasonable time, but its still sufficiently low enough to prevent
exploding the heap of a very large process like Eclipse or Gerrit
Code Review.
We have to revisit the streaming delta application process and do
something much smarter, like flatten the delta chain before we apply
it to the base. But even that is ugly, I've seen a 155 MiB delta
sitting on top of a 450 MiB file to produce a 300 MiB result object.
If the chain is deep, we may have trouble flatting it down.
Change-Id: If5a0dcbf9d14ea683d75546f104b09bb8cd8fdbb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We miscomputed the CRC32 checksum for a REF_DELTA type of object, by
not including the full 20 byte ObjectId of the delta base in the CRC
code we use when the delta is too large to go through our two faster
small reuse code paths. This resulted in a corruption error during
packing, where the PackFile erroneously suspected the data was wrong
on the local filesystem and aborted writing, because the CRC didn't
match what we had read from the index.
Change-Id: I7d12cdaeaf2c83ddc11223ce0108d9bd6886e025
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We didn't fully cover what we support and what we don't. It was
also a bit hard to follow the syntaxes supported. Clean that up
by documenting it.
Change-Id: I7b96fa6cbefcc2364a51f336712ad361ae42df2d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We can now resolve expressions that reference a path within a
commit, designating a specific revision of a specific tree or
file in the project.
Change-Id: Ie6a8be629d264d72209db894bd680c5900035cc0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We now match on the -gABBREV style output created by git describe
when its describing a non-tagged commit, and resolve that back to
the full ObjectId using the abbreviation resolution feature that
we already support.
Change-Id: Ib3033f9483d9e1c66c8bb721ff48d4485bcdaef1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Its wrong to return null if we are resolving an abbreviation and we
have proven it matches more than one object. We know how to resolve
it if we had more nybbles, as there are two or more objects with the
same prefix. Declare that to the caller quite clearly by giving them
an AmbiguousObjectException.
Change-Id: I01bb48e587e6d001b93da8575c2c81af3eda5a32
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we are given a DiffEntry header that already has abbreviated
ObjectIds on it, we may still be able to resolve those locally and
output the difference. Try to do that through the new resolve API
on ObjectReader.
Change-Id: I0766aa5444b7b8fff73620290f8c9f54adc0be96
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Parsing is rewritten to use the size limited form of getCachedBytes,
thus freeing the revwalk infrastructure from needing to care about
a large object vs. a small object when it gets an ObjectLoader.
Right now we hardcode our upper bound for a commit or annotated
tag to be 15 MiB. I don't know of any that is more than 1 MiB in
the wild, so going 15x that should give us some reasonable headroom.
Change-Id: If296c211d8b257d76e44908504e71dd9ba70ffa8
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Rather than duplicating this block everywhere, reuse the limited size
form of getCachedBytes to acquire the content of an object.
Change-Id: I2e26a823e6fd0964d8f8dbfaa0fc2e8834c179c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Some algorithms are coded in a way that requires us to provide them
the entire object contents as a contiguous byte array. The parsers
in RevCommit and RevTag, or our RawText objects are really good
examples of these.
Instead of duplicating this logic everywhere, lets put it into the
base ObjectLoader type. That way the caller only needs to give us
their upper size bound, and we'll do the rest of the heavy work to
figure out if the object still fits within that bound, and get them
an array that has the complete contents.
Change-Id: Id95a7f79d2b97e39f6949370ccca2f2c9cfb1a0f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
A chunk of code that throws LargeObjectException may or may not have
the specific ObjectId on hand when its thrown. If it does, we want
to cache it in the exception, and put that in the message. If it is
missing we want to be able to set it later from a higher level stack
frame that does have the object handy.
Change-Id: Ife25546158868bdfa886037e4493ef8235ebe4b9
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
If the loader's stream is broken and returns to us more content
than it originally declared as the size of the object, don't
copy that onto the output stream. Instead throw EOFException
and abort fast. This way we don't follow an infinite stream,
but instead will at least stop when the size was reached.
Change-Id: I7ec0c470c875f03b1f12a74a9b4d2f6e73b659bb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the stream is a delta decompression stream, getting the size
can be expensive. Its cheaper to get it from the stream itself
rather than from the object loader.
Change-Id: Ia7f0af98681f6d56ea419a48c6fa8eea09274b28
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we can't resolve an abbreviation, it might be because there is
a new pack file we haven't picked up yet. Try scanning the packs
again and recheck each pack if there were differences from the last
scan we did.
Because of this, we don't have to open a pack during the test where
we generate a pack on the fly. We'll miss on the first loop during
which the PackList is the NO_PACKS magic initialization constant,
and pick up the newly created index during this retry logic.
Change-Id: I7b97efb29a695ee60c90818be380f7ea23ad13a3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ObjectReader implementations are now responsible for creating the
unique abbreviation of an ObjectId, or for resolving an abbreviation
back to its full form. In this latter case the reader can offer up
multiple candidates to the caller, who may be able to disambiguate
them based on context.
Repository.resolve() doesn't take multiple candidates into account
right now, but it could in the future by looking for a remaining
^0 or ^{commit} suffix and take an expansion if there is only one
commit that matches the input abbreviation. It could also use
the distance from an annotated tag to resolve "tag-NNN-gcommit"
style strings that are often output by `git describe`.
Change-Id: Icd3250adc8177ae05278b858933afdca0cbbdb56
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ObjectWriter is a deprecated API that people shouldn't be using.
So get rid of it in favor of the ObjectInserter API.
Change-Id: I6218bcb26b6b9ffb64e3e470dba5dca2e0a62fd4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This makes it easier for abstract tools like AddCommand to open the
file from the working tree, without knowing internal details about
how the tree is managed.
Change-Id: Ie64a552f07895d67506fbffb3ecf1c1be8a7b407
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Applications should favor the long style interface, especially when
their source input is a long type, e.g. coming from java.io.File.
This way when the index format is later changed to support a
larger file size than 2 GiB we can handle it by just changing the
entry code, and not need to fix a lot of applications.
Change-Id: I332563caeb110014e2d544dc33050ce67ae9e897
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These objects should be responsible for their own formatting,
rather than delegating it to some obtuse type called ObjectInserter.
While we are at it, simplify the way we insert these into a database.
Passing in the type and calling format in application code turned
out to be a huge mistake in terms of ease-of-use of the insert API.
Change-Id: Id5bb95ee56aa2a002243e9b7853b84ec8df1d7bf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since these types no longer support reading, calling them a Builder
is a better description of what they do. They help the caller to
build a commit or a tag object.
Change-Id: I53cae5a800a66ea1721b0fe5e702599df31da05d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since we stopped supporting these types for reading, but their
name is a natural candidate for someone to try and use in code,
explain where they should be looking instead.
Change-Id: I091a1b0ef71b842016020f938ba3161431aab9c9
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
There where 3 cases where a JGitInternalExcption was created
without specifying the root cause. This has been fixed.
Change-Id: I2ee08d04732371cd9e30874b1437b61217770b6a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
WorkingTreeIterator now optionally performs CRLF to LF conversion for
text files. A basic framework is left in place to support enabling
(or disabling) this feature based on gitattributes, and also to
support the more generic smudge/clean filter system. As there is
no gitattribute support yet in JGit this is left unimplemented,
but the mightNeedCleaning(), isBinary() and filterClean() methods
will provide reasonable places to plug that into in the future.
[sp: All bugs inside of WorkingTreeIterator are my fault, I wrote
most of it while cherry-picking this patch and building it on
top of Marc's original work.]
CQ: 4419
Bug: 301775
Change-Id: I0ca35cfbfe3f503729cbfc1d5034ad4abcd1097e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These classes need to be visible if an application wants to define
its own native pack based protocol embedded within another layer,
much like we already support for smart HTTP.
Change-Id: I7e2ac3ad01d15b94d340128a395fe0b2f560ff35
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The reuse system used by an object database may be able to benefit
from knowing what objects are coming next, and even improve data
throughput by delaying (or moving up) objects that are stored near
each other in the source database.
Pushing the iteration down into the reuse code makes it possible
for a smarter implementation to aggregate reuse. But for the
standard pack file format on disk we don't bother, its quite
efficient already.
Change-Id: I64f0048ca7071a8b44950d6c2a5dfbca3be6bba6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some instances may benefit from having access to memory efficient
storage for some small values, like single flag bits. Give up a
portion of our delta depth field to make 4 bits available to any
subclass that wants it.
This still gives us room for delta chains of 1,048,576 objects,
and that is just insane. Unpacking 1 million objects to get to
something is longer than most users are willing to wait for data
from Git.
Change-Id: If17ea598dc0ddbde63d69a6fcec0668106569125
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
An ObjectReader implementation may be very slow for a single object,
but yet support bulk queries efficiently by batching multiple small
requests into a single larger request. This easily happens when the
reader is built on top of a database that is stored on another host,
as the network round-trip time starts to dominate the operation cost.
RevWalk, ObjectWalk, UploadPack and PackWriter are the first major
users of this new bulk interface, with the goal being to support an
efficient way to pack a repository for a fetch/clone client when the
source repository is stored in a high-latency storage system.
Processing the want/have lists is now done in bulk, to remove
the high costs associated with common ancestor negotiation.
PackWriter already performs object reuse selection in bulk, but it
now can also do the object size lookup and object counting phases
with higher efficiency. Actual object reuse, deltification, and
final output are still doing sequential lookups, making them a bit
more expensive to perform.
Change-Id: I4c966f84917482598012074c370b9831451404ee
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
By giving the reader information about the roots of a revision
traversal, some readers may be able to prefetch information from
their backing store using background threads in order to reduce
data access latency. However this isn't typically necessary so
the default reader implementation doesn't react to the advice.
Change-Id: I72c6cbd05cff7d8506826015f50d9f57d5cda77e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ObjectReader implementations may wish to use multiple threads in
order to evaluate object reuse faster. Let the reader make that
decision by passing the iteration down into the reader.
Because the work is pushed into the reader, it may need to locate a
given ObjectToPack given its ObjectId. This can easily occur if the
reader has sent a list of ObjectIds to the object database and gets
back information keyed only by ObjectId, without the ObjectToPack
handle. Expose lookup using the PackWriter's own internal map,
so the reader doesn't need to build a redundant copy to track the
assocation of ObjectId back to ObjectToPack.
Change-Id: I0c536405a55034881fb5db92a2d2a99534faed34
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When the output stream is deeply buffered (e.g. 1 MiB or more in
an HTTP servlet on some containers) trying to kick out the header
earlier will prevent the client from stalling hard while the first
1 MiB is received and it can process the pack header. Forcing a
flush here lets the client see the header and start its progress
monitor for "Receiving objects: (1/N)" so the user knows there
is still activity occurring, even though the buffering may cause
there to be some lag as the buffer fills up on the sending side.
Change-Id: I3edf39e8f703fe87a738dc236d426b194db85e3a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Callers catching a MissingObjectException may need programmatic
access to the ObjectId that wasn't available in the repository.
Change-Id: I2be0380251ebe7e4921fa74e246724e48ad88b0e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Storage implementations or application code using an ObjectReader
may want to access this constant without being inside of a subclass
of the reader.
Change-Id: I6c871a03d5846b9bb899de4d14a265e8b204d8e0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Storage implementations may find this useful when implementing the
ObjectReuseAsIs interface on their ObjectReader. Expose it so we
don't force them to create a redundant copy of the information.
Change-Id: I802ec8113c00884fccde5d0e92b9849716316f62
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This permits formatting in hex into an existing byte array
supplied by the caller, and mirrors our copyRawTo method
with the same parameter signature.
Change-Id: Ia078d83e338b09b903bfd2d04284e5283f885a19
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Callers might have a canonical tag encoding on hand that they
wish to convert into a clean structure for presentation purposes,
and the object may not be available in a repository. (E.g. maybe
its a "draft" tag being written in an editor.)
Change-Id: I387a462afb70754aa7ee20891e6c0262438fdf32
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Callers might have a canonical commit encoding on hand that they
wish to convert into a clean structure for presentation purposes,
and the object may not be available in a repository. (E.g. maybe
its a "draft" commit being written in an editor.)
Change-Id: I21759cff337cbbb34dbdde91aec5aa4448a1ef37
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The Tag class now only supports the creation of an annotated tag
object. To read an annotated tag, applictions should use RevTag.
This permits us to have exactly one implementation, and RevTag's
is faster and more bug-free.
Change-Id: Ib573f7e15f36855112815269385c21dea532e2cf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The Commit class now only supports the creation of a commit object.
To read a commit, applictions should use RevCommit. This permits
us to have exactly one implementation, and RevCommit's is faster
and more bug-free.
Change-Id: Ib573f7e15f36855112815269385c21dea532e2cf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Git doesn't store millisecond accuracy in person identity lines,
so a line that we create in Java and round-trip through a Git object
wouldn't compare as being equal. Truncate to seconds when comparing
values to ensure the same identity is equal.
Change-Id: Ie4ebde64061f52c612714e89ad34de8ac2694b07
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we need the canonical form of a commit or a tag in order to
parse it into our RevCommit or RevTag fields, we really need it as a
single contiguous byte array. However the ObjectDatabase may choose
to give us a large loader. In general commits or tags are always
under the several MiB limit, so even if the loader calls it "large"
we should still be able to afford the JVM heap memory required to
get a single byte array. Coerce even large loaders into a single
byte array anyway.
Change-Id: I04efbaa7b31c5f4b0a68fc074821930b1132cfcf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since equals() is now final and does not permit being overridden,
we should do the same thing with compareTo() to prevent different
subclasses from having different ordering behaviors. This could
lead to the same mess that we had with different equals() behaviors.
Change-Id: I35a849b6efccee5fe74cc5788a3566a1516004b7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since equals() is now final and does not permit being overridden,
we should do the same thing with hashCode() to prevent different
subclasses from having different hashing behaviors. This could
lead to the same mess that we had with different equals() behaviors.
Change-Id: I35a849b6efccee5fe74cc5788a3566a1516004b7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When RevObject overrode equals() to provide only reference equality
we used to need to convert a RevObject into an ObjectId by copy()
just to use standard Java tools like JUnit assertEquals(), or to
use contains() or get() on standard java.util collection types.
Now that we have removed this override and made ObjectId's equals()
final (preventing any of this mess in the future), some copy()
calls are unnecessary. Anytime the value is being used as an input
to a lookup routine, or to an equals, we can avoid the copy().
However we still want to use copy() anytime we are given an ObjectId
that may exist long-term, where we don't want the high cost of the
additional storage from a RevCommit extension. So we can't remove
all uses of copy(), just some of them.
Change-Id: Ief275dace435c0ddfa362ac8e5d93558bc7e9fc3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Configuration change events were not being triggered, now they are
forwarded from the FileConfig up to the Repository's listeners.
Change-Id: Ida94a59f5a2b7fa8ae0126e33c13343275483ee5
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The MergeResult class is enhanced to report more data about a
three-way merge. Information about conflicts and the base, ours,
theirs commits can be retrived.
Change-Id: Iaaf41a1f4002b8fe3ddfa62dc73c787f363460c2
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
We should be more lenient when tagging without an
tagger or message. Currently, we will throw an NPE
which is incorrect behavior.
Change-Id: I04e30ce25a9432e4ca56c3f29658ecb24fb18d24
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
There was a duplicated getter and setter for tagger in Tag.
There's no needed to have two getters and setters that represent
the same things. The appropriate tests were updated also.
Change-Id: If46dc00c4c0f31ea4234c6d3bda3c03e6ebbafac
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Added a utility method to set the reset an index to match exactly
some content in the filesystem. This can be used by tests to prepare
commits in the working-tree and set the index in one shot.
[sp: Cleaned up formatting, added getEntryFile(), released inserter.]
Change-Id: If38b1f7cacaaf769f51b14541c5da0c1e24568a5
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Such events make no sense, it has never been visible to this
process so no client can have a stale value of the ref.
Change-Id: Iea3a5035b0a1410b80b09cf53387b22b78b18018
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Add a convenience API in FileRepository to pass in a String that
points to the GIT_DIR location. This is converted to a File and
sent through the usual constructor.
Change-Id: I588388f37e89b8c690020f110a1bc59f46170c40
Exclude ignored files from IndexDiff tree walk.
This makes EGit commit much faster.
Change-Id: I398499510c22c37667b7612db32eac3b31d325f0
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
This can result in an infinitely hanging IDE.
Change-Id: I669bc8d220a07011a42edf79de31825305ff3763
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
If the exact rename matrix for a particular ObjectId isn't square we
crashed with an ArrayIndexOutOfBoundsException because the matrix
entries were encoded backwards. The encode function accepts the
source (aka deleted) index first, not second. Add a unit test to
cover this non-square case to ensure we don't have this regression
in the future.
Change-Id: I5b005e5093e1f00de2e3ec104e27ab6820203566
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* changes:
Enable configuration of non-standard pack settings
Pass PackConfig down to PackWriter when packing
Simplify UploadPack use of options during writing
Move PackWriter configuration to PackConfig
Allow PackWriter callers to manage the thread pool
It is useful to be able to replace an existing Change-Id
in the message, for example if the user decides not to
amend the previous commit.
Bug: 321188
Change-Id: I594e7f9efd0c57d794d2bd26d55ec45f4e6a47fd
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
TreeWalk calls this value "path", while "name" is the stuff after the
last slash. FileHeader should do the same thing to be consistent.
Rename getOldName to getOldPath and getNewName to getNewPath.
Bug: 318526
Change-Id: Ib2e372ad4426402d37939b48d8f233154cc637da
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A bug in rename detection would cause file scores to be wrong. The
bug was due to the way rename detection would judge the similarity
between files. If file A has three lines containing 'foo', and file
B has 5 lines containing 'foo', the rename detection phase should
record that A and B have three lines in common (the minimum of the
number of times that line appears in both files). Instead, it would
choose the the number of times the line appeared in the destination
file, in this case file B. I fixed the bug by having the
SimilarityIndex instead choose the minimum number, as it should. I
also added a test case to verify that the bug had been fixed.
Change-Id: Ic75272a2d6e512a361f88eec91e1b8a7c2298d6b
IndexDiff was re-implemented and now uses TreeWalk instead
of GitIndex. Additionally, gitignore support and retrieval of
untracked files was added.
Change-Id: Ie6a8e04833c61d44c668c906b161202b200bb509
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
When the add parameter is set all modified and deleted files
are staged prior to commit.
Change-Id: Id23bc25730fcdd151386cd495a7cdc0935cbc00b
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
This change is mainly done for a subsequent commit
which will introduce the "all" parameter to the Commit
command.
Bug: 318439
Change-Id: I85a8a76097d0197ef689a289288ba82addb92fc9
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
The CommitCommand should not use java.io to delete MERGE_HEAD and MERGE_MSG
files since Repository already has utility methods for that.
Change-Id: If66a419349b95510e5b5c2237a91f06c1d5ba0d4
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
For daemons we might want to disable delta compression entirely, or
in some strange case an administrator might need to turn of delta
reuse. Expose these normally internal pack settings through the pack
configuration section.
Change-Id: I39bfefee8384c864cc04ffac724f197240c8a11a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we are creating a pack the higher level application should be able
to override the PackConfig used, allowing it to control the number of
threads used or how much memory is allocated per writer.
Change-Id: I47795987bb0d161d3642082acc2f617d7cb28d8c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We only use these variables once, so just put them at the proper
use site and avoid assigning the local variable. The code is a
bit shorter and the intent is a little bit more clear.
Change-Id: I70d120fb149b612ac93055ea39bc053b8d90a5db
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This refactoring permits applications to configure global per-process
settings for all packing and easily pass it through to per-request
PackWriters, ensuring that the process configuration overrides the
repository specific settings.
For example this might help in a daemon environment where the server
wants to cap the resources used to serve a dynamic upload pack
request, even though the repository's own pack.* settings might be
configured to be more aggressive. This allows fast but less bandwidth
efficient serving of clients, while still retaining good compression
through a cron managed `git gc`.
Change-Id: I58cc5e01b48924b1a99f79aa96c8150cdfc50846
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Currently, a NullPointerException occurs in this case. We should
instead throw a more meaningful Exception with a proper message.
This is a very "stupid" implementation which simply checks for
the existence of a ".gitmodules" file.
Bug: 300731
Bug: 306765
Bug: 308452
Bug: 314853
Change-Id: I155aa340a85cbc5d7d60da31dba199fc30689b67
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
the following tests fail under windows because certain inputstreams
are not closed and files cannot be deleted because of that. The
main problem I found is UnpackedObject.InflaterInputStream.close().
This method may throw exceptions found by checkValidEndOfStream()
but doesn't call super.close() before leaving. It is not clear to me
which resources a close() method should release before it throws an
exception. But those reseources which are not published to the
outside and which therefore cannot be closed by other means have to
be closed in all cases.
I changed the close() method to call super.close() under all
circumstances.
failing tests:
testStandardFormat_LargeObject_TruncatedZLibStream(org.eclipse.jgit.storage.file.UnpackedObjectTest)
testStandardFormat_LargeObject_TrailingGarbage(org.eclipse.jgit.storage.file.UnpackedObjectTest)
testPackFormat_SmallObject(org.eclipse.jgit.storage.file.UnpackedObjectTest)
Change-Id: Id2e609a29e725aad953ff9bd88af6381df38399d
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
By deferring tag sorting until the commit is produced by the walker
we can avoid an infinite loop that was triggered by trying to sort
tags while allocating a commit. This also avoids needing to look
at commits which aren't going to be produced in the result.
Bug: 321103
Change-Id: I25acc739db2ec0221a50b72c2d2aa618a9a75f37
Reviewed-by: Mathias Kinzler <mathias.kinzler@sap.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These need to be dynamic based on the current thread's environment
at time of execution in order to be properly localized for the end
user that will be seeing these messages.
Change-Id: I4976f462cfe606edd2761c0e36b2f6b20f63d53c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
By permitting the caller of PackWriter to select the Executor it
uses for task execution, we give the caller the ability to manage
the lifecycle of the thread pool, including reusing it across
concurrent pack generators.
This is the first step to supporting application thread pools
within Daemon or another managed service like Gerrit Code Review.
Change-Id: I96bee7b9c30ff9885f2bd261d0b6daaac713b5a4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add a method isDirectoryFileConflict() to NameConflictTreeWalk which
tells whether the current path is part of a directory/file conflict.
Change-Id: Iffcc7090aaec743dd6f3fd1a333cac96c587ae5d
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This is caused by a recursion in PlotWalk.getTags().
As a hotfix, the sort was simply removed. The sort
must be re-implemented so that parseAny() is not called
again (currently, this happens in the PlotRefComparator).
Change-Id: I060d26fda8a75ac803acaf89cfb7d3b4317328f3
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
File pairs that are very dissimilar during a diff were not being
broken apart into their constituent ADD/DELETE pairs. The leads to
sub-optimal rename detection. Take, for example, this situation:
A file exists at src/a.txt containing "foo". A user renames src/a.txt
to src/b.txt, then adds a new src/a.txt containing "bar".
Even though the old a.txt and the new b.txt are identical, the
rename detection algorithm would not detect it as a rename since
it was already paired in a MODIFY. I added code to split all
MODIFYs below a certain score into their constituent ADD/DELETE
pairs. This allows situations like the one I described above to be
more correctly handled.
Change-Id: I22c04b70581f206bbc68c4cd1ee87a1f663b418e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Add methods to the Repository class which write into MERGE_HEAD
and MERGE_MSG files. Since we have the read methods in the same
class this seems to be the right place.
Change-Id: I5dd65306ceb06e008fcc71b37ca3a649632ba462
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
LockFile.commit fails if another thread concurrently reads
the base file. The problem is fixed by retrying the rename
operation if it fails.
Change-Id: I6bb76ea7f2e6e90e3ddc45f9dd4d69bd1b6fa1eb
Bug: 308506
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
There were some broken links, incorrect uses of @value, an invalid
tag and an outdated comment.
Change-Id: I22886bcc869a4b62bd606ebed40669f7b4723664
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This simplifies the logic for those who already have an ObjectReader
on hand want to reuse it to lookup a single path.
Change-Id: Ief17d6b2a0674ddb34bbc9f43121b756eae960fb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This exposes a load and save method, allowing a Repository to denote
that it has a persistent configuration of some kind which can be
accessed by the application, without needing to know exact details
of how its stored .
Change-Id: I7c414bc0f975b80f083084ea875eca25c75a07b2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* delta: (103 commits)
Discard the uncompressed delta as soon as its compressed
Honor pack.windowlimit to cap memory usage during packing
Honor pack.threads and perform delta search in parallel
Cache small deltas during packing
Implement delta generation during packing
debug-show-packdelta: Dump a pack delta to the console
Initial pack format delta generator
Add debugging toString() method to ObjectToPack
Make ObjectToPack clearReuseAsIs signal available to subclasses
Correctly classify the compressing objects phase
Refactor ObjectToPack's delta depth setting
Configure core.bigFileThreshold into PackWriter
Add doNotDelta flag to ObjectToPack
Add more configuration options to PackWriter
Save object path hash codes during packing
Add path hash code to ObjectWalk
Add getObjectSize to ObjectReader
Allow TemporaryBuffer.Heap to allocate smaller than 8 KiB
Define a constant for 127 in DeltaEncoder
Cap delta copy instructions at 64k
...
Conflicts:
org.eclipse.jgit.pgm/src/org/eclipse/jgit/pgm/Diff.java
org.eclipse.jgit/resources/org/eclipse/jgit/JGitText.properties
org.eclipse.jgit/src/org/eclipse/jgit/JGitText.java
org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RewriteTreeFilter.java
Change-Id: I7c7a05e443a48d32c836173a409ee7d340c70796
This is e.g. useful when a client of the AddCommand has
additional rules to ignore files. In Eclipse a resource can
be set to derived or be excluded by preferences.
Change-Id: I6c47e54a1ce26315faf5ed0723298ad2c2db197c
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Chris Aniszczyk