Commit Graph

5820 Commits

Author SHA1 Message Date
Shawn O. Pearce 2bb8defa54 IndexPack: Tighten up new and base object bookkeeping
The only current consumer of these collections is ReceivePack,
where it needs to test ObjectId equality between a RevObject and an
ObjectId.  There we were copying from a traditional HashSet<ObjectId>
into an ObjectIdSubclassMap<ObjectId>, as the latter can perform
hashing using ObjectId's native value support, bypassing RevObject's
override on hashCode() and equals().  Instead of doing that copy,
directly create ObjectIdSubclassMap instances inside of ReceivePack.

We also only need to record the objects that do not appear in the
incoming pack, and were therefore copied from the local repositiory
in order to complete delta resolution.  Instead of listing everything
that used an OBJ_REF_DELTA format, list only the objects that we
pulled from the destination repository via a normal ObjectLoader.

ReceivePack can now discard the IndexPack object, and all of its
other data, as soon as these collections are held by the check
connectivity method.  This frees up memory for the ObjectWalk's
own RevObject pool.

Change-Id: I22ef71b45c2045a0202e7fd550a770ee1f6f38a6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 17:04:26 -07:00
Shawn O. Pearce 329a0e1689 ReceivePack: Remove need new,base object id properties
These are more like internal implementation details of how IndexPack
works with ReceivePack to validate the incoming object stream.
Callers who are embedding the ReceivePack logic in their own
application don't really need to know the details of which objects
were used for delta bases in the incoming thin pack, or exactly
which objects were newly transmitted.

Hide these from the API, as exposing them through ReceivePack was
an early mistake.

Change-Id: I7ee44a314fa19e6a8520472ce05de92c324ad43e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 16:32:33 -07:00
Shawn O. Pearce 8279361de8 ReceivePack: Discard IndexPack as soon as possible
The IndexPack object carries a good bit of state within itself about
the objects received over the wire.  The earlier we can discard it,
the sooner the GC is able to reclaim this chunk of memory for other
uses.  So drop it as soon as we are certain the pack is valid and we
have no connectivity concerns.

Change-Id: I1e8bc87c2e9183733043622237a064e55957891f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 16:32:33 -07:00
Shawn O. Pearce 7a91b180c1 ReceivePack: fix ensureProvidedObjectsVisible on thin packs
If ensureProvidedObjectsVisible is enabled we expected any trees or
blobs directly reachable from an advertised reference to be marked
with UNINTERESTING.  Unfortunately ObjectWalk doesn't bother setting
this until the traversal is complete.  Even then it won't necessarily
set it on every tree if the corresponding commit wasn't popped.

When we are going to check the base objects for the received pack,
ensure the UNINTERESTING flag gets carried into every immediately
reachable tree or blob, because these are the ones that the client
might try to use as delta bases in a thin pack.

Change-Id: I5d5fdcf07e25ac9fc360e79a25dff491925e4101
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 16:32:23 -07:00
Shawn O. Pearce 466bec3cc9 ObjectIdSubclassMap: Correct Iterator to throw NoSuchElementException
The Iterator contract says next() shall throw NoSuchElementException
if there are no more items remaining in the iteration.  We got this
wrong when I originally wrote the implementation, so fix it.

Change-Id: Iea25e6569ead5c8b3128b8a368c5b2caebec7ecc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 16:30:21 -07:00
Shawn O. Pearce 4cc7b1c5b0 ObjectIdSubclassMap: Add isEmpty() method
This class behaves like a cross between a Set and a Map, sometimes
we might expect to use the method isEmpty() to test for size() == 0.
So implement it, reducing the surprise folks get when they are given
one of these objects.

Change-Id: I0d68e1243da8e62edf79c6ba4fd925f643e80a88
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 16:30:21 -07:00
Shawn O. Pearce 06ee913c8d IndexPack: Correct thin pack fix using less than 20 bytes
If we need to append less than 20 bytes in order to fix a thin pack
and make it complete, we need to set the length of our file back to
the actual number of bytes used because the original SHA-1 footer was
not completely overwritten.  That extra data will confuse the header
and footer fixup logic when it tries to read to the end of the file.

This isn't a very common case to occur, which is why we've never
seen it before.  Getting a delta that requires a whole object which
uses less than 20 bytes in pack representation is really hard.
Generally a delta generator won't make these, because the delta
would be bigger than simply deflating the whole object.  I only
managed to do this with a hand-crafted pack file where a 1 byte
delta was pointed to a 1 byte whole object.

Normally we try really hard to avoid truncating, because its
typically not safe across network filesystems.  But the odds of
this occurring are very low.  This truncation is done on a file
we have open for writing, will append more content onto, and is
a temporary file that we won't move into position for others to
see until we've validated its SHA-1 is sane.  I don't think the
truncate on NFS issue is something we need to worry about here.

Change-Id: I102b9637dfd048dc833c050890d142f43c1e75ae
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-04-16 15:56:23 -07:00
Robin Rosenberg 6da38b9474 Optimize ref scanning
We can avoid one stat call by trying to perform a directory
listing without checking if the reference File is a directory.
Attempting a directory listing is defined to return. The other
case for null returns from list is when an I/O error occcurs.

Both cases are now intepreted as a possible plain reference. I/O
errors when reading plain references will be handled (ignored)
in scanRef().

Change-Id: I9906ed8c42eab4d6029c781aab87b3b07c1a1d2c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-04-13 23:00:53 +02:00
Matthias Sohn d29618dd41 Merge "Make Repository.getConfig aware of changed config" 2010-04-13 04:16:58 -04:00
Jens Baumgart cc905e7d4b Make Repository.getConfig aware of changed config
In the current implementation Repository reads user and repository 
config only at creation point of time.
The new implementatiopn checks in Repository.getConfig if user or 
repository config have changed on disk and reload the config if 
required. 

Change-Id: Ibd97515919ef66c6f8aa1a4fe8a11a6711335dad
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
2010-04-12 11:48:45 +02:00
Robin Rosenberg b919a94348 Speed up check for modifications of tracked resources
We only need to check file existense if some other stat returns
a value that may mean that the file does not exist. File.length() == 0
or File.lastModified() == 0 are two such properties. We use length
here.

Change-Id: If626b12e7bb4da994b5c086f6a5b7a12c187261c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-04-10 17:17:35 +02:00
Marc Strapetz 2b6c555aee Make parsing of PersonIdent from raw byte array fault-tolerant.
RawParseUtils.parsePersonIdent handles now those invalid byte sequences
which would result in IndexOutOfBoundsException and returns null in this
case.
2010-03-23 09:21:18 +01:00
Shawn O. Pearce 8014dbe9bf Merge branch 'stable-0.7'
* stable-0.7:
  Fix EGit deadlock listing branches of SSH remote
2010-03-22 08:20:39 -07:00
Shawn O. Pearce 0dc93a2f4a Fix EGit deadlock listing branches of SSH remote
When listing branches, EGit only reads the advertisement and
then disconnects.  When it closes down the pack channel the remote
side is waiting for the client to send our list of commands, or a
flush-pkt to let it know there is nothing to do.

However if an error thread is open watching the SSH stderr stream,
we ask for it to finish before we send the flush-pkt.  Unfortunately
the thread won't terminate until the main output stream closes,
which is waiting for the flush-pkt.  A classic network deadlock.

If the output stream needs a flush-pkt we send it before we wait
for the error stream to close.  If the flush-pkt is rejected, we
close down the output stream early, assuming that the remote side
is broken and we will get error information soon.

Change-Id: I8d078a339077756220c113f49d206b1bf295d434
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-20 19:18:14 -07:00
Nico Sallembien 0f95d2d046 Add a paranoid 'must be provided' option to ReceivePack
By default a receive pack assumes that its user will only provide
references to objects that the user already has access to on their
local client.  In certain cases, an additional check to verify the
references point only to reachable objects is necessary.

This additional checking is useful when the code doesn't trust
the client not to provide a forged SHA-1 reference to an object,
in an attempt to access parts of the DAG that they weren't allowed
to see by the configured RefFilter.

Change-Id: I3e4b8505cb2992e3e4be253abb14a1501e47b970
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-18 11:37:59 -07:00
Shawn O. Pearce 6fabb6d204 Merge branch 'stable-0.7'
* stable-0.7:
  Reuse the line buffer between strings in PacketLineIn
  http.server: Use TemporaryBuffer and compress some responses
  Reduce multi-level buffered streams in transport code
  Fix smart HTTP client buffer alignment
  Use "ERR message" for early ReceivePack problems
  Catch and report "ERR message" during remote advertisements
  Wait for EOF on stderr before finishing SSH channel
  Capture non-progress side band #2 messages and put in result
  ReceivePack: Enable side-band-64k capability for status reports
  Use more restrictive patterns for sideband progress scraping
  Prefix remote progress tasks with "remote: "
  Decode side-band channel number as unsigned integer
  Refactor SideBandInputStream construction
  Refactor SideBandOutputStream to be buffered
2010-03-12 17:04:48 -08:00
Shawn O. Pearce 23bd331cb2 Merge branch 'push-sideband' into stable-0.7
* push-sideband:
  Reuse the line buffer between strings in PacketLineIn
  http.server: Use TemporaryBuffer and compress some responses
  Reduce multi-level buffered streams in transport code
  Fix smart HTTP client buffer alignment
  Use "ERR message" for early ReceivePack problems
  Catch and report "ERR message" during remote advertisements
  Wait for EOF on stderr before finishing SSH channel
  Capture non-progress side band #2 messages and put in result
  ReceivePack: Enable side-band-64k capability for status reports
  Use more restrictive patterns for sideband progress scraping
  Prefix remote progress tasks with "remote: "
  Decode side-band channel number as unsigned integer
  Refactor SideBandInputStream construction
  Refactor SideBandOutputStream to be buffered

Change-Id: Ic9689e64e8c87971f2fd402cb619082309d5587f
2010-03-12 17:00:54 -08:00
Shawn O. Pearce 89cdc3b713 Reuse the line buffer between strings in PacketLineIn
When reading pkt-lines off an InputStream we are quite likely to
consume a whole group of fairly short lines in rapid succession, such
as in the have exchange that occurs in the fetch-pack/upload-pack
protocol.  Rather than allocating a throwaway buffer for each
line's raw byte sequence, reuse a buffer that is equal to the small
side-band packet size, which is 1000 bytes.  Text based pkt-lines
are required to be less than this size because many widely deployed
versions of C Git use a statically allocated array of this length.

Change-Id: Ia5c8e95b85020f7f80b6d269dda5059b092d274d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:10:30 -08:00
Shawn O. Pearce 2156aa894c Reduce multi-level buffered streams in transport code
Some transports actually provide stream buffering on their own,
without needing to be wrapped up inside of a BufferedInputStream in
order to smooth out system calls to read or write.  A great example
of this is the JSch SSH client, or the Apache MINA SSHD server.
Both use custom buffering to packetize the streams into the encrypted
SSH channel, and wrapping them up inside of a BufferedInputStream
or BufferedOutputStream is relatively pointless.

Our SideBandOutputStream implementation also provides some fairly
large buffering, equal to one complete side-band packet on the main
data channel.  Wrapping that inside of a BufferedOutputStream just to
smooth out small writes from PackWriter causes extra data copies, and
provides no advantage.  We can save some memory and some CPU cycles
by letting PackWriter dump directly into the SideBandOutputStream's
internal buffer array.

Instead we push the buffering streams down to be as close to the
network socket (or operating system pipe) as possible.  This allows
us to smooth out the smaller reads/writes from pkt-line messages
during advertisement and negotation, but avoid copying altogether
when the stream switches to larger writes over a side band channel.

Change-Id: I2f6f16caee64783c77d3dd1b2a41b3cc0c64c159
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:14 -08:00
Shawn O. Pearce 882d03f70e Fix smart HTTP client buffer alignment
This proved to be a pretty difficult to find bug.  If we read exactly
the number of response bytes from the UnionInputStream and didn't
try to read beyond that length, the last connection's InputStream is
still inside of the UnionInputStream, and UnionInputStream.isEmpty()
returns false.  But there is no data present, so the next read
request to our UnionInputStream returns EOF at a point where the
HTTP client code should have started a new request in order to get
more data.

Instead of wrapping the UnionInputStream, push an dummy stream onto
the end of it which when invoked always starts the next request and
then returns EOF.  The UnionInputStream will automatically pop that
dummy stream out, and then read the next request's stream.

This way we never get into the state where we don't think we need
to run another request in order to satisfy the current read request,
but we really do.

The bug was hidden for so long because BasePackConnection.init()
was always wrapping the InputStream into a BufferedInputStream
with an 8 KiB buffer.  This made the odds of us reading from the
UnionInputStream the exact number of available bytes quite low, as
the BufferedInputStream would always try to read a full buffer size.

Change-Id: I02b5ec3ef6853688687d91de000a5fbe2354915d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:14 -08:00
Shawn O. Pearce d8c3e98d73 Use "ERR message" for early ReceivePack problems
If the application wants to, it can use sendError(String) to send one
or more error messages to clients before the advertisements are sent.
These will cause a C Git client to break out of the advertisement
parsing loop, display "remote error: message\n", and terminate.

Servers can optionally use this to send a detailed error to a client
explaining why it cannot use the ReceivePack service on a repository.
Over smart HTTP these errors are sent in a 200 OK response, and
are in the payload, allowing the Git client to give the end-user
the custom message rather than the generic error "403 Forbidden".

Change-Id: I03f4345183765d21002118617174c77f71427b5a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:14 -08:00
Shawn O. Pearce 1f4a30b80d Catch and report "ERR message" during remote advertisements
GitHub broke the native git protocol a while ago by interjecting an
"ERR message" line into the upload-pack or receive-pack advertisement
list.  This didn't match the expected pattern, so it caused existing
C Git clients to abort with a protocol exception.

These days, C Git clients actually look for this message and abort
with a more graceful notice to the end-user.  JGit should do the
same, including setting up a custom exception type that makes it
easier for higher-level UIs to identify a message from the remote
site and present it to the user.

Change-Id: I51ab62a382cfaf1082210e8bfaa69506fd0d9786
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:14 -08:00
Shawn O. Pearce 243b0d64a6 Wait for EOF on stderr before finishing SSH channel
JSch will allow us to close the connection and then just drop
any late messages coming over the stderr stream for the command.
This makes it easy to lose final output on a command, like from
Gerrit Code Review's post receive hook.

Instead spawn a background thread to copy data from JSch's pipe
into our own buffer, and wait for that thread to receive EOF on the
pipe before we declare the connection closed. This way we don't
have a race condition between the stderr data arriving and JSch
just tearing down the channel.

Change-Id: Ica1ba40ed2b4b6efb7d5e4ea240efc0a56fb71f6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce 673b3984bd Capture non-progress side band #2 messages and put in result
Any messages received on side band #2 that aren't scraped as a
progress message into our ProgressMonitor are now forwarded to a
buffer which is later included into the OperationResult object.
Application callers can use this buffer to present the additional
messages from the remote peer after the push or fetch operation
has concluded.

The smart push connections using the native send-pack/receive-pack
protocol now request side-band-64k capability if it is available
and forward any messages received through that channel onto this
message buffer.  This makes hook messages available over smart HTTP,
or even over SSH.

The SSH transport was modified to redirect the remote command's
stderr stream into the message buffer, interleaved with any data
received over side band #2.  Due to buffering between these two
different channels in the SSH channel mux itself the order of any
writes between the two cannot be ensured, but it tries to stay close.

The local fork transport was also modified to redirect the local
receive-pack's stderr into the message buffer, rather than going to
the invoking JVM's System.err.  This gives applications a chance
to log the local error messages, rather than needing to redirect
their JVM's stderr before startup.

To keep things simple, the application has to wait for the entire
operation to complete before it can see the messages.  This may
be a downside if the user is trying to debug a remote hook that is
blocking indefinitely, the user would need to abort the connection
before they can inspect the message buffer in any sort of UI built
on top of JGit.

Change-Id: Ibc215f4569e63071da5b7e5c6674ce924ae39e11
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce d33f939e8e ReceivePack: Enable side-band-64k capability for status reports
We now advertise the side-band-64k capability inside of ReceivePack,
allowing hooks to echo status messages down the side band channel
instead of over the optional stderr stream.

This change permits hooks running inside of an http:// based push
invocation to still message the end-user with more detailed errors
than the small per-command string in the status report.

Change-Id: I64f251ef2d13ab3fd0e1a319a4683725455e5244
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce 4c44810df4 Use more restrictive patterns for sideband progress scraping
To avoid scraping a non-progress message as though it were a progress
item for the progress monitor, use a more restrictive pattern to
watch the remote side's messages.  These two regexps should match
any message produced by C Git since 42e18fbf5f94 ("more compact
progress display", Oct 2007), and which first appeared in Git 1.5.4.

Change-Id: I57e34cf59d42c1dbcbd1a83dd6f499ce5e39d15d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce 3a9295b894 Prefix remote progress tasks with "remote: "
When we pull task messages off the remote peer via sideband #2
prefix them with the string "remote: " to make it clear to the
user these are coming from the other system, and not from their
local client.

Change-Id: I02c5e67c6be67e30e40d3bc4be314d6640feb519
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce b7e8cefc92 Decode side-band channel number as unsigned integer
This field is unsigned in the protocol, so treat it
as such when we report the channel number in errors.

Change-Id: I20a52809c7a756e9f66b3557a4300ae1e11f6d25
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce f2dc9f0bfe Refactor SideBandInputStream construction
Typically we refer to the raw InputStream (the stream without the
pkt-line headers on it) as rawIn, and the pkt-line header variant
as pckIn.  Refactor our fields to reflect that.  To ensure these
are actually the same underlying InputStream, we now create our own
PacketLineIn wrapper around the supplied raw InputStream.  Its a
very low-cost object since it has only the 4 byte length buffer.

Instead of hardcoding the header length as 5, use the constant from
SideBandOutputStream.  This makes it a bit more clear what we are
consuming, exactly here.

Change-Id: Iebd05538042913536b88c3ddc3adc3a86a841cc5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:08:13 -08:00
Shawn O. Pearce 0af5944cac Refactor SideBandOutputStream to be buffered
Instead of relying on our callers to wrap us up inside of a
BufferedOutputStream and using the proper block sizing, do the
buffering directly inside of SideBandOutputStream.  This ensures
we don't get large write-throughs from BufferedOutputStream that
might overflow the configured packet size.

The constructor of SideBandOutputStream is also beefed up to check
its arguments and ensure they are within acceptable ranges for the
current side-band protocol.

Change-Id: Ic14567327d03c9e972f9734b8228178bc448867d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-12 16:07:45 -08:00
Shawn O. Pearce ab51665336 Merge branch 'stable-0.7'
* stable-0.7:
  Fix NLS to build under Java 5
2010-03-12 10:29:03 -08:00
Shawn O. Pearce d42603578c Merge "Fix NLS to build under Java 5" into stable-0.7 2010-03-12 13:28:45 -05:00
Shawn O. Pearce 24875de6bd Fix NLS to build under Java 5
The tests were using a Locale.ROOT constant which was introduced
in Java 6.  However, we need to retain Java 5 support.

Change-Id: I75c5648fcfc728a9aea2e839d2ad0320f5cf742f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Sasa Zivkov <sasa.zivkov@sap.com>
2010-03-12 10:26:06 -08:00
Karthik K 2c14e78a22 s/StringBuffer/StringBuilder as appropriate where no concurrency is needed 2010-03-11 23:31:38 -08:00
Shawn O. Pearce 4a73596ce4 eclipse-iplog: Skip the initial contribution
The initial contribution was handled through a CQ, and does not need
to be reported as an individual bug record in the project's IP log.
Its an odd corner case that the EMO IP team doesn't want to see,
even though its technically a contribution written by at least
some non-committers.

The project.skipCommit variable can now be used to mask out any
particular change from the IP log.  Currently within JGit we want
to mask only the initial commit, but others could be masked if the
need arises.

Change-Id: I598e08137ddc5913284471ee2aa545f4df685023
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-03-11 15:15:03 -08:00
Sasa Zivkov 2ae9a85045 Provide NLS support in JGit.
The support for NLS relies on java.util API to load a standard
ResourceBundle and then uses java reflection API to inject localized
strings into public String fields of the corresponding instance
of TranslationBundle.

Locale setting is supported per thread to enable concurrent threads
to use different locales. This is useful when JGit runs in a server
context where (error) messages might need to differ per-request to
suit the user's preference.

Change-Id: Ie0e63a0d7bb74eaad495dbe8248595d8a3a76883
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
2010-03-11 15:19:34 +01:00
Shawn Pearce ecb22e4bc8 Merge "Don't doubly wrap TransportException in smart HTTP client" 2010-02-11 20:39:10 -05:00
Nico Sallembien 19126f70e9 Allow users of ReceivePack access to the objects being sent
When implementing branch read access, we need to prove that the
newly created reference(s) point to objects that the user can see.

There are two ways that an object is reachable:
1)  It's reachable from a branch or change the user can see
2)  It was uploaded as part of the pack file the user sent us

This change adds additional methods in ReceivePack that will allow a
server to check the above conditions, in order to ensure that a user
is not trying to create a reference that they cannot see, or that a
malicious user isn't attempting to forge the SHA-1 of an object that
they cannot see in order to base a change off of it.

Change-Id: Ieba75b4f0331e06a03417c37f4ae1ebca4fbee5a
2010-02-11 13:01:27 -08:00
Shawn O. Pearce dd931bd978 Don't doubly wrap TransportException in smart HTTP client
If the readAdvertisedRefs() method throws an exception, its already
closed the connection and wrapped the underlying cause inside of a
suitable TransportException object that it is throwing.  We shouldn't
catch IOException and rethrow a wrapped copy here, because we'll double
wrap the exception thrown by readAdvertisedRefs.  This may obsecure the
root cause of the connection failure from the end-user.

Change-Id: I0ca61560f9888c666323dac8a5582aab25e897ff
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-11 11:44:37 -08:00
Nico Sallembien e54d33b687 Add a RefFilter interface to ReceivePack and UploadPack
When a user of ReceivePack or UploadPack wants to control what refs
are sent to the client, for instance when some refs should be hidden
from some clients, this interface can be extended to provide a fine
grained control over what refs are sent to the client.

Change-Id: Ie6320b0f8922e1a5e1bad91c016bd476ea094366
2010-02-10 11:54:38 -08:00
Shawn O. Pearce efdcb16be3 Remove pointless boolean during native push
The boolean field sentCommand is always true at this point, as it
was assigned just 5 lines above.  So we always set the status of
the update command object to AWAITING_REPORT.

Simplify the logic by dropping the ?: operator.  I assume this is
older code from an attempt to manage dry-run push support within
the native connection, but in fact dry-run support is done higher
up inside of PushProcess.

Change-Id: I450d491bbbb5afecdbf5444ab7169222e856a3bb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-09 19:51:36 -08:00
Robin Rosenberg 23827fefc0 Intermediate workaround for JGit's lack of core.autocrlf support
Windows users by default have core.autocrlf set to true. JGit
does not recognize the flags and thus works as if it is set. In order
to make JGit more compatible with msysgit we set the flag to false
in repositories that JGit creates.

Bug: 301775
Change-Id: I7ea462fe3516e5060b87aa1f7ed63689936830c2
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-02-04 07:17:18 +01:00
Shawn O. Pearce 76b9823005 Use keep(1) instead of add() when skipping an entry
Doing a keep call with a length of 1 will copy the current entry just
like the previous add was doing, but it avoids doing any validation
on the entry.  This is sane because the entry can be assumed to be
already valid, since its originating from the destination index.

Change-Id: I250d902fc98580444af1ba4b8fedceb654541451
Originally: http://thread.gmane.org/gmane.comp.version-control.git/128214/focus=128213
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-03 19:58:20 -08:00
Shawn O. Pearce 29b8fa84e6 Don't allow DirCacheEntry with mode of 0
A 0 file mode in a DirCacheEntry is not a valid mode.  To C git
such a value indicates the record should not be present.  We already
were catching this bad state and exceptioning out when writing tree
objects to disk, but we did not fail when writing the dircache back
to disk.  This allowed JGit applications to create a dircache file
which C git would not like to read.

Instead of checking the mode during writes, we now check during
mutation.  This allows application bugs to be detected sooner and
closer to the cause site.  It also allows us to avoid checking most
of the records which we read in from disk, as we can assume these
are formatted correctly.

Some of our unit tests were not setting the FileMode on their test
entry, so they had to be updated to use REGULAR_FILE.

Change-Id: Ie412053c390b737c0ece57b8e063e4355ee32437
Originally: http://thread.gmane.org/gmane.comp.version-control.git/128214/focus=128213
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Adam W. Hawks <awhawks@writeme.com>
2010-02-03 19:58:20 -08:00
Shawn O. Pearce 329abf7212 Disallow creating invalid DirCacheEntry records
A dircache record must not use a path string like "/a" or "a//b"
as this results in a tree entry being written with a zero length
name component in the record.  C git does not support an empty name,
and neither does any modern filesystem.

A record also must not have a stage outside of the standard 0-3
value range, as there are only 2 bits of space available in the
on-disk format of the record to store the stage information.
Any other values would be truncated into this space, storing a
different value than the caller expected.

If an application tries to create a DirCache record with either of
these wrong values, we abort with an IllegalArgumentException.

Change-Id: I699de149efdfccd85d8adde07d3efd080e3b49c2
Originally: http://thread.gmane.org/gmane.comp.version-control.git/128214
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Adam W. Hawks <awhawks@writeme.com>
2010-02-03 19:58:20 -08:00
Shawn O. Pearce c581672557 Ensure RawText closes the FileInputStream when read is complete
Rather than implementing the file reading logic ourselves, and
wind up leaking the FileInputStream's file descriptor until the
next GC, use IO.readFully(File) which wraps the read loop inside
of a try/finally to ensure the stream is closed before it exits.

Change-Id: I85a3fe87d5eff88fa788962004aebe19d2e91bb4
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Reviewed-by: Roland Grunberg <rgrunber@redhat.com>
2010-02-03 08:23:34 -08:00
Shawn O. Pearce 0e137c4d9e Micro-optimize CanonicalTreeParser next() for ObjectWalk
ObjectWalk is invoking next() for each record we consider in a tree.
Rather than doing several method calls against the current parser,
and testing if we are at eof() at least twice per next() invocation,
do it only once and inline the logic to move the parser forward.

Change-Id: If5938f5d7b3ca24f500a184c9bd2ef193015414e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-02 14:27:49 -08:00
Shawn O. Pearce db54736e71 Fix ObjectWalk corruption when skipping over empty trees
The supplied test case comes out of the example tree identified by
Robert de Wilde and Ilari on #git:

  $ git ls-tree -rt a54f1a85ebf6a7f53aa60a45a1be33f8b078fb7e
  040000 tree bfe058ad536cdb12e127cde63b01472c960ea105    A
  040000 tree 4b825dc642    A/A
  040000 tree 4b825dc642    A/B
  100644 blob abbbfafe3129f85747aba7bfac992af77134c607    B

In this tree, "B" was being skipped because "A/A" as an empty tree
was immediately followed by "A/B", also an empty tree, but the
ObjectWalk broke out too early and never visited "B".

Bug: 286653
Change-Id: I25bcb0bc99d0cbbbdd9c2bd625ad6a691a6d0335
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-02 14:27:45 -08:00
Shawn O. Pearce 0d94a5ca66 Ensure the tree parser resets in ObjectWalk
During dispose() or reset() we are suppose to be restoring the
ObjectWalk instance back to the original pre-walk state, but we
failed to reset the tree parser.  This can lead to confusing state
if the ObjectWalk was reused by the caller, as entries from the
old walk might be reported as part of the new walk.

Change-Id: I6237bae7bfd3794e8b9a92b4dd475559cc72e634
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-02 14:23:30 -08:00
Shawn O. Pearce 986d616ed4 Correct ObjectWalk error message when bad object is found
Instead of including "ObjectId[SHA-1]" in the message, just
us the formatted SHA-1 name of the object by calling name().

Change-Id: I0d1d0e8207f8a3f02188e60242e4e9bf7420e88f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-02 10:49:36 -08:00
Shawn O. Pearce 784b24dde1 Correctly skip over unrecognized optional dircache extensions
We didn't skip the correct number of bytes when we skipped over an
unrecognized but optional dircache extension.  We missed skipping
the 8 byte header that makes up the extension's name and length.

We also didn't include the skipped extension's payload as part of
our index checksum, resuting in a checksum failure when the index
was done reading.  So ensure we always scan through a skipped
section and include it in the checksum computation.

Add a test case for a currently unsupported index extension, 'ZZZZ',
to verify we can still read the DirCache object even though we
don't know what 'ZZZZ' is supposed to mean.

Bug: 301287
Change-Id: I4bdde94576fffe826d0782483fd98cab1ea628fa
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-02 09:09:26 -08:00
Shawn O. Pearce efa8b52df5 Fix server side NPE during push to empty repository
If the repository is empty, we have no HEAD branch, which means we
can't test to see if the HEAD is detached and should be advertised
as a .have line.

Change-Id: I6e85f836e7db057cede812d0d6c1aecbd6cbe6c5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-01 00:52:23 -08:00
Shawn Pearce 79bb1594b4 Merge "Check for remote server exec failures and report" 2010-01-29 13:45:17 -05:00
Robin Rosenberg baaa78f1f0 Merge "Add unsetSection to Config to remove an entire block" 2010-01-28 23:49:38 -05:00
Shawn O. Pearce 48e9a010ae Add unsetSection to Config to remove an entire block
The unsetSection method can be used to delete an entire configuration
block, such as a [branch ""] or [remote ""] section in a file.

Change-Id: I93390c9b2187eb1b0d51353518feaed83bed2aad
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-01-29 05:48:31 +01:00
Robin Rosenberg 94599930e7 Merge "Relax ObjectChecker to permit missing tagger lines" 2010-01-28 23:43:40 -05:00
Shawn Pearce 7588a57bc6 Merge "Added caching for loose object lookup during pack indexing" 2010-01-27 17:04:50 -05:00
Shawn Pearce 6595ab1007 Merge "Don't confuse empty configuration variables with booleans" 2010-01-25 10:26:22 -05:00
Shawn O. Pearce 869c8434f6 Don't confuse empty configuration variables with booleans
Config was confusing the following two variables when writing the
file back to text format:

  [my]
    empty =
    enabled

When parsed, we say that my.empty has 1 value, null, and my.enabled
is an empty string value that in boolean context should be evaluated
as true.

Saving this configuration file back to text format was ignoring the
null value for my.empty, producing a completely different file than
what Config read:

  [my]
    empty
    enabled

Instead handle the writing differently to ensure the original format
is output.  New tests cases cover the expected behavior and return
values from accessor methods.

Change-Id: Id37379ce20cb27e3330923cf989444dd9f2bdd96
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 17:28:14 -08:00
Shawn O. Pearce e905d93f9f Disable the JRE HTTP cache, if any
We don't want to use the JRE cache when fetching content.

Change-Id: Id76f3e618967c98ed4fbc47a1a2a9e77acbe41ab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 14:07:15 -08:00
Shawn O. Pearce 08a77c04b4 Check for remote server exec failures and report
If remote.name.uploadpack or .receivepack is misconfigured and points
to a non-existent command on the remote system, we should receive back
exit status 127.  Report this case specially with the command we used
so the user knows what is going.

Bug: 293703
Change-Id: I7504e7b6238d5d8e698d37db7411c4817a039d08
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 13:11:58 -08:00
Shawn O. Pearce 7c82df1114 Relax ObjectChecker to permit missing tagger lines
Annotated tags created with C Git versions before the introduction
of c818566 ([PATCH] Update tags to record who made them, 2005-07-14),
do not have a "tagger" line present in the object header.  This line
did not appear in C Git until v0.99.1~9.

Ancient projects such as the Linux kernel contain such tags, for
example Linux 2.6.12 is older than when this feature first appeared
in C Git.  Linux v2.6.13-rc4 in late July 2005 is the first kernel
version tag to actually contain a tagger line.

It is therefore acceptable for the header to be missing, and for
the RevTag.getTaggerIdent() method to return null.

Since the Javadoc for getTaggerIdent() already explained that the
identity may be null, we just need to test that this is true when
the header is missing, and allow the ObjectChecker to pass anyway.

Change-Id: I34ba82e0624a0d1a7edcf62ffba72260af6f7e5d
See: http://code.google.com/p/gerrit/issues/detail?id=399
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 12:34:54 -08:00
Shawn O. Pearce 3103abe4e2 Merge branch 'ref-abstract'
* ref-abstract:
  Optimize RefAdvertiser performance by avoiding sorting
  branch: Add -m option to rename a branch
  Replace writeSymref with RefUpdate.link
  Rewrite reference handling to be abstract and accurate
  Create new RefList and RefMap utility types

Change-Id: If43aacf5aa4013edbd0a6e84d84c4f9e94de5be0
2010-01-23 11:11:12 -08:00
Shawn O. Pearce 36f05a9c27 Optimize RefAdvertiser performance by avoiding sorting
Don't copy and sort the set of references if they are passed through
in a RefMap or a SortedMap using the key's natural sort ordering.
Either map is already in the order we want to present the items
to the client in, so copying and sorting is a waste of local CPU
and memory.

Change-Id: I49ada7c1220e0fc2a163b9752c2b77525d9c82c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 11:10:57 -08:00
Shawn O. Pearce 73b6efc928 Replace writeSymref with RefUpdate.link
By using RefUpdate for symbolic reference creation we can reuse
the logic related to updating the reflog with the event, without
needing to expose something such as the legacy ReflogWriter class
(which we no longer have).

Applications using writeSymref must update their code to use the
new pattern of changing the reference through the updateRef method:

    String refName = "refs/heads/master";
    RefUpdate u = repository.updateRef(Constants.HEAD);
    u.setRefLogMessage("checkout: moving to " + refName, false);
    switch (u.link(refName)) {
    case NEW:
    case FORCED:
    case NO_CHANGE:
        // A successful update of the reference
        break;
    default:
        // Handle the failure, e.g. for older behavior
        throw new IOException(u.getResult());
    }

Change-Id: I1093e1ec2970147978a786cfdd0a75d0aebf8010
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 11:10:56 -08:00
Shawn O. Pearce 01b5392cdb Rewrite reference handling to be abstract and accurate
This commit actually does three major changes to the way references
are handled within JGit.  Unfortunately they were easier to do as
a single massive commit than to break them up into smaller units.

Disambiguate symbolic references:
---------------------------------

  Reporting a symbolic reference such as HEAD as though it were
  any other normal reference like refs/heads/master causes subtle
  programming errors.  We have been bitten by this error on several
  occasions, as have some downstream applications written by myself.

  Instead of reporting HEAD as a reference whose name differs from
  its "original name", report it as an actual SymbolicRef object
  that the application can test the type and examine the target of.

  With this change, Ref is now an abstract type with different
  subclasses for the different types.

  In the classical example of "HEAD" being a symbolic reference to
  branch "refs/heads/master", the Repository.getAllRefs() method
  will now return:

      Map<String, Ref> all = repository.getAllRefs();
      SymbolicRef HEAD = (SymbolicRef) all.get("HEAD");
      ObjectIdRef master = (ObjectIdRef) all.get("refs/heads/master");

      assertSame(master,               HEAD.getTarget());
      assertSame(master.getObjectId(), HEAD.getObjectId());

      assertEquals("HEAD",              HEAD.getName());
      assertEquals("refs/heads/master", master.getName());

  A nice side-effect of this change is the storage type of the
  symbolic reference is no longer ambiguous with the storge type
  of the underlying reference it targets.  In the above example,
  if master was only available in the packed-refs file, then the
  following is also true:

      assertSame(Ref.Storage.LOOSE,  HEAD.getStorage());
      assertSame(Ref.Storage.PACKED, master.getStorage());

  (Prior to this change we returned the ambiguous storage of
   LOOSE_PACKED for HEAD, which was confusing since it wasn't
   actually true on disk).

  Another nice side-effect of this change is all intermediate
  symbolic references are preserved, and are therefore visible
  to the application when they walk the target chain.  We can
  now correctly inspect chains of symbolic references.

  As a result of this change the Ref.getOrigName() method has been
  removed from the API.  Applications should identify a symbolic
  reference by testing for isSymbolic() and not by using an arcane
  string comparsion between properties.

Abstract the RefDatabase storage:
---------------------------------

  RefDatabase is now abstract, similar to ObjectDatabase, and a
  new concrete implementation called RefDirectory is used for the
  traditional on-disk storage layout.  In the future we plan to
  support additional implementations, such as a pure in-memory
  RefDatabase for unit testing purposes.

Optimize RefDirectory:
----------------------

  The implementation of the in-memory reference cache, reading, and
  update routines has been completely rewritten.  Much of the code
  was heavily borrowed or cribbed from the prior implementation,
  so copyright notices have been left intact as much as possible.

  The RefDirectory cache no longer confuses symbolic references
  with normal references.  This permits the cache to resolve the
  value of a symbolic reference as late as possible, ensuring it
  is always current, without needing to maintain reverse pointers.

  The cache is now 2 sorted RefLists, rather than 3 HashMaps.
  Using sorted lists allows the implementation to reduce the
  in-memory footprint when storing many refs.  Using specialized
  types for the elements allows the code to avoid additional map
  lookups for auxiliary stat information.

  To improve scan time during getRefs(), the lists are returned via
  a copy-on-write contract.  Most callers of getRefs() do not modify
  the returned collections, so the copy-on-write semantics improves
  access on repositories with a large number of packed references.

  Iterator traversals of the returned Map<String,Ref> are performed
  using a simple merge-join of the two cache lists, ensuring we can
  perform the entire traversal in linear time as a function of the
  number of references: O(PackedRefs + LooseRefs).

  Scans of the loose reference space to update the cache run in
  O(LooseRefs log LooseRefs) time, as the directory contents
  are sorted before being merged against the in-memory cache.
  Since the majority of stable references are kept packed, there
  typically are only a handful of reference names to be sorted,
  so the sorting cost should not be very high.

  Locking is reduced during getRefs() by taking advantage of the
  copy-on-write semantics of the improved cache data structure.
  This permits concurrent readers to pull back references without
  blocking each other.  If there is contention updating the cache
  during a scan, one or more updates are simply skipped and will
  get picked up again in a future scan.

  Writing to the $GIT_DIR/packed-refs during reference delete is
  now fully atomic.  The file is locked, reparsed fresh, and written
  back out if a change is necessary.  This avoids all race conditions
  with concurrent external updates of the packed-refs file.

  The RefLogWriter class has been fully folded into RefDirectory
  and is therefore deleted.  Maintaining the reference's log is
  the responsiblity of the database implementation, and not all
  implementations will use java.io for access.

  Future work still remains to be done to abstract the ReflogReader
  class away from local disk IO.

Change-Id: I26b9287c45a4b2d2be35ba2849daa316f5eec85d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-23 11:10:12 -08:00
Shawn O. Pearce ab697ff18b Create new RefList and RefMap utility types
These types can be used by RefDatabase implementations to manage
the collection.

A RefList stores items sorted by their name, and is an immutable
type using copy-on-write semantics to perform modifications to
the collection.  Binary search is used to locate an existing item
by name, or to locate the proper insertion position if an item does
not exist.

A RefMap can merge up to 3 RefList collections at once during its
entry iteration, allowing items in the resolved or loose RefList
to override items by the same name in the packed RefList.

The RefMap's goal is O(log N) lookup time, and O(N) iteration time,
which is suitable for returning from a RefDatabase.  By relying on
the immutable RefList we might be able to make map construction
nearly constant, making Repository.getAllRefs() an inexpensive
operation if the caches are current.  Since modification is not
common, changes require up to O(N + log N) time to copy the internal
list and collapse or expand the list's array.  As most changes
are made to the loose collection and not the packed collection,
in practice most changes would require less than the full O(N)
time, due to a significantly smaller N in the loose list.

Almost complete test coverage is included in the corresponding
unit tests.  A handful of methods on RefMap are not tested in this
change, as writing the proper test depends on a future refactoring
of how the Ref class represents symbolic reference names.

Change-Id: Ic2095274000336556f719edd75a5c5dd6dd1d857
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-22 16:27:30 -08:00
Constantine Plotnikov cc64794b24 Added caching for loose object lookup during pack indexing
On Windows systems, file system lookup is a slow operation, so
checking each object if it exists during indexing (after receiving
the pack) could take a siginificant time. This patch introduces
CachedObjectDirectory that pre-caches lookup results.

Bug: 300397
Change-Id: I471b93f9bb3ee173eb37cae1d75e9e4eb49985e7
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-21 10:16:42 -08:00
Robin Rosenberg 1c785d6902 Introduce a named constant for the ".git" directory extension
Change-Id: Icfe9205994c6810fcd880054a586e9eef29df9a1
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-01-15 07:17:52 +01:00
Robin Rosenberg 0b8b6b5309 Introduce a named constant for the .git directory.
Not all occurrences of ".git" are replaced by this constant, only
those where it actually refers to the directory with that name, i.e
not the ".git" directory suffix.

Asserts and comment are also excluded from replacement.

Change-Id: I65a9da89aedd53817f2ea3eaab4f9c2bed35d7ee
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2010-01-15 07:17:34 +01:00
Shawn O. Pearce f5eb0d9366 Add JUnit tests for HTTP transport
No Eclipse support for this project is provided, because the
Jetty project does not publish a complete P2 repository.

Change-Id: Ic5fe2e79bb216e36920fd4a70ec15dd6ccfd1468
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 12:30:42 -08:00
Shawn O. Pearce d5bc8be743 Download HEAD by itself if not in info/refs
The dumb HTTP transport needs to download the HEAD ref and
resolve it manually if HEAD does not appear in info/refs.

Its typically for it to not be in the info/refs file.

Change-Id: Ie2a58fdfacfeee530b10edb433b8f98c85568585
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 12:01:28 -08:00
Shawn O. Pearce 8c836c6f21 client side smart HTTP
During fetch over http:// clients now try to take advantage of
the info/refs?service=git-upload-pack URL to determine if the
remote side will support a standard upload-pack command stream.
If so each block of 32 have lines is sent in one POST request,
prefixed by all of the 'want' lines and any previously discovered
common bases as 'have' lines.

During push over http:// clients now try to take advantage of
the info/refs?service=git-receive-pack URL to determine if the
remote side will support a standard receive-pack command stream.
If so, commands are sent along with their pack in a single HTTP
POST request.

Bug: 291002
Change-Id: I8c69b16ac15c442e1a4c3bd60b4ea1a47882b851
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 12:01:28 -08:00
Shawn O. Pearce 5e33a1de83 Simple dumb HTTP server for Git
This is a simple HTTP server that provides the minimum server side
support required for dumb (non-git aware) transport clients.

We produce the info/refs and objects/info/packs file on the fly
from the local repository state, but otherwise serve data as raw
files from the on-disk structure.

In the future we could better optimize the FileSender class and the
servlets that use it to take advantage of direct file to network
APIs in more advanced servlet containers like Jetty.

Our glue package borrows the idea of a micro embedded DSL from
Google Guice and uses it to configure a collection of Filters
and HttpServlets, all of which are matched against requests using
regular expressions.  If a subgroup exists in the pattern, it is
extracted and used for the path info component of the request.

Change-Id: Ia0f1a425d07d035e344ae54faf8aeb04763e7487
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 12:01:24 -08:00
Shawn O. Pearce 71b3484729 Expose PacketLineOut for reuse outside of the transport package
Change-Id: Iaa331a476e28cf2880df5607de36bc9f67d041df
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:56 -08:00
Shawn O. Pearce 7ed6805425 Expose RefAdvertiser for reuse outside of the transport package
By making this class and its methods public, and the actual writing
abstract, we can reuse this code for other formats like writing an
info/refs file for HTTP transports.

Change-Id: Id0e349c30a0f5a8c1527e0e7383b80243819d9c5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:56 -08:00
Shawn O. Pearce e187618b6b Teach UploadPack how to use an RPC style interface
If biDirectionalPipe is false UploadPack does not start out with
the advertisement but instead assumes it should read one block of
want/have lines, process that, and write the ACK/NAKs out.

This means it only is doing one read through the input followed by
one write to the output, which fits with the HTTP request processing
model, and any other type of RPC system.

Change-Id: Ia9f7c46ee556f996367180f15d2caa8572cdd59f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce 2a5c8cb46c Teach ReceivePack how to use an RPC style interface
If biDirectionalPipe is false ReceivePack does not start out with the
advertisement but instead assumes it should read the command set once,
process that, and write the status report out.  This means it only is
doing one read through the input followed by one write to the output,
which fits with the HTTP request processing model, and any other type
of RPC system... assuming that the payload for input can be a very big
entity like the command stream followed by the pack file.

Change-Id: I6f31f6537a3b7498803a8a54e10b0622105718c1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce 3f8fdc0325 Refactor TemporaryBuffer to support reuse in other contexts
Later we are going to add support for smart HTTP, which requires us to
buffer at least some of the request created by a client before we ship
it to the server.  For many requests, we can fit it completely into a
1 MiB buffer, but if it doesn't we can drop back to using the chunked
transfer encoding to send an unknown stream length.

Rather than recoding the block based memory buffer, we refactor the
local file overflow strategy into a subclass, allowing the HTTP client
code to replace this portion of the logic with its own approach to
start the chunked encoding request.

Change-Id: Iac61ea1017b14e0ad3c4425efc3d75718b71bb8e
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce a22b8f5fac Implement multi_ack_detailed protocol extension
The multi_ack_detailed extension breaks out the "ACK %s continue" status
code into "ACK %s common" and "ACK %s ready" states, making it easier to
discover which objects are truely common, and which objects are simply
on a chain the server doesn't care learning about.

Change-Id: Ie8e907424cfbbba84996ca205d49eacf339f9d04
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce 23cb7f9d5c Fix PersonIdent to always use SystemReader
Under unit tests we want the when and timezone to come from the
MockSystemReader and be stable.  We did this for the default
constructor based on the Repository, but failed to do it for the
name,emailAddress variant of the constructor.

Change-Id: I608ac7cf01673729303395e19b379b38fef136b3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce de45869e1d Fix RefWriter creation of info/refs to omit HEAD
We really mean to omit HEAD here, but botched the difference between
getOrigName and getName on the Ref object.  We tested on the wrong
value, picking up the target of the symbolic ref and therefore
included it twice.

Change-Id: If780c65166ccada2e63a4f42bbab752a56b16564
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Shawn O. Pearce 15e2b45a81 Correct spelling error in StringUtils javadoc
Change-Id: Idd98530d5f6fca4de8631aa865e4bcd6e6cf9306
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-12 11:56:55 -08:00
Robin Rosenberg aa4b115791 Read core.worktree regardless of whether .git was discovered or not
The C Git documentation stated that the core.worktree config was
not read when the .git directory was found implicitly (from the
working directory).

This was not true, and had not been so for a long time. The
documentation has been updated to document the existing behaviour.

Change-Id: If1e81b6a981b9d70e849f24872f01c110e9bc950
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-06 18:33:15 -08:00
Igor Fedorenko 7085c0e4ac Explicitly release resources used by java.util.zip.Deflater
Deflater can use significant amount of native (i.e. C) heap
space. Failure to promptly release this memory results
in native memory leak in some cases, particularly severe for
VMs with large java max heap size. For example, running
Team->Commit in one of my EGit workspaces results in ~500M
java process size increase without any significant change
to amount of used java heap when JVM is started with -Xmx1024m.

Change-Id: I649679a8df5683ebedd9380d703513d31c625932
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-06 18:28:43 -08:00
Shawn O. Pearce f50294461b Remove unnecessary semicolon in MergeChunk
Change-Id: I5526edca9816b90f5df2d7f14f24f11d3f5d2ead
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-06 11:23:03 -08:00
Shawn O. Pearce 1b4f76d7bc Merge branch 'cq-diff'
Per CQ 3559 "JGit - Eugene Myers O(ND) difference algorithm" we
have approval to check this into our master branch.

* cq-diff:
  Add file content merge algorithm
  Add performance tests for MyersDiff
  Add javadoc comments, remove unused code, shift comments to correct place
  Fixed MyersDiff to be able to handle more than 100k
  Fix some warnings regarding unnecessary imports and accessing static methods
  Add the "jgit diff" command
  Prepare RawText for diff-index and diff-files
  Add a test class for Myers' diff algorithm
  Add Myers' algorithm to generate diff scripts
  Add set to IntList

Conflicts:
	org.eclipse.jgit/src/org/eclipse/jgit/diff/RawText.java

Change-Id: Ia8e98d81ba1ab52f84d0258a40e6ef5eece9a5b1
CC: Christian Halstrick <christian.halstrick@sap.com>
2010-01-06 09:55:19 -08:00
Christian Halstrick 6d930cd572 Add file content merge algorithm
Adds the file content merge alorithm and tests for merge to jgit.
The merge algorithm:

- Gets as input parameters the common base, the two new contents
  called "ours" and "theirs".

- Computes the Edits from base to ours and from base to theirs with
  the help of MyersDiff.

- Iterates over the edits.

- Independent edits from ours or from theirs will just be applied
  to the result.

- For conflicting edits we first harmonize the ranges of the edits
  so that in the end we have exactly two edits starting and ending
  at the same points in the common base. Then we write the two
  conclicting contents into the result stream.

Change-Id: I411862393e7bf416b6f33ca55ec5af608ff4663
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
[sp: Fixed up two awkard comments in documentation.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-06 09:49:24 -08:00
Shawn O. Pearce 2ef29ed106 UnionInputStream: combines sequential InputStreams into one
The UnionInputStream utility class combines multiple sequential
InputStreams so they appear to the caller as a single stream with
no gaps.  This can be used to concentate streams coming from multiple
independent HTTP connections (for example).

The companion unit test covers the class's full functionality.

Change-Id: I0676c7b5e082a5886bf0e8f43f9fd6c46a666228
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-01-04 08:03:50 -08:00
Shawn O. Pearce 0b821817fc Add getPacks to ObjectDirectory
This exposes the list of known packs, allowing callers to list them
into a context like the objects/info/packs file.

Change-Id: I0b889564bd176836ff5c77ba310c6d229409dcd5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2010-01-04 08:03:36 -08:00
Robin Rosenberg eb63bfc1b8 Recognize Git repository environment variables
This makes the jgit command line behave like the C Git implementation
in the respect.

These variables are not recognized in the core, though we add support
to do the overrides there. Hence other users of the JGit library, like
the Eclipse plugin and others, will not be affected.

GIT_DIR
	The location of the ".git" directory.

GIT_WORK_TREE
	The location of the work tree.

GIT_INDEX_FILE
	The location of the index file.

GIT_CEILING_DIRECTORIES
	A colon (semicolon on Windows) separated list of paths that
	which JGit will not cross when looking for the .git directory.

GIT_OBJECT_DIRECTORY
	The location of the objects directory under which objects are
	stored.

GIT_ALTERNATE_OBJECT_DIRECTORIES
	A colon (semicolon on Windows) separated list of object directories
	to search for objects.

In addition to these we support the core.worktree config setting when
the git directory is set deliberately instead of being found.

Change-Id: I2b9bceb13c0f66b25e9e3cefd2e01534a286e04c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-12-28 15:58:47 -08:00
Robin Rosenberg 5b13adcea9 Add support for creating detached heads
An extra flag when creating a RefUpdate object allows the
caller to destroy the symref and replace it with an object
ref, a.k.a. detached HEAD.

Change-Id: Ia88d48eab1eb4861ebfa39e3be9258c3824a19db
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-12-28 15:58:40 -08:00
Shawn O. Pearce 1ec393e744 Use Constants.OBJECT_ID_STRING_LENGTH instead of LEN * 2
A few locations were doing OBJECT_ID_LENGTH * 2 on their own, as
the old STR_LEN constant wasn't visible.  Replace them with the
new public constant OBJECT_ID_STRING_LENGTH.

Change-Id: Id39bddb52de8c65bb097de042e9d4ed99598201f
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-12-28 15:58:37 -08:00
Robin Rosenberg db9f8126db Get rid of a duplicate constant for SHA-1 length
Since Constants.OBJECT_ID_LENGTH is a compile time constant we
can be sure that it will always be inlined. The same goes for the
associated constant STR_LEN which is now refactored to the Constant
class and given a name better suited for wider use.

Change-Id: I03f52131e64edcd0aa74bbbf36e7d42faaf4a698
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2009-12-28 16:54:43 +01:00
Code Review 2086fdaedd Merge change I5b232f67
* changes:
  Fix typo in ReceivePack.java
2009-12-22 14:10:56 -05:00
Nico Sallembien a8817ed77f Fix typo in ReceivePack.java
The comment indicates that a well-behaved client should not have
sent an update for a ref that already exists, but this in a block
that corresponds to a create command.
2009-12-22 11:02:24 -08:00
Mykola Nikishov 21b6f3434e Method to get a 'humanish' name from a path
Change-Id: Iec0688232bd59d4626111d77633109918e8e1df3
Signed-off-by: Mykola Nikishov <mn@mn.com.ua>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
2009-12-19 00:47:06 +01:00
Mykola Nikishov 7eef835ad5 Replace a hard-coded default remote's name with a constant
Change-Id: I4e369382b8a83a5c0772462d1b176b1e23910ada
2009-11-29 19:31:01 +02:00
Shawn O. Pearce 27a497f83e Move AWT based SSH authenticator to ui bundle
This way SWT based applications don't wind up loading this AWT
based code when using SSH.

Change-Id: I9080f3dd029c2a087e6b687480018997cc5c5d23
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-11-02 17:55:55 -08:00
Shawn O. Pearce 9108035763 Refactor the cached Authenticator data out of AwtAuthenticator
This makes it easier to swap out authenticator implementations and
yet still rely upon being able to configure at least one Authenticator
instance in the JVM and program it with data obtained from outside
of the user interface.

Change-Id: I8c1a0eb8acee1d306f4c3b40a790b7fa0c3abb70
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-11-02 17:55:52 -08:00
Mykola Nikishov cf2edb6518 Move AWT UI code to new org.eclipse.jgit.ui bundle
This new UI bundle contains the org.eclipse.jgit.awtui package,
which was moved out of the org.eclipse.jgit bundle.

org.eclipse.jgit.pgm depends on org.eclipse.jgit.ui, so we need
to update the classpath and make_jgit.sh to include it.

This move takes the awtui classes out of the Maven build, which
means we are no longer able to distribute these classes to our
downstream Maven customers.  The entire Maven package structure
needs to be overhauled so that Eclipse bundle matches 1:1 with the
Maven artifact.

Bug: https://bugs.eclipse.org/291124
Change-Id: Ibf1a9968387e3d11fdce54592f710ec4cc7f1ddb
Signed-off-by: Mykola Nikishov <mn@mn.com.ua>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-31 16:35:10 -07:00
Sasa Zivkov 99ab22c474 Don't create .git/remotes folders on init
For historical reasons JGit was creating .git/remotes folder when
initialization an empty repository.  However, this folder is never
used and the new Git repository format doesn't contain them.

Removed the unnecessary remotes folder creation, it was removed from
C Git in 75c384efb52d ("Do not create $GIT_DIR/remotes/ directory
anymore") by Junio C Hamano on Dec 19 2006.  C Git hasn't made it
for almost 3 years.

Change-Id: I5473f3676a3642afdf9aaac44b7e6319c6e170b9
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-31 16:13:30 -07:00
Robin Rosenberg d4e7b70606 Move pure IO utility functions to a utility class of its own.
According the javadoc, and implied by the name of the class, NB
is about network byte order. The purpose of moving the IO only,
and non-byte order related functions to another class is to
make it easier for new contributors to understand that they
can use these functions in general and it's also makes it easier
to understand where to put new IO related utility functions

Change-Id: I4a9f6b39d5564bc8a694b366e7ff3cc758c5181b
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-31 16:09:12 -07:00
Alex Blewitt 4d91645e89 Remove trailing whitespace at end of line
As discussed on the egit-dev mailing list, we prefer not to have
trailing whitespace in our source code.  Correct all currently
offending lines by trimming them.

Change-Id: I002b1d1980071084c0bc53242c8f5900970e6845
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-31 14:48:44 -07:00
Mykola Nikishov 5208e50b4e Add gitignore to constants
Bug: 291133
Change-Id: I24272d029cff446ffc80063870478577d628f338
Reviewed-by: Alex Blewitt <alex.blewitt@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-16 09:35:28 -07:00
Christian Halstrick 2484ad6fe0 Add javadoc comments, remove unused code, shift comments to correct place
This change only fixes warnings of the eclipse build regarding missing javadocs.
Some comments where just missing, so they have been added. Other comments where
at the wrong (from eclipse point of view) place, so eclipse was complaining.
Also two method which existed for debugging purposes have been removed to get
rid of Eclipse warngins about unused code.
2009-10-09 16:18:59 +02:00
Christian Halstrick b3e4ac2622 Fixed MyersDiff to be able to handle more than 100k
MyersDiff was crashing with ArrayIndexOutOfBoundsException when
diffing huge files. This was because the snake data (begin and
end position while processing chunks of the text) was sequeezed
into a single int. But with longer texts to diff these values
exceeded the range of java int values. It is fixed by squeezing
the two ints into a long and by adding the LongList helper
class.

Change-Id: Iacb9082e1b076e994d1486aa8e512342ad7432b3
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
2009-10-09 00:35:44 +02:00
Constantine Plotnikov 982515cf10 Make the default encoding when reading commits UTF-8
When reading commits the system default encoding was used if no
encoding was specified in the commit. The patch modifies the test
to add a check that commit message was encoded correctly (the
test fails on old implementation if system encoding is not UTF-8)
and fixes Commit.decode() method to use UTF-8 if encoding is not
specified in the commit object.

Change-Id: I27101da3c2eb6edd0c4a9e4c0523e48b286e3cd5
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-08 10:19:34 -07:00
Johannes Schindelin a3aa43ecb1 Add the "jgit diff" command
This commit contains fixes provided by Christian Halstrick.

Bug: 291083
Eclipse-CQ: 3559
Change-Id: If8d187e70a51f31040f99098489e513791dc6415
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-06 07:43:29 -07:00
Johannes Schindelin e34865b88b Prepare RawText for diff-index and diff-files
Bug: 291083
Eclipse-CQ: 3559
Change-Id: Ia02f346a96b5f1e24f8bc9676bd428b968a41222
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-06 07:43:25 -07:00
Johannes Schindelin d8a3b4eb31 Add Myers' algorithm to generate diff scripts
Myers' algorithm is the standard way to generate diff scripts in
an efficient manner (especially memory-wise).

The source contains extensive documentation about the principal
ideas of the algorithm.

Bug: 291083
Eclipse-CQ: 3559
Change-Id: Id873e7c2c204692cb4a1a05d24b733154b9ed49c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-06 07:43:16 -07:00
Johannes Schindelin 9c056fcace Add set to IntList
Some applications may wish to modify an int list.

Bug: 291083
Eclipse-CQ: 3559
Change-Id: Iea871443ec661230aec92397229f1eda6c74216f
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-06 07:42:49 -07:00
Shawn O. Pearce 41e4b2fed4 Move HttpSupport's configureHttpProxy to jgit-pgm
This is the last chunk of code in jgit-core which references the awtui
package.  Moving it to the only consumer in jgit-pgm allows us to move
the awtui package over to the jgit-awtui module.

Change-Id: I2fd81be2076117b2f2c5f8ed45de7f29272af6cf
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-05 12:17:11 -07:00
Shawn O. Pearce 8d5510d51c Move AmazonS3 command line utility to jgit-pgm
This removes one of the few remaining dependencies on AWTAuthenticator
from the core library.  For the most part the interface is identical to
the prior main method.  The jgit-pgm Main class already sets up the
HTTP proxy and authenticator for us, so we don't need to do that in
our new run method.

Change-Id: Ia2cce34d34c97b88214a8bd8f2cc542845e19032
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-05 12:17:11 -07:00
Code Review 720734983b Merge change I908e4c77
* changes:
  Add support for logAllRefUpdates configuration parameter
2009-10-05 10:06:24 -04:00
Christian Halstrick ffc8b42957 Add support for logAllRefUpdates configuration parameter
Honor the configuration parameter core.logAllRefUpdates when writing
reflogs.  Instead of writing reflog entries always only write
reflogs if this parameter is set to true or if the corresponding
file in the <git-dir>/logs directory already exists. In other words:
if you are updating a ref and this parameter is set to false and
there is no file corresponding to your ref in the <git-dir>/logs
folder then no reflog will be written.

This is a fix for the issue http://code.google.com/p/egit/issues/detail?id=4

Change-Id: I908e4c77e3630dc3223b2d2a47cb4534dbe4ed42
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-02 20:49:15 +02:00
Shawn O. Pearce 6828372f3e Document protected members of RevObjectList
Recently Eclipse started complaining about undocumented protected
members here, so lets document them to make the errors go away.

Change-Id: Ia3bc9fa69854953db5dc9a982821a2f3abe8d05a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-02 09:27:42 -07:00
Jonas Fonseca 36b0dcf6cb Make it possible to clear a PlotCommitList
This allows SwingGraphPanes to be reused by simply clearing and
re-filling. Requires RevObjectList initialization to not call clear()
from its constructor, because this will lead PlotCommitList.clear()
to be called before all variables have been initialized.

Change-Id: I14a07124441b58cd88c67da088ba52ef9c30b043
Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-01 16:57:32 -07:00
Sasa Zivkov 96690904f5 Include description for missing bundle prereqs
When throwing MissingBundlePrerequisiteException we
also include the short description, if available, of
each missing object.

This is the fix for the following issue:
http://code.google.com/p/egit/issues/detail?id=25

Change-Id: I5d45aec7873af76a12170d9a500626a7264f2c42
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-10-01 16:41:43 -07:00
Git Development Community 1a6964c827 Initial JGit contribution to eclipse.org
Per CQ 3448 this is the initial contribution of the JGit project
to eclipse.org.  It is derived from the historical JGit repository
at commit 3a2dd9921c8a08740a9e02c421469e5b1a9e47cb.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2009-09-29 16:47:03 -07:00