Commit Graph

5329 Commits

Author SHA1 Message Date
Matthias Sohn dfb9884dbc Add missing @since 4.9 for new API PackParser.setExpectedObjectCount()
Change-Id: I58fa956aea37c696dbc35ecd229d8971d532923f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-07-08 00:06:31 +02:00
Dave Borowitz 106ed5fea0 Merge changes from topic 'packed-batch-ref-update'
* changes:
  RefList: Support capacity <= 0 on new builders
  Short-circuit writing packed-refs if no refs were packed
  BatchRefUpdate: Clarify some ref prefix calls
2017-07-07 13:51:25 -04:00
Zhen Chen abe2a87cb3 Make possible to overwrite the object count
Right now, PackParser relies on the object count from the pack header.
However, when creating Dfs INSERT packs, the object count is not known
at the beginning of the operation. And when we append the base to a
RECEIVE pack, we can't modify the pack header for object count in most
Dfs implementations.

Make it possible to tell PackParser the expected object count by adding
a setter for expectedObjectCount, implementation can overwrite the
object count in onPackHeader function.

Change-Id: I646ca33ab2b843de84edc287abfb65803a56a927
Signed-off-by: Zhen Chen <czhen@google.com>
2017-07-05 14:12:42 -07:00
Dave Borowitz 40748e8303 RefList: Support capacity <= 0 on new builders
Callers may estimate the size, and their estimate may be zero. Silently
allow this, rather than throwing IndexOutOfBoundsException later during
add.

Change-Id: Ife236f9f4ce469c57b18e76cf4fad6feb52cb2b0
2017-07-05 15:51:26 -04:00
Dave Borowitz e08fa5afcd Short-circuit writing packed-refs if no refs were packed
Change-Id: Id691905599b242e48f590138a96e0c86132308fd
2017-07-05 15:51:26 -04:00
Dave Borowitz 28adcce862 BatchRefUpdate: Clarify some ref prefix calls
Inline the old addRefToPrefixes, since it was just a glorified addAll.
Split getPrefixes into a variant, addPrefixesTo, that doesn't allocate a
small Collection on every invocation. Use this in the tight loop of
getTakenPrefixes.

Change-Id: I25cc7feef0c8e312820d85b7ed48559da49b83d2
2017-07-05 15:51:26 -04:00
Christian Halstrick 1968b20066 Merge "Support -merge attribute in binary macro" 2017-07-03 07:48:19 -04:00
Shawn Pearce 5fdbcc1081 Use read ahead during copyPackThroughCache
If a block is missing from the block cache, open the pack stream,
retain the ReadableChannel, and turn on read-ahead.  This should help
to load a medium sized pack into a cold cache more quickly from a
slower IO stream, as the pack is scanned sequentially and missing
blocks are more likely to be available through the read-ahead.

Change-Id: I3300d936b9299be6d9eb642992df7c04bb439cde
2017-06-27 09:52:41 -07:00
Mathieu Cartaud f7e233e450 Support -merge attribute in binary macro
The merger is now able to react to the use of the merge attribute.
The value unset and the custom value 'binary' are handled (-merge
and merge=binary)

Since the specification of the merge attribute states that when the
attribute is unset, ours version must be kept in case of a conflict, we
don't overwrite the file but keep the local version.

Bug: 517128
Change-Id: Ib5fbf17bdaf727bc5d0e106ce88f2620d9f87a6f
Signed-off-by: Mathieu Cartaud <mathieu.cartaud@obeo.fr>
2017-06-27 10:33:50 +02:00
David Turner 695e38a83b Add a test for parsing fsck config options and expose FsckMode enum
These config options allow overriding the message type (error, warn or
ignore) of a specific message ID such as missingEmail.
The supported fsck message IDs are defined in ObjectChecker.ErrorType.

Since TransferConfig.FsckMode wasn't public parsing fsck configuration
options like e.g. fsck.missingEmail=ignore failed with an
IllegalAccessException. Fix this by declaring this enum public.

Change-Id: I3f41ff7a76a846250a63ce92a9fd111eb347269f
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-23 00:10:20 +02:00
Oliver Lockwood 43672700e7 Add --match option for `jgit describe` to CLI
This adds --match option for glob(7) matchers on git tags to jgit
describe in CLI.

Bug: 518377
Change-Id: I745988d565dd4391e8b3e5a91bbfbae575333819
Signed-off-by: Oliver Lockwood <oliver.lockwood@cantab.net>
2017-06-22 09:08:55 +02:00
Oliver Lockwood 060f3699d4 Fix bug in multiple tag handling on DescribeCommand
In the case of multiple tags on the same commit, jgit previously
only ever looked at the last of those tags; git behaviour is to
return the first tag (or first matching one if --match is
specified).

Bug: 518377
Change-Id: I3b6b58ad9f8aa3879ae35b84542b7bddc74a27d6
Signed-off-by: Oliver Lockwood <oliver.lockwood@cantab.net>
2017-06-21 17:25:19 +01:00
Oliver Lockwood af0867cb86 Support --match functionality in DescribeCommand
A `match()` method has been added to the DescribeCommand, allowing
users to specify one or more `glob(7)` matchers as per Git convention.

Bug: 518377
Change-Id: Ib4cf34ce58128eed0334adf6c4a052dbea62c601
Signed-off-by: Oliver Lockwood <oliver.lockwood@cantab.net>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-20 00:23:26 +02:00
Matthias Sohn df638e0cfc Allow to programmatically set FastForwardMode for PullCommand
Bug: 517847
Change-Id: I70d12dbe347a3d7a3528687ee04e52a2052bfb93
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-16 23:20:20 +02:00
Mattias Neuling 0d447b1660 Add support for config "pull.ff
When the configuration entry 'pull.ff' exists the merge of the pull will
use the value as fast forward option.

Bug: 474174
Change-Id: Ic8db2f00095ed81528667b064ff523911e6c122e
Signed-off-by: Mattias Neuling <neuling@dakosy.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-16 23:20:20 +02:00
David Pursehouse b4a46b5ed0 Fetch/PullCommand: Improve Javadoc of setRecurseSubmodules
Annotate the `recurse` parameter as @Nullable and expand the
Javadoc to clarify the precedence of options.

Change-Id: I7aee800cdbf8243133a0d353ef79b97b67ce011e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-06-16 12:52:31 +09:00
Matthias Sohn a45b045c73 Improve javadoc for MergeCommand.setFastForward()
- mark parameter to be nullable
- explain that we fallback to value of merge.ff if set to null and to
--ff if also not configured there

Change-Id: Id077763b95195d21543ac637f9939a6d4179e982
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-15 23:11:46 +02:00
David Pursehouse caefdf54ee Add tests for SubmoduleConfig
Change-Id: Idcc93c2ca95938995d489cffda649c7d7b26c50e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-15 22:29:26 +02:00
Matthias Sohn 0719ac04e6 Merge branch 'stable-4.8'
* stable-4.8:
  Prepare 4.8.1-SNAPSHOT builds
  JGit v4.8.0.201706111038-r

Change-Id: I32024b36093eb095539e02b1788d074bc5237d9f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-15 19:05:12 +02:00
Terry Parker 8dd53135cb Add a new singlePack option to PackConfig
If set, "singlePack" will create a single GC pack file for all
objects reachable from refs/*. If not set, the GC pack will contain
object reachable from refs/heads/* and refs/tags/*, and the GC_REST
pack will contain all other reachable objects.

Change-Id: I56bcb6a9da2c10a0909c2f940c025db6f3acebcb
Signed-off-by: Terry Parker <tparker@google.com>
2017-06-14 15:38:11 -07:00
Matthias Sohn 7922f31fa3 Prepare 4.8.1-SNAPSHOT builds
Change-Id: I7ca4186bbfe5ccc3fed4509a1fe4fc47bb2e8c50
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-12 22:19:30 -04:00
Matthias Sohn 03b8d1a202 JGit v4.8.0.201706111038-r
Change-Id: Ie33623a2191ffffc2ca5756fd078a7003c0c660f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-11 16:39:41 +02:00
David Pursehouse 2dc66e93ca Merge branch 'stable-4.8'
* stable-4.8:
  Use a dedicated executor to run auto-gc in command line interface
  Allow to use an external ExecutorService for background auto-gc
  Fetch: Add --recurse-submodules and --no-recurse-submodules options
  Fix capitalization of command help summaries

Change-Id: I7c85f11daa34c11c7f6389de885a2183a686197e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-06-11 20:24:12 +09:00
Matthias Sohn 8c59e64412 Use a dedicated executor to run auto-gc in command line interface
WorkQueue uses daemon threads so auto-gc would not be executed after
short-lived commands run in command line. Hence use a dedicated executor
which we shutdown when the command finishes.

Change-Id: I0c2429ecfa04387389d159168ba78a020a696228
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-11 12:24:12 +02:00
Matthias Sohn 18ae9bb57d Allow to use an external ExecutorService for background auto-gc
If set use the external executor, otherwise use JGit's own simple
WorkQueue. Move WorkQueue to an internal package so we can reuse it
without exposing it in the public API.

Change-Id: I060d62ffd6692362a88b4bf13ee07b0dc857abe9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-11 12:24:12 +02:00
David Pursehouse b6f954ad42 Fetch: Add --recurse-submodules and --no-recurse-submodules options
Add options to control recursion into submodules on fetch.

Add a callback interface on FetchCommand, to allow Fetch to display
an update "Fetching submodule XYZ" for each submodule.

Change-Id: Id805044b57289ee0f384b434aba1dbd2fd317e5b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-11 12:24:12 +02:00
David Pursehouse 325b51dbd4 Fix capitalization of command help summaries
Change-Id: Ibb245fdca5470efa6340af019b452dd17b01e9c2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-06-11 16:18:13 +09:00
David Pursehouse a7949c1e35 Merge branch 'stable-4.8'
* stable-4.8:
  SubmoduleUpdateCommand#setCallback should return 'this'
  CloneCommand#setCallback should return 'this'
  Prepare 4.7.2-SNAPSHOT builds
  JGit v4.7.1.201706071930-r
  ArchiveCommand: Create prefix entry with commit time
  Run auto GC in the background
  Update Orbit to the Oxygen version R20170516192513

Change-Id: Ibf90b4899d097474e7836e6baab8829e66fca524
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-06-10 14:14:18 +09:00
Matthias Sohn 4acad15086 SubmoduleUpdateCommand#setCallback should return 'this'
The other methods in this class follow the builder pattern, and
return 'this', allowing multiple method calls to be chained in a
single statement.

Update the setCallback method to do the same.
Change-Id: I4ddaacd6d50601f47f61eb6be8b62c8d59cce062
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-10 00:58:23 +02:00
Zhen Chen 9a3e037726 Defer object collision check until pack stream is done
Object collision check requires read from local storage which may be
slow. We already delay this check for blobs, this change will also delay
other objects until the pack stream is closed. In this way, there is no
readCurs call until the pack stream is closed.

Change-Id: I3c8c4720dd19a5f64f8c7ddf07d815ed6877b6aa
Signed-off-by: Zhen Chen <czhen@google.com>
2017-06-08 21:57:03 -07:00
David Pursehouse 9c7b95684c CloneCommand#setCallback should return 'this'
The other methods in this class follow the builder pattern, and
return 'this', allowing multiple method calls to be chained in a
single statement.

Update the setCallback method to do the same.

Change-Id: I0366d28bf66ba47f08ee7eee636d613c9fe079f5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-06-08 22:45:33 +02:00
Matthias Sohn 7ffb574b68 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Prepare 4.7.2-SNAPSHOT builds

Change-Id: Icbbc642d37cc8261ed11d7e34548198ce2a003f4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-08 22:18:17 +02:00
Matthias Sohn 8afd9b1648 Prepare 4.7.2-SNAPSHOT builds
Change-Id: I7c127bd402cd84c68d8f33a32c6aad093a2264c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-08 13:33:44 +02:00
David Pursehouse 39ea39e817 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  JGit v4.7.1.201706071930-r
  ArchiveCommand: Create prefix entry with commit time

Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Change-Id: Id4df76da84fde253ce04484f3437816dc145b4f2
2017-06-08 09:03:25 +09:00
Matthias Sohn 1d14296975 JGit v4.7.1.201706071930-r
Change-Id: I28cd8fbe995d76c8a00e7db6ddf826e983d89043
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-08 01:19:38 +02:00
Matthias Sohn 94c06009aa Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Run auto GC in the background

Change-Id: I5e25765f65d833f13cbe99696ef33055d7f5c4cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-07 16:58:18 +02:00
Yasuhiro Takagi a66e60a986 ArchiveCommand: Create prefix entry with commit time
The cgit archive command creates a prefix (root) directory entry
in the archive file. That entry's time is set to the commit time.

This patch makes jgit's behavior consistent with with cgit:

prefix: hoge/     -> creates prefix directory "hoge/" entry.
prefix: hoge////  -> creates prefix directory "hoge/" entry.
prefix: hoge/foo  -> does not create prefix directory entry, but for
                     each file/directory entry, prefix is added.

Change-Id: I2610e40ce37972c5f7456fdca6337e7fb07176e5
Signed-off-by: Yasuhiro Takagi <ytakagi@bea.hi-ho.ne.jp>
2017-06-05 19:35:46 -04:00
David Turner 6b1e3c58b1 Run auto GC in the background
When running an automatic GC on a FileRepository, when the caller
passes a NullProgressMonitor, run the GC in a background thread. Use a
thread pool of size 1 to limit the number of background threads spawned
for background gc in the same application. In the next minor release we
can make the thread pool configurable.

In some cases, the auto GC limit is lower than the true number of
unreachable loose objects, so auto GC will run after every (e.g) fetch
operation.  This leads to the appearance of poor fetch performance.
Since these GCs will never make progress (until either the objects
become referenced, or the two week timeout expires), blocking on them
simply reduces throughput.

In the event that an auto GC would make progress, it's still OK if it
runs in the background. The progress will still happen.

This matches the behavior of regular git.

Git (and now jgit) uses the lock file for gc.log to prevent simultaneous
runs of background gc. Further, it writes errors to gc.log, and won't
run background gc if that file is present and recent. If gc.log is too
old (according to the config gc.logexpiry), it will be ignored.

Change-Id: I3870cadb4a0a6763feff252e6eaef99f4aa8d0df
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-06-06 01:18:29 +02:00
Shawn Pearce 0d20573d9c fetch: Accept any SHA-1 on lhs of refspec
Allow fetch to accept a SHA-1 on the left hand side of a RefSpec,
enabling callers to pass a specific SHA-1 they want that may not have
been advertised by the remote repository. This can be passed along to
the network protocol to be sent in a "want" line.

Rest of the plumbing only cares about the ObjectId of the Ref in
the askFor map, so make up a fake name using ObjectId.name() to
pass the desired ObjectId into the network code.

Change-Id: I620a189f3de005c403aa68b7d0442d6aa94e6056
2017-06-04 13:58:16 -07:00
Matthias Sohn 5bfeec4fdb Update Orbit to the Oxygen version R20170516192513
Change-Id: I13575c77540149d383747a34754ac338ed60fb52
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-05-30 15:26:31 +02:00
Matthias Sohn df9ce4b981 Prepare 4.9.0-SNAPSHOT builds
Change-Id: I52a4153d573799e861ab104939f51fac1aceb9ee
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-05-30 13:42:07 +02:00
Matthias Sohn 7e1a11f292 Merge branch 'master' into stable-4.8
* master:
  Fix out-of-bounds exception in RepoCommand#relative
  Fix null return from FS.readPipe when command fails to launch
  RenameDetector: Clarify rename limits <= 0
  Remove unnecessary cast for DfsReader
  Allow DfsReader to be subclassed
  Track read IO for DfsReader
  Fix javadoc of TooLargeObjectInPackException
  Exclude refs/tags from bitmap commit selection

Change-Id: I9cd20ded108d2e5d81fa1f0c2cb9aa0eabe1f256
2017-05-30 13:02:02 +02:00
Han-Wen Nienhuys 832808bd50 Fix out-of-bounds exception in RepoCommand#relative
Change-Id: I9c91aa2ff037bff27a8131fba54be22f5f27d80d
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-05-24 23:43:59 +02:00
Bryan Donlan 2204cc9866 Fix null return from FS.readPipe when command fails to launch
When a command invoked from readPipe fails to launch (i.e. the exec call
fails due to a missing command executable), Process.start() throws,
which gets caught by the generic IOException handler, resulting in a
null return. This change detects this case and rethrows a
CommandFailedException instead.

Additionally, this change uses /bin/sh instead of bash for its posix
command failure test, to accomodate building in environments where bash
is unavailable.

Change-Id: Ifae51e457e5718be610c0a0914b18fe35ea7b008
Signed-off-by: Bryan Donlan <bdonlan@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-05-24 23:35:39 +02:00
Dave Borowitz a46b28808b RenameDetector: Clarify rename limits <= 0
Change-Id: I8da386e02272316b8e5e5c2f31ce10ad98bcdb28
2017-05-24 09:26:40 -04:00
Zhen Chen 099dbe6ef5 Remove unnecessary cast for DfsReader
Change-Id: I22aaccfc9d589750f9d1d711b655dd0fd543fa57
Signed-off-by: Zhen Chen <czhen@google.com>
2017-05-22 10:27:20 -07:00
David Pursehouse 9a4486003f Merge "Fix javadoc of TooLargeObjectInPackException" 2017-05-22 01:12:10 -04:00
Shawn Pearce 1513a5632d Allow DfsReader to be subclassed
Necessary if a DFS implementation wants to override close()
to record DfsReaderIoStats.

Change-Id: I144575f9bf1abf2c1fd72030550c4f0795fcf44d
2017-05-19 13:50:36 -07:00
Shawn Pearce 562de51239 Track read IO for DfsReader
Compute how much disk IO a DfsReader is performing, and how long the
sum of those operations took on this reader instance. Implementations
of DFS and interested applications can get the stats by calling the
new DfsReader.getIoStats() method at or after close().

Change-Id: If585741301f29182617933d6406d4a70497f2ca7
2017-05-19 12:23:02 -07:00
Matthias Sohn ef0237564e Fix javadoc of TooLargeObjectInPackException
The API exception should have the same javadoc like the internal
exception org.eclipse.jgit.errors.TooLargeObjectInPackException

Change-Id: Ia7508c77609e53c8e808412ac523a93194648e49
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-05-19 11:22:05 +02:00