Commit Graph

4123 Commits

Author SHA1 Message Date
Marc Strapetz 2352561bfb TransportLocal should reuse FS from source repo
This is necessary to make sure that the FS set to e.g. the
CloneCommand will be passed on and used by the new repository

Change-Id: I9f81f65df784099b07e548b91482e7ace3f5a17e
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-29 12:37:24 +01:00
David Turner 243fba9a0a Add a command to deinitialize submodules
Change-Id: Iaaefc2cbafbf083d6ab158b1c378ec69cc76d282
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-27 23:47:25 +01:00
David Turner 4bbc74ba40 Use submodule name instead of path as key in config
When a submodule is moved, the "name" field remains the same, while
the "path" field changes.  Git uses the "name" field in .git/config
when a submodule is initialized, so this patch makes JGit do so too.

Change-Id: I48d8e89f706447b860c0162822a8e68170aae42b
Signed-off-by: David Turner <dturner@twosigma.com>
2017-12-27 22:55:54 +01:00
Marc Strapetz 8cc783ca7d URIish: support for empty ports
Properly parse URLs like "ssh://host:/path"

Bug: 519187
Change-Id: I0054868e30509e4ba919444be16c2a20f741545a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-23 21:18:12 +01:00
Karsten Thoms 3832527b02 Add header Automatic-Module-Name for Java 9
Bug: 529075
Change-Id: I4532ce2c80eb91531d46026676502d636ccda706
Signed-off-by: Karsten Thoms <karsten.thoms@itemis.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-23 01:00:01 +01:00
Matthias Sohn 32775124d1 Merge branch 'stable-4.9'
* stable-4.9:
  Strings#convertGlob: fix escaping of patterns like [\[].

Change-Id: I18d55537002b3153db35f8a6b60f2f5317d17248
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-23 00:36:54 +01:00
Dmitry Pavlenko 279eaf0690 Strings#convertGlob: fix escaping of patterns like [\[].
Originally the patterns were escaped twice leading
to wrong matching results.

Bug: 528886
Change-Id: I26e201b4b0ef51cac08f940b76f381260fa925ca
Signed-off-by: Dmitry Pavlenko <pavlenko@tmatesoft.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-21 18:33:22 -05:00
Dave Borowitz 83a7a3482e Merge changes I0f1df93b,Ifd40129b,I1b059e1a
* changes:
  ConfigTest: Add some additional comment parsing tests
  Config: Drop backslash in invalid escape sequences in subsections
  Config: Match C git behavior more closely in escaping values
2017-12-21 12:01:04 -05:00
Dave Borowitz d8a24ac1cf Merge branch 'stable-4.9'
* stable-4.9:
  Make PackInserter public

Change-Id: I367c6111661f3005419e2becd695e04e65202f97
2017-12-21 11:12:13 -05:00
Dave Borowitz 085d1f9599 Make PackInserter public
The intent with the setCompressionLevel and checkExisting methods (which
are already public) is for callers to be able to call them, but they
can't do that if the class itself is not public.

Change-Id: I014044fec3bfa1d33775500345efd60eb5d45bde
2017-12-21 09:46:55 -05:00
Dave Borowitz be224f8d4c Config: Drop backslash in invalid escape sequences in subsections
These are ignored by C git when parsing:

$ git config -f - --list <<EOF
[foo "x\0y"]
bar = baz
[foo "x\qy"]
bar = baz
[foo "x\by"]
bar = baz
[foo "x\ny"]
bar = baz
[foo "x\ty"]
bar = baz
EOF
foo.x0y.bar=baz
foo.xqy.bar=baz
foo.xby.bar=baz
foo.xny.bar=baz
foo.xty.bar=baz

This behavior is different from value parsing, where an invalid escape
sequence is an error (which JGit already does as well):

$ git config -f - --list <<EOF
[foo]
bar = x\qy
EOF
fatal: bad config line 2 in standard input

Change-Id: Ifd40129b37d9a62df3d886d8d7e22f766f54e9d1
2017-12-21 09:30:52 -05:00
Dave Borowitz fa24425c60 Config: Match C git behavior more closely in escaping values
So this happened:

$ git config -f foo.config foo.a 'x"y'
$ git config -f foo.config foo.b x\\y
$ git config -f foo.config foo.c $'x\ny'
$ git config -f foo.config foo.d $'x\ty'
$ git config -f foo.config foo.e $'x\by'
$ cat foo.config
[foo]
        a = x\"y
        b = x\\y
        c = x\ny
        d = x\ty
        e = y

That last line is my shell interpreting the backspace byte:

$ python -c 'print repr(open("foo.config").read())'
'[foo]\n\ta = x\\"y\n\tb = x\\\\y\n\tc = x\\ny\n\td = x\\ty\n\te = x\x08y\n'

Change-Id: I1b059e1afb544c39ddb587c07bf79a06e99d0a09
2017-12-21 09:30:52 -05:00
Matthias Sohn 2464fa440f Fix remaining javadoc errors raised by doclint
For now ignore doclint "missing" warnings.

Change-Id: I0e5af7a757f4d92ffeeb113f30576a35414d6781
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-21 02:42:14 +01:00
Matthias Sohn fdbaa25db9 Fix javadoc in org.eclipse.jgit util packages
Change-Id: Ia655f45153bcf1d422ffffce6dcf914847e14c4c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-20 22:10:11 +01:00
Matthias Sohn c4207c5fbe Fix javadoc in org.eclipse.jgit treewalk package
Change-Id: I1f9d9a005c7228c9881762a78228accad2c7a6d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-20 22:10:10 +01:00
Matthias Sohn 781e106a26 Fix javadoc in org.eclipse.jgit transport package
Change-Id: Id70e2c27e38bf12f87e66b28fbd0b06908494764
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-20 22:10:09 +01:00
Dave Borowitz c1a02f497a Merge branch 'stable-4.9'
* stable-4.9:
  PackInserter: Ensure objects are written at the end of the pack
  ObjectInserter: Add warning about mixing read-back with writes

Change-Id: I308e7c1c6b72e8d4d9b5d0f4f51e9815fc92d7d7
2017-12-20 14:39:11 -05:00
Dave Borowitz 43ef5dabf1 PackInserter: Ensure objects are written at the end of the pack
When interleaving reads and writes from an unflushed pack, we forgot to
reset the file pointer back to the end of the file before writing more
new objects. This had at least two unfortunate effects:
  * The pack data was potentially corrupt, since we could overwrite
    previous portions of the file willy-nilly.
  * The CountingOutputStream would report more bytes read than the size
    of the file, which stored the wrong PackedObjectInfo, which would
    cause EOFs during reading.

We already had a test in PackInserterTest which was supposed to catch
bugs like this, by interleaving reads and writes. Unfortunately, it
didn't catch the bug, since as an implementation detail we always read a
full buffer's worth of data from the file when inflating during
readback. If the size of the file was less than the offset of the object
we were reading back plus one buffer (8192 bytes), we would completely
accidentally end up back in the right place in the file.

So, add another test for this case where we read back a small object
positioned before a large object. Before the fix, this test exhibited
exactly the "Unexpected EOF" error reported at crbug.com/gerrit/7668.

Change-Id: I74f08f3d5d9046781d59e5bd7c84916ff8225c3b
2017-12-20 12:43:31 -05:00
Dave Borowitz f63ee965d4 ObjectInserter: Add warning about mixing read-back with writes
Change-Id: Ib0460d3c7df315d86f9adca5f66a8fd4c39e4060
2017-12-20 12:16:40 -05:00
Matthias Sohn 7f6aaa5768 Fix javadoc leftovers in org.eclipse.jgit api package
Change-Id: I2150889b5ed04e8739e2367fc9023b750b516398
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-20 08:51:16 +09:00
Matthias Sohn 1c35379100 Remove unused imports
Change-Id: I35370c66e54d93d9b0aa3995e300706956ec0923
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-20 08:21:26 +09:00
David Pursehouse 0c259eaf1d Replace explicit calls to initCause where possible
Where the exception being thrown has a constructor that takes a
Throwable, use that instead of instantiating the exception and then
explicitly calling initCause.

Change-Id: I06a0df407ba751a7af8c1c4a46f9e2714f13dbe3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-20 08:02:34 +09:00
David Pursehouse 19864c6c02 Merge changes I50095928,Idadec0ab,I1f2747d6,I6d2a7e28
* changes:
  LargeObjectException: Add constructor that takes Throwable
  InvalidPatternException: Add constructor that takes Throwable
  Don't unnecessarily explicitly call CorruptObjectException#initCause
  Use new StoredObjectRepresentationNotAvailableException constructor
2017-12-19 18:02:11 -05:00
David Pursehouse c228f201a6 LargeObjectException: Add constructor that takes Throwable
Change-Id: I500959286075f8ba4d0aa5820febd28c9c94ea1b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 23:27:53 +09:00
David Pursehouse 06bab95e00 InvalidPatternException: Add constructor that takes Throwable
Change-Id: Idadec0abc1c6d76f19f51a65a4b45df444c20486
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 23:27:53 +09:00
David Pursehouse 0f1c160aa7 Don't unnecessarily explicitly call CorruptObjectException#initCause
CorruptObjectException has a constructor that takes Throwable and
calls initCause with it.  Use that instead of instantiating the
exception and explicitly calling initCause.

Change-Id: I1f2747d6c4cc5249e93401b9787eb4ceb50cb995
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 23:27:53 +09:00
David Pursehouse 365c6cb387 Use new StoredObjectRepresentationNotAvailableException constructor
In 5e7eed4 a new StoredObjectRepresentationNotAvailableException
constructor was added, that takes a Throwable to initialize the
exception cause.

Update more call sites to use this constructor instead of first
instantiating it and explicitly calling initCause().

All callers now use the new constructor, so annotate the other one as
deprecated.

Change-Id: I6d2a7e289a95f0360ddebf904cfd8b6c18fef10c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 23:27:53 +09:00
Matthias Sohn 7b93ae0987 Fix javadoc in org.eclipse.jgit submodule package
Change-Id: Idc2a32a2e10d1cfc0b23a4f3fd1bf29840724336
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 14:03:21 +01:00
Matthias Sohn 56cc6afeba Fix javadoc in org.eclipse.jgit storage/file and pack package
Change-Id: If1fee165782823dc21d896073f60ee838365463d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 14:00:29 +01:00
Matthias Sohn 0cba440277 Fix javadoc in org.eclipse.jgit revwalk package
Change-Id: I3fabab8afa284b1919ab7bc656cab19e56ed474e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:57:14 +01:00
Matthias Sohn 23f3b6ab24 Fix javadoc in org.eclipse.jgit patch and revplot package
Change-Id: I6f1aba98bf7034cb51494c546806cf1b72dd1473
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:57:11 +01:00
Matthias Sohn 463dad2ed6 Fix javadoc in org.eclipse.jgit nls and notes package
Change-Id: I1b65fba5b4856f98974dc10f549540d401ef916f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:49:59 +01:00
Matthias Sohn a2f2f65390 Fix javadoc in org.eclipse.jgit merge package
Change-Id: I735d0f2e9f10531ed43b7b3dd6314c574392422f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:48:55 +01:00
Matthias Sohn 8a6af368c4 Fix javadoc in org.eclipse.jgit lib package
Change-Id: I1e85a951488177993521ce177fdea212615e6164
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:43:54 +01:00
Matthias Sohn d0342be42d Fix javadoc in org.eclipse.jgit storage/reftree package
Change-Id: Ie206b5340ad2019a1e0bd2bcede2c1e5a279f2d5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:39:10 +01:00
Matthias Sohn 43e26de7ec Fix javadoc in org.eclipse.jgit storage/reftable package
Change-Id: I14764c5d5b2309d82fe2ccc0593120b51328bf41
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:39:10 +01:00
Matthias Sohn 783dbf1b03 Fix javadoc in org.eclipse.jgit storage/pack package
Change-Id: Id1b7d392e1bb36079edaf16450e73a044a318e7e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:39:09 +01:00
Matthias Sohn c6d17443ad Fix javadoc in org.eclipse.jgit storage/io package
Change-Id: I2db781311af79510b26fd05cc0556623e75e3eff
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:39:09 +01:00
Matthias Sohn 5480da5999 Fix javadoc in org.eclipse.jgit storage/file package
Change-Id: Ieb2f66aef2cab7e2a6d8e35c5f5047da881994dd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:39:07 +01:00
Matthias Sohn a224b78675 Fix javadoc in org.eclipse.jgit dfs package
Change-Id: I1f5e3dc3ba34b323ee7244dbefee207ce19e6021
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-19 13:35:25 +01:00
David Pursehouse 305b488975 StoredObjectRepresentationNotAvailableException: Add TODO to fix API in 5.0
Change-Id: I9a0e6acac1b88ddf257fe7b2b7d0142be20250ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 10:48:06 +09:00
David Pursehouse 5e7eed41f6 StoredObjectRepresentationNotAvailableException: Add constructor that takes cause
If the cause can be passed into the constructor, callers don't need to
instantiate it and then explicitly call initCause.

Note that the constructors in this class cause "non-API parameter type"
warnings because ObjectToPack is internal, however it's probably OK
since the only non-internal reference to it is in the pgm.debug package.

Change-Id: Ia4eab24e79f9afe6214ea8160137d941d4048319
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-19 10:40:57 +09:00
David Pursehouse 1e141469f1 Merge changes Ifc34e96a,I72be0a0d
* changes:
  Fix javadoc in org.eclipse.jgit rewalk package
  Fix javadoc in org.eclipse.jgit ketch package
2017-12-18 18:33:40 -05:00
Dave Borowitz 31a2d09c9c Config: Rewrite subsection and value escaping and parsing
Previously, Config was using the same method for both escaping and
parsing subsection names and config values. The goal was presumably code
savings, but unfortunately, these two pieces of the git config format
are simply different.

In git v2.15.1, Documentation/config.txt says the following about
subsection names:

  "Subsection names are case sensitive and can contain any characters
  except newline (doublequote `"` and backslash can be included by
  escaping them as `\"` and `\\`, respectively).  Section headers cannot
  span multiple lines.  Variables may belong directly to a section or to
  a given subsection."

And, later in the same documentation section, about values:

  "A line that defines a value can be continued to the next line by
  ending it with a `\`; the backquote and the end-of-line are stripped.
  Leading whitespaces after 'name =', the remainder of the line after
  the first comment character '#' or ';', and trailing whitespaces of
  the line are discarded unless they are enclosed in double quotes.
  Internal whitespaces within the value are retained verbatim.

  Inside double quotes, double quote `"` and backslash `\` characters
  must be escaped: use `\"` for `"` and `\\` for `\`.

  The following escape sequences (beside `\"` and `\\`) are recognized:
  `\n` for newline character (NL), `\t` for horizontal tabulation (HT,
  TAB) and `\b` for backspace (BS).  Other char escape sequences
  (including octal escape sequences) are invalid."

The main important differences are that subsection names have a limited
set of supported escape sequences, and do not support newlines at all,
either escaped or unescaped. Arguably, it would be easy to support
escaped newlines, but C git simply does not:

  $ git config -f foo.config $'foo.bar\nbaz.quux' value
  error: invalid key (newline): foo.bar
  baz.quux

I468106ac was an attempt to fix one bug in escapeValue, around leading
whitespace, without having to rewrite the whole escaping/parsing code.
Unfortunately, because escapeValue was used for escaping subsection
names as well, this made it possible to write invalid config files, any
time Config#toText is called with a subsection name with trailing
whitespace, like {foo }.

Rather than pile hacks on top of hacks, fix it for real by largely
rewriting the escaping and parsing code.

In addition to fixing escape sequences, fix (and write tests for) a few
more issues in the old implementation:

* Now that we can properly parse it, always emit newlines as "\n" from
  escapeValue, rather than the weird (but still supported) syntax with a
  non-quoted trailing literal "\n\" before the newline. In addition to
  producing more readable output and matching the behavior of C git,
  this makes the escaping code much simpler.
* Disallow '\0' entirely within both subsection names and values, since
  due to Unix command line argument conventions it is impossible to pass
  such values to "git config".
* Properly preserve intra-value whitespace when parsing, rather than
  collapsing it all to a single space.

Change-Id: I304f626b9d0ad1592c4e4e449a11b136c0f8b3e3
2017-12-18 17:46:37 -05:00
Matthias Sohn 5bd3e75094 Fix javadoc in org.eclipse.jgit rewalk package
Change-Id: Ifc34e96a56df30e3b73bef812eea943623af7906
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 17:11:46 +01:00
Matthias Sohn e3a04b6c41 Fix javadoc in org.eclipse.jgit ketch package
Change-Id: I72be0a0d5ac7d3e957597803bd103dbbec141ba4
2017-12-18 17:10:20 +01:00
Matthias Sohn 6dca3cc024 Fix javadoc in org.eclipse.jgit internal and fsck packages
Change-Id: Ib12da10aacda9389d594bf4f521e9a58d1935701
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:25:04 +01:00
Matthias Sohn 107c71a6e6 Fix javadoc in org.eclipse.jgit ignore package
Change-Id: I1a81d371420cd4cf90ab9e048026c0ab8a763018
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:10 +01:00
Matthias Sohn d1804d3f74 Fix javadoc in org.eclipse.jgit hooks package
Change-Id: I3b644048eb0fc19f94ba8f9799b5a2310481103f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:09 +01:00
Matthias Sohn 5e2e111280 Fix javadoc in org.eclipse.jgit gitrepo package
Change-Id: I2604c7bca87f005004d6d6c80fa9a39a7a1e3807
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:09 +01:00
Matthias Sohn 6c83cc1660 Fix javadoc in org.eclipse.jgit fnmatch package
Change-Id: I14384c3bf3c41f8e1c62ec117837c2fc782a832f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:08 +01:00
Matthias Sohn dc91f4ef2c Fix javadoc in org.eclipse.jgit events package
Change-Id: Ifcb9c67daeb9da84a1461a4f731523ac5df3a223
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:08 +01:00
Matthias Sohn 6478577899 Fix javadoc in org.eclipse.jgit errors package
Change-Id: I46aa5c3073b99a311a1a97fc57d8f29d32524482
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:07 +01:00
Matthias Sohn c281692c06 Fix javadoc in org.eclipse.jgit dircache package
Change-Id: Ib485eb217ac6be70519816f8cc0396931043a3d1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 13:19:07 +01:00
David Pursehouse 1463bb9b87 EmtpyCommitException: Add TODO to fix misspelled class name in 5.0
Change-Id: I5bb145d6d6cbaaa2d8cfb2febad7a008ca921236
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-18 05:51:14 -05:00
Matthias Sohn 4d8233f237 Fix javadoc org.eclipse.jgit diff package
Change-Id: I7162d72916abc8533ad37e8b17335ff4a70d6519
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 10:47:19 +01:00
Matthias Sohn 0210e0e299 Fix javadoc in org.eclipse.jgit annotations and api packages
Change-Id: I2761ea91f8dfed16ea54e7a6646af03a30c15ec9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-18 10:45:50 +01:00
Matthias Sohn 4e27a16081 Fix javadoc in org.eclipse.jgit blame package
Change-Id: I116dacc7f4ace341ec6ffae2ab96b53496d89f64
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-17 20:23:33 -05:00
Matthias Sohn df2bcbfb23 Fix javadoc in org.eclipse.jgit attributes package
Change-Id: I12968f4be805266f285a518a014b8bd1d0c23bae
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-17 20:23:24 -05:00
David Pursehouse ed38840eba Merge branch 'stable-4.9'
* stable-4.9:
  Prepare 4.9.3-SNAPSHOT builds
  JGit v4.9.2.201712150930-r
  Write packed-refs directly when cloning

Change-Id: I3b0fad9c45cdf3a918ddb8ea4a37b8d3403aae90
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-16 14:04:01 +01:00
Matthias Sohn 1e56842742 Prepare 4.9.3-SNAPSHOT builds
Change-Id: Ife3f2b0b5407227f89ded42358adbf01d53e14cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-16 03:49:03 +01:00
Zhen Chen 99e70530b9 DfsFsck: Skip unborn branches and symrefs to nowhere
The map returned by getAllRefs includes all refs, including symrefs like
HEAD that may not point to any object yet. That is a valid state (e.g.,
in a new repository that has just been created by "git init"), so skip
such refs.

Change-Id: Ieff8a1aa738b8d09a2990d075eb20601156b70d3
Signed-off-by: Zhen Chen <czhen@google.com>
2017-12-15 14:16:33 -08:00
Matthias Sohn 24b7e91264 JGit v4.9.2.201712150930-r
Change-Id: I013964045d532659a4be3b81d6612b59bc9ffb14
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-15 15:29:36 +01:00
Thomas Wolf c86327979f Write packed-refs directly when cloning
When we are cloning we have no refs at all yet, and there cannot
(or at least should not) be any other thread doing something with
refs yet.

Locking loose refs is thus not needed, since there are no loose
refs yet and nothing should be trying to create them concurrently.

Let's skip the whole loose ref locking when we are cloning a repository.
As a result, JGit will write the refs directly to the packed-refs
file, and will not create the refs/remotes/ directories nor the
lock files underneath when cloning and packed refs are used. Since
no lock files are created, any problems on case-insensitive file
systems with tag or branch names that differ only in case are avoided
during cloning.

Detect if we are cloning based on the following heuristics:
* HEAD is a dangling symref
* There is no loose ref
* There is no packed-refs file

Note, however, that there may still be problems with such tag or
branch names later on. This is primarily a five-minutes-past-twelve
stop-gap measure to resolve the referenced bug, which affects the
Oxygen.2 release.

Bug: 528497
Change-Id: I57860c29c210568165276a123b855e462b6a107a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-15 14:14:03 +01:00
David Pursehouse ec7f88eec8 Config: Remove the include functionality
The Config class must be safe to run against untrusted input files.
Reading arbitrary local system paths using include.path is risky for
servers, including Gerrit Code Review.

This was fixed on master [1] by making "readIncludedConfig" a noop
by default. This allows only FileBasedConfig, which originated from
local disk, to read local system paths.

However, the "readIncludedConfig" method was only introduced in [2]
which was needed by [3], both of which are only on the master branch.
On the stable branch only Config supports includes. Therefore this
commit simply disables the include functionality.

[1] https://git.eclipse.org/r/#/c/113371/
[2] https://git.eclipse.org/r/#/c/111847/
[3] https://git.eclipse.org/r/#/c/111848/

Bug: 528781
Change-Id: I9a3be3f1d07c4b6772bff535a2556e699a61381c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-15 08:58:00 +09:00
David Pursehouse 5fa129555a Merge branch 'stable-4.9'
* stable-4.9:
  InMemoryRepository: Make inner class MemObjDatabase static

Change-Id: I62bb5957de1ae3bc6030ea2181b09efccc48252b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-14 11:05:05 +09:00
Shawn Pearce 3a7704638a Make Config.readIncludedConfig a noop by default
The Config class must be safe to run against untrusted input files.
Reading arbitrary local system paths using include.path is risky for
servers, including Gerrit Code Review.  Return null by default to
incide the include should be ignored.

Only FileBasedConfig which originated from local disk should be trying
to read local system paths.  FileBasedConfig already overrides this
method with its own implementation.

Change-Id: I2ff31753868aa1bbac4a6843a4c23e50bd6f46f3
2017-12-13 17:50:52 -08:00
Shawn Pearce f635aa51f8 DfsBlockCache.hasBlock0: quickly check for file in cache
This can be useful for sophisticated pre-read algorithms to quickly
determine if a file is likely already in cache, especially small
reftables which may be smaller than a typical DFS block size.

Change-Id: I7756948063b722ff650c9ba82060ff9ad554b0ba
2017-12-13 17:10:51 -08:00
David Pursehouse c09ed93e85 Merge branch 'stable-4.9'
* stable-4.9:
  TransportCommand#setTimeout: Specify units for timeout in Javadoc
  Fix typo in key of a JGitText externalized string

Change-Id: Icb60537d2e99cb6e928d9fe07f66695ed69081b5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-13 09:51:57 +09:00
David Pursehouse 2edf66bdd7 Remove unused API filters
Change-Id: Ie8bfa4e8473c2a68a6308de7dbf29ccbca97495d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-13 01:03:55 +01:00
David Pursehouse c40e150790 InMemoryRepository: Make inner class MemObjDatabase static
FindBugs reports:

  This class is an inner class, but does not use its embedded reference
  to the object which created it. This reference makes the instances
  of the class larger, and may keep the reference to the creator object
  alive longer than necessary. If possible, the class should be made
  static.

Change-Id: I9f49de32b4cd81b7ef1239b390353689263bf66e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-12 00:39:21 -05:00
David Turner 623851ec16 TransportCommand#setTimeout: Specify units for timeout in Javadoc
Change-Id: Iadf0e2c6483794e43a5326f6be6bb34c19b00564
Signed-off-by: David Turner <dturner@twosigma.com>
2017-12-12 13:25:05 +09:00
Matthias Sohn 9d79dc95e2 Fix typo in key of a JGitText externalized string
Change-Id: I0d22e24a0aa3b17339ef68849554f7c99b350dde
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-11 09:35:15 +09:00
Matthias Sohn b46656077a Fix typo in key of a JGitText externalized string
Change-Id: I0d22e24a0aa3b17339ef68849554f7c99b350dde
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-10 18:33:23 +01:00
Matthias Sohn 032750acbf Merge branch 'stable-4.9'
* stable-4.9:
  Fix IllegalThreadStateException if stderr closed without exiting

Change-Id: I8a6a6788c2bb000171233b88d9592ed0640ad15e
2017-12-10 18:10:34 +01:00
Dmitry Pavlenko 7ccf38adc6 Fix IllegalThreadStateException if stderr closed without exiting
If some process executed by FS#readPipe lived for a while after
closing stderr, FS#GobblerThread#run failed with an
IllegalThreadStateException exception when accessing p.exitValue()
for the process which is still alive.

Add Process#waitFor calls to wait for the process completion.

Bug: 528335
Change-Id: I87e0b6f9ad0b995dbce46ddfb877e33eaf3ae5a6
Signed-off-by: Dmitry Pavlenko <pavlenko@tmatesoft.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-10 18:00:55 +01:00
David Pursehouse 0e5b9f8a39 CloneCommand: Refactor listing of files in deleteChildren
File.listFiles() returns null if the given File does not represent a
directory, so we can just test for null instead of making a separate
call to FS.DETECTED.isDirectory()

This also avoids a false-positive error from SpotBugs which claims
that there is a potential null-pointer exception on dereferencing the
result of Files.listFiles().

Change-Id: I18e09e391011db997470f5a09d8e38bb604c0213
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-08 17:16:34 +09:00
David Pursehouse c89a11213e DfsBlockCache#creditSpace: release clockLock in finally block
Enclose the call to getStat in a `try`, and release the previously
acquired lock in the `finally`. This prevents that the lock is left
unreleased in the case of an exception being raised in getStat.

Change-Id: I17b4cd134dae887e23a1165253be0ac2d4fd452c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 61790cb931 FS_POSIX: Fix boxing/unboxing of Boolean
Boolean is being abused to represent three possible states of atomic
file creation support (true/enabled, false/disabled, null/undefined).

Replace this with an enum of the three explicit states.

Change-Id: I2cd7fa6422311dc427823304b082ce8da50d2fbe
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 171f84a041 Use constants from StandardCharsets instead of hard-coded strings
Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and
"ISO-8859-1", use the corresponding constants from StandardCharsets.

UnsupportedEncodingException is not thrown when the StandardCharset
constants are used, so remove the now redundant handling.

Because the encoding names are no longer hard-coded strings, also
remove redundant $NON-NLS warning suppressions.

Also replace existing usages of the constants with static imports.

Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 34d56fd277 Merge branch 'stable-4.9'
* stable-4.9:
  LfsStore: Make inner class AppServer static
  DirCacheCheckout#processEntry: Fix typo in javadoc

Change-Id: Id8e4a3c4dc741e6e0182522e72ecb4b34ae419eb
2017-12-07 08:13:45 +09:00
Thomas Wolf 7f2ef4b6ba TransportHttp: retry on IOException with another mechanism
When a 401 occurs on POST and the server advertises Negotiate, we
may get an exception from GSSAPI if the client isn't configured
at all for Kerberos.

Add exception logic similar to the GET case: keep trying other
authentication mechanisms if this occurs.

Bug: 501167
Change-Id: Ic3a3368378d4b3408a35aec93e78ef425d54b3e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-12-06 17:12:48 -05:00
David Pursehouse 2f0a880bd6 DirCacheCheckout#processEntry: Fix typo in javadoc
Change-Id: Ie08a6275f4656ebb46abe8748b73b0fb52a98a35
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-06 20:00:21 +09:00
David Pursehouse df3a7c32a4 ConfigTest: Move pathToString to FileUtils
ConfigTest#pathToString is not visible to FileBasedConfigTest when
bulding with bazel.

Move it to FileUtils rather than messing about with the bazel build
rules to make it visible.

Change-Id: Idcfd4822699dac9dc4a426088a929a9cd31bf53f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-06 09:34:07 +09:00
Marc Strapetz 26d78902f8 FileBasedConfig: support for relative includes
Relative include.path are now resolved against the config's parent
directory. include.path starting with ~/ are resolved against the
user's home directory

Change-Id: I91911ef404126618b1ddd3589294824a0ad919e6
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-04 23:38:24 +01:00
Marc Strapetz b8f2577472 Config: extract method to read included config
Change-Id: I2c55af3e5fe29e0ce8e3d05e4b54ec961d3f863a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-04 23:38:24 +01:00
Matthias Sohn 68c77a4d39 Prepare 4.9.2-SNAPSHOT builds
Change-Id: I5879ad4aee94ff6783b5589728912117f2495dd3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 14:17:43 +01:00
Matthias Sohn a3588cbb2a JGit v4.9.1.201712030800-r
Change-Id: I8bf477778c9dac41cb65233a9e7d590531a836b7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 13:59:36 +01:00
Matthias Sohn d2a23afb53 Silence API warnings for reintroduced ResolveMerger#processEntry
This was silenced before but suppression was unintentionally lost in
merge commit 6858339c1e.
This method was removed in 4.9.0 and reintroduced in 4.9.1 to avoid
breaking EMF compare versions which were built against older versions.

See: abf420302b
Change-Id: I152d58ac885e044bcab682b9423f6cc83b667989
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 13:58:34 +01:00
Matthias Sohn 470629a237 Merge branch 'stable-4.9'
* stable-4.9:
  GC: Delete stale temporary packs and indexes

Change-Id: I49b37845ee8a465404b801a2d8de0205a2e7ba30
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-30 23:24:25 +09:00
Shawn Pearce 64ec65ee47 Merge "Break down DfsBlockCache stats by pack file extension." 2017-11-28 15:37:30 -05:00
Jonathan Nieder 04b8168beb RepoCommand: Skip RemoteReader when encountering a full SHA-1
There is no point in calling back to the RemoteReader to resolve a
40-digit hex SHA-1 to itself.  We already skip that call when not
ignoring remote failures; skip it when ignoring remote failures, too.

This should simplify RemoteReader implementations.

Reported-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I7566968ed1f39b1ad73574fa903faf3ee308eb87
2017-11-28 08:17:04 -08:00
Minh Thai 159da6dacc Break down DfsBlockCache stats by pack file extension.
Change-Id: Iaecf0580279b33e3e2439784528cae7b69fb28bc
Signed-off-by: Minh Thai <mthai@google.com>
2017-11-27 21:55:21 -08:00
Hector Caballero bac4d32d39 GC: Delete stale temporary packs and indexes
When a GC operation is interrupted, temporary packs and indexes can be
left on the pack folder. In big, busy repositories this can lead to
significant amounts of wasted disk space if this interruption is done
with a certain frequency.

Remove stale temporary packs and indexes at the end of the GC process so
they do not accumulate. To avoid interfering with a possible concurrent
JGit GC process in the same repository, only delete temporary files that
are older than one day.

Change-Id: If9b6c1e57fac8a6a0ecc0a703089634caba4caae
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
2017-11-24 05:13:24 -05:00
Stephen Lawson e3f19a5298 Performance improvement on writing a large index
The index header consists of a 4-byte version number. The current
supported version numbers are 2 and 3. The code checks if any entries
are extended. If it finds any entries that are extended it picks version
'3', otherwise it chooses version '2'.


DirCache.java
-Changed the 'extended' check to exit early when any entry is considered
'extended' in the index.

(Of course, I maybe missing a bitwise optimization that is made in
the Java bytecode.)

Change-Id: If70db9454befe683319b974ebd3774060be9445d
Signed-off-by: Stephen Lawson <slawson@ptc.com>
2017-11-23 20:07:41 -05:00
Matthias Sohn ced98986f9 Fix discouraged access warnings in RefsUnreadableInMemoryRepository
Change-Id: I16e291f533625c80b1d39f7cbaf40efbdfcc2c3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:23:07 +01:00
Matthias Sohn f0c119de4f Merge branch 'stable-4.9'
* stable-4.9:
  Ignore warning for minor version change without API change
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Icc33d2e36f140e8714fce088379673a8834ae9de
2017-11-24 01:18:13 +01:00
Matthias Sohn 03abd1dff2 Ignore warning for minor version change without API change
- this is a new warning option in Eclipse 4.7 and higher
- we always change version of all bundles in a release to keep release
engineering simple

Change-Id: Ic7523d77b67b2802f1bab3bc70af250d712a034f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:12:14 +01:00
Matthias Sohn 6858339c1e Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: I7cf2e48934195430b3945b6d74b092f93a3ccd36
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:08:23 +01:00
Matthias Sohn 4ac790fda8 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Iaa99ec84594baf733c993c2d6768281ff14f545a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 23:30:13 +01:00
Matthias Sohn 5f79ef0ba4 Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: I8f6bc09540727c6273d22775a9f9ca382a729c9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 23:07:01 +01:00
Matthias Sohn 182d3b3dad Silence boxing warning
Change-Id: I36c40eb91ce0c51f89b47911fa14beffcbc0a7cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 21:34:37 +01:00
Matthias Sohn 3ac44f1b38 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Ie9c8e0d9172c8d53f075c284bf2a9677980d8dfb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 21:34:08 +01:00
Matthias Sohn 7608de5e5d Prepare 4.5.5-SNAPSHOT builds
Change-Id: I71f946f2875716670a2d74c21a8ab38a1f53a25c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 19:27:22 +01:00
Matthias Sohn 6372c60ab2 JGit v4.5.4.201711221230-r
Change-Id: Ia1079da239c5b3fde1ba8d2acc4e465a46297b4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 18:31:24 +01:00
Christian Halstrick 10e65cb4fa Fix LockFile semantics when running on NFS
When running on NFS there was a chance that JGits LockFile
semantic is broken because File#createNewFile() may allow
multiple clients to create the same file in parallel. This
change provides a fix which is only used when the new config
option core.supportsAtomicCreateNewFile is set to false. The
default for this option is true. This option can only be set in the
global or the system config file. The repository config file is not
taken into account in this case.

If the config option core.supportsAtomicCreateNewFile is true
then File#createNewFile() is trusted and the behaviour doesn't
change.

But if core.supportsAtomicCreateNewFile is set to false then after
successful creation of the lock file a hardlink to that lock file is
created and the attribute nlink of the lock file is checked to be 2. If
multiple clients manage to create the same lock file nlink would be
greater than 2 showing the error.

This expensive workaround is described in
 https://www.time-travellers.org/shane/papers/NFS_considered_harmful.html
section III.d) "Exclusive File Creation"

Change-Id: I3d2cc48d8eb280d5f7039eb94da37804f903be6a
2017-11-22 18:15:11 +01:00
David Pursehouse bd052b94aa Merge branch 'stable-4.9'
* stable-4.9:
  Yet another work-around for a Jsch bug: timeouts

Change-Id: I7cf227c62a3c06f91cee1a6c61719b6fe50da883
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-22 11:31:29 +09:00
Christian Halstrick 218cf3403d Honor trustFolderStats also when reading packed-refs
Then list of packed refs was cached in RefDirectory based on mtime of
the packed-refs file. This may fail on NFS when attributes are cached.
A cached mtime of the packed-refs file could cause JGit to trust the
cached content of this file and to overlook that the file is modified.

Honor the config option trustFolderStats and always read the packed-refs
content if the option is false. By default this option is set to true
and this fix is not active.

Change-Id: I2b65cfaa8f4aba2efbf8a5e865d3f09f927e2eec
2017-11-21 21:21:22 +01:00
Thomas Wolf 5284cc1bf7 Yet another work-around for a Jsch bug: timeouts
Jsch 0.1.54 passes on the values from ~/.ssh/config for
"ServerAliveInterval" and "ConnectTimeout" as read from
the config file to java.net.Socket.setSoTimeout(). That
method expects milliseconds, but the values in the config
file are seconds!

The missing conversion in Jsch means that the timeout is
set way too low, and if the server doesn't respond within
that very short time frame, Jsch kills the connection and
then throws an exception with a message such as "session is
down" or "timeout in waiting for rekeying process".

As a work-around, do the conversion to milliseconds in the
Jsch-facing Config interface of OpenSshConfig. That way Jsch
already gets these values as milliseconds.

Bug: 526867
Change-Id: Ibc9b93f7722fffe10f3e770dfe7fdabfb3b97e74
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-11-20 22:44:23 +01:00
Dave Borowitz 8b3ab4343c Config: Handle leading/trailing single whitespaces
Change-Id: I468106acd2006d0a174c76dfd4bce231f1c7a6f8
2017-11-20 13:55:25 -05:00
David Pursehouse 3efea067a3 Merge branch 'stable-4.9'
* stable-4.9:
  Fix NPE in TransportGitSsh.ExtSession.exec()
  Add missing help text for rev-parse's --verify option
  Remove final modifier on args4j argument field in RevParse

Change-Id: I5ac9e2f185f2210ee76970501710b99b12e93e75
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-20 15:38:43 +09:00
Matthias Sohn 29c5f49f63 Fix NPE in TransportGitSsh.ExtSession.exec()
Bug: 527383
Change-Id: I965860b4e0e9aa1adf6a1bb0735940cbd22adced
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-20 00:35:37 -05:00
Minh Thai 15a189e4e0 Add flag for keeping ref tombstones in GC reftable
A tombstone will prevent a delayed reference update from resurrecting the
deleted reference.

Change-Id: Id9f4df43d435a299ff16cef614821439edef9b11
Signed-off-by: Minh Thai <mthai@google.com>
2017-11-15 22:48:04 -08:00
Minh Thai f3897ac6c3 Preserve ref tombstone when compact top retable stack
Change-Id: I07f5ad572b4642a7cdd76d92aa98f3b6c63d9fa2
Signed-off-by: Minh Thai <mthai@google.com>
2017-11-15 22:47:09 -08:00
David Pursehouse 6ec5d8ddb1 TreeWalk: Make getEolStreamType(OperationType) public
and deprecate getEolStreamType().

This resolves a TODO that was apparently supposed to be done in
version 4.4.

Change-Id: I5c9861aedabdc3f99dcf47519b3959a979e6a591
2017-11-15 10:37:56 +09:00
David Pursehouse 2a15a63005 TreeWalk: Fix indentation
Change-Id: I040aaaf4e3f1e3458e0ece52d1d046f501272fc2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-15 10:37:56 +09:00
David Pursehouse ef12214a7c Merge "ObjectDirectory: Add pack directory getter" 2017-11-14 20:37:26 -05:00
David Pursehouse 33b796869c Merge "Add return tag" 2017-11-14 20:37:14 -05:00
Jonathan Nieder 295c5ea7d3 Merge "ReftableCompactor should accept 0 for minUpdateIndex" 2017-11-14 15:05:19 -05:00
Minh Thai 0e5abbfafc ReftableCompactor should accept 0 for minUpdateIndex
Do not use 0 as the unset value for minUpdateIndex, as input reftables
may have minUpdateIndex starting at 0.

Change-Id: Ie040a6b73d4a5eba5521e51d0ee4580713c84a3e
Signed-off-by: Minh Thai <mthai@google.com>
2017-11-14 10:50:24 -08:00
Hector Caballero 4334b27d3c ObjectDirectory: Add pack directory getter
So far, in order to get the pack directory it was necessary to resolve
it from the object directory. This resolution is already done when
creating the object directory, so simplify the call by just adding a
getter to the pack directory.

Change-Id: I69e783141dc6739024e8b3d5acc30843edd651a7
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
2017-11-14 10:08:42 -05:00
Hector Caballero 178c5d2f0e Add return tag
Eclipse was complaining about the missing tag for the return type.

Change-Id: I43c7e823c3090b19dc8202c1e4d7968e1fa8e6bb
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
2017-11-14 10:08:42 -05:00
Marc Strapetz 9bb126d12d FileUtils.toPath to convert File to Path
When invoking File.toPath(), an (unchecked) InvalidPathException may be
thrown which should be converted to a checked IOException.

For now, we will replace File.toPath() by FileUtils.toPath() only for
code which can already handle IOExceptions.

Change-Id: I0f0c5fd2a11739e7a02071adae9a5550985d4df6
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-11-14 10:07:37 +01:00
Matthias Sohn 4bdf0ec679 Silence invalid @since 3.5 API warning on ResolveMerge#processEntry
This method was removed in 4.9 and reintroduced in
I48ba4308dee73925fa32d6c2fd6b5fd89632c571 as deprecated in 4.9.1 in
order to help EMF Compare to avoid breakage.

Change-Id: Ia638517178313da42ae13ebcf88ad535d9a02723
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-13 22:41:57 +01:00
David Pursehouse 1a091ff5f9 Merge changes I335587ee,I4182a174
* changes:
  Merge branch 'stable-4.9'
  Remove an unused import from PackParserTest
2017-11-10 22:05:50 -05:00
Zhen Chen c93c653ccf Add a setting for fsck to check connectivity only
The object checks may take a long time and sometimes we are only
interested in connectivity check.

This is similar to 'git fsck --connectivity-only'.

Change-Id: I654e8fdccdb16d796f920088429d188cc96734bc
Signed-off-by: Zhen Chen <czhen@google.com>
2017-11-10 19:33:28 -05:00
Matthias Sohn 19ef730d74 Merge branch 'stable-4.9'
* stable-4.9:
  Work around a Jsch bug: ensure the user name is set from URI
  Reintroduce protected method which removal broke EMF Compare

Change-Id: I335587eee279f91bd36c9ba9fc149b17a6db6110
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-11 01:01:03 +01:00
Jonathan Nieder b88204edfb Merge changes I22a8874b,I68ed4abd,I740bc4bf,Icbd17d15
* changes:
  BitmapWalker: do not revisit objects in bitmap
  Use bitmaps for non-commit reachability checks
  Make PackWriterBitmapWalker public
  UploadPackTest: construct commits in test method
2017-11-10 18:52:53 -05:00
Jonathan Tan c9d25505d8 BitmapWalker: do not revisit objects in bitmap
Currently, BitmapWalker walks through every object returned by the
internal ObjectWalk, regardless of whether that object has already
been marked in the bitmap. Set an object filter to ensure that only
bitmap-unmarked objects are walked through.

Change-Id: I22a8874b1e571df3c33643b365036d95f52fe7c7
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2017-11-10 15:41:31 -08:00
Jonathan Tan d3021788d2 Use bitmaps for non-commit reachability checks
Currently, unless RequestPolicy#ANY is used, UploadPack rejects all
non-commit "want" lines unless they were advertized. This is fine,
except when "uploadpack.allowreachablesha1inwant" is true
(corresponding to RequestPolicy#REACHABLE_COMMIT), in which case one
would expect that "want"-ing anything reachable would work.

(There is no restriction that "want" lines must only contain commits -
it is allowed for refs to directly point to trees and blobs, and
requesting for them using "want" lines works.)

This commit has been written to avoid performance regressions as much
as possible. In the usual (and currently working) case where the only
unadvertized things requested are commits, we do a standard RevWalk in
order to avoid incurring the cost of loading bitmaps. However, if
unadvertized non-commits are requested, bitmaps are used instead, and
if there are no bitmaps, a WantNotValidException is thrown (as is
currently done).

Change-Id: I68ed4abd0e477ff415c696c7544ccaa234df7f99
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2017-11-10 15:41:31 -08:00
Jonathan Tan ccf25f9541 Make PackWriterBitmapWalker public
Make PackWriterBitmapWriter class public and move it to a more central
location, in preparation for its use by another class (in a subsequent
commit).

One of its inner static classes, AddUnseenToBitmapFilter, previously
package-private, is also used directly in its former package. Therefore,
AddUnseenToBitmapFilter and its sibling class have been moved to an
internal package instead.

Change-Id: I740bc4bfc4e4e3c857d1ee7d25fe45e90cd22a75
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2017-11-10 15:41:31 -08:00
Thomas Wolf f67af4e16b Work around a Jsch bug: ensure the user name is set from URI
JSch unconditionally overrides the user name given in the connection
URI by the one found in ~/.ssh/config (if that does specify one for
the used host). If the SSH config file has a different user name,
we'll end up using the wrong name, which typically results in an
authentication failure or in Eclipse/EGit asking for a password for
the wrong user.

Unfortunately there is no way to prevent or circumvent this Jsch
behavior up front; it occurs already in the Session constructor at
com.jcraft.jsch.Session() and the Session.applyConfig() method. And
while there is a Session.setUserName() that would enable us to correct
this, that latter method has package visibility only.

So resort to reflection to invoke that setUserName() method to ensure
that Jsch uses the user name from the URI, if there is one.

Bug: 526778
Change-Id: Ia327099b5210a037380b2750a7fd76ff25c41a5a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-11-10 10:16:58 +01:00
Shawn Pearce 2ec71a7c0e Reject pack if delta exceeds array size limit
JGit's delta handling code requires the target to be a single byte
array. Any attempt to inflate a delta larger than fits in the 2GiB
limit will fail with some form of array index exceptions. Check for
this overflow early and abort pack parsing.

Change-Id: I5bb3a71f1e4f4e0e89b8a177c7019a74ee6194da
2017-11-09 09:27:54 -08:00
Zhen Chen fdf8620ec9 Check non-flushed packs for existence of object
Change-Id: Id466909d70ec6f828c904692139891dec880305e
Signed-off-by: Zhen Chen <czhen@google.com>
2017-11-07 14:12:19 -08:00
Matthias Sohn 021e72b801 Update spotbugs-maven-plugin to 3.1.0-RC7
Change-Id: Ic1c2c5b6e3955a1404780e51206513d226600e4b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-04 12:45:22 -04:00
Matthias Sohn 031646963f Merge "Reintroduce protected method which removal broke EMF Compare" into stable-4.9 2017-11-02 12:59:11 -04:00
David Pursehouse 651e17baca Merge branch 'stable-4.9'
* stable-4.9:
  PackInserter: Implement newReader()
  Move some strings from DfsText to JGitText
  FileRepository: Add pack-based inserter implementation
  ObjectDirectory: Factor a method to close open pack handles
  ObjectDirectory: Remove last modified check in insertPack

Change-Id: Ifc9ed6f5d8336bc978818a64eae122bceb933e5d
2017-11-02 08:30:01 +09:00
Dave Borowitz 678c99c057 PackInserter: Implement newReader()
Change-Id: Ib9e7f6439332eaed3d936f895a5271a7d514d3e9
2017-11-01 13:00:24 -04:00
Dave Borowitz 080b4770e7 Move some strings from DfsText to JGitText
Change-Id: I60050e5127d12b6139d81859dba929fcfaabe504
2017-11-01 12:41:38 -04:00
Dave Borowitz f7ceeaa23f FileRepository: Add pack-based inserter implementation
Applications that use ObjectInserters to create lots of individual
objects may prefer to avoid cluttering up the object directory with
loose objects. Add a specialized inserter implementation that produces a
single pack file no matter how many objects. This inserter is loosely
based on the existing DfsInserter implementation, but is simpler since
we don't need to buffer blocks in memory before writing to storage.

An alternative for such applications would be to write out the loose
objects and then repack just those objects later. This operation is not
currently supported with the GC class, which always repacks existing
packs when compacting loose objects. This in turn requires more
CPU-intensive reachability checks and extra I/O to copy objects from old
packs to new packs.

So, the choice was between implementing a new variant of repack, or not
writing loose objects in the first place. The latter approach is likely
less code overall, and avoids unnecessary I/O at runtime.

The current implementation does not yet support newReader() for reading
back objects.

Change-Id: I2074418f4e65853b7113de5eaced3a6b037d1a17
2017-11-01 12:40:53 -04:00
Dave Borowitz 5ce1cc3d43 ObjectDirectory: Factor a method to close open pack handles
Change-Id: I9490c4583fdf801de9c0bb09595ebb8fb4926988
2017-11-01 12:40:13 -04:00
Dave Borowitz 5b5c3a2e1d ObjectDirectory: Remove last modified check in insertPack
GC explicitly handles the case where a new pack has the same name as an
existing pack due to it containing the exact same set of objects. In
this case, the pack passed to insertPack will have the same name as an
existing pack, but it will also almost certainly have a later mtime than
the existing pack.

The loop in insertPack tried to short-circuit when inserting a new pack,
to avoid walking more of the pack list than necessary. Unfortunately,
this means it will never get to the check for an identical name,
resulting in a duplicate entry for the same PackFile in the pack list.
Remove the short-circuit so that insertPack does not insert a duplicate
entry.

Change-Id: I00711b28594622ad3bd104332334e8a3592cda7f
2017-11-01 10:47:59 -04:00
Saša Živkov 176466ed28 Move loggers to the top of their class
There is a possibility of hitting NPE on a logger if it is not the first
statically initialized member. For example, if another static
initializer creates an instance of its class and the logger is used
from the constructor.

Change-Id: I51fa855a8883c107f2e4ef5ac039dc12a571a7ae
2017-10-30 04:43:51 -04:00
Terry Parker 7fcb7854b2 Merge "Align PackStatistics and DfsPackDescription list length" 2017-10-26 18:22:49 -04:00
Matthias Sohn abf420302b Reintroduce protected method which removal broke EMF Compare
So far we follow OSGi semantic versioning [1] which says the following:

"A change in the second (minor) part of the version signals that the
change is backward compatible with consumers of the API package but not
with the providers of that API. That is, when the API package goes from
version 1.5 to 1.6 it is no longer compatible with a provider of that
API but consumers of that API are backward compatible with that API
package."

The change Ib5fbf17bdaf727bc5d0e106ce88f2620d9f87a6f broke EMF Compare
which subclasses ResolveMerger since we added a new parameter to the
protected ResolveMerger.processEntry() method. According to the above
cited OSGi semantic versioning this is ok, implementers should expect
that they break on minor version changes of the API they implement.

This change reintroduces the old processEntry() method in order to help
avoid breakage for existing EMF Compare versions which expect breakage
also for the implementer case only for major version change (in this
case from JGit 4.x to 5.x).

[1] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.pdf

See: https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03431.html
Change-Id: I48ba4308dee73925fa32d6c2fd6b5fd89632c571
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-10-24 23:13:37 +02:00
Han-Wen NIenhuys dc24383b6b Revert "Throw BinaryBlobException from RawParseUtils#lineMap."
This reverts commit f2e64cd895.

The newly added throws clause breaks backward compatibility. 

Change-Id: Ifa76a1b95935e52640b81cd53c171eb17da175c2
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2017-10-24 11:26:10 -04:00
Han-Wen Nienhuys f5ae26f74d Fix typo in comment
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I04912b32b55dd61ad2d2e7ccb5d688655b1dc335
2017-10-24 15:32:50 +02:00
Han-Wen Nienhuys f2e64cd895 Throw BinaryBlobException from RawParseUtils#lineMap.
This makes detection of binaries exact for ResolveMerger and
DiffFormatter: they will classify files as binary regardless of where
the '\0' occurs in the text.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id4342a199628d9406bfa04af1b023c27a47d4014
2017-10-24 15:31:34 +02:00
Han-Wen Nienhuys ced658c445 Avoid loading and merging binary data in ResolveMerger
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Ide4b68872d426aa262142f224acf636c776b35d3
2017-10-24 15:07:04 +02:00
Han-Wen Nienhuys 2b444d781c Introduce ObjectLoader.Filter for delegation
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I78ada3f4023c4304b19f94fb044024f0f0b2cbbe
2017-10-24 15:03:38 +02:00