Commit Graph

5543 Commits

Author SHA1 Message Date
David Pursehouse 74bfdcd555 FileLfsServlet: Do not pretty print json response
Pretty printing the response is useful for human readers, but most
(if not all) of the time, the response will be read by programs.

Remove it to avoid the additional overhead of the formatting and
extra bytes in the response. Adjust the test accordingly.

Note that LfsProtocolServlet already doesn't use pretty printing,
so this change makes FileLfsServlet's behavior consistent. In fact,
both classes now have duplicate Gson handling; this will be cleaned
up in a separate change.

Change-Id: I113a23403f9222f16e2c0ddf39461398b721d064
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-08 07:41:15 +09:00
David Pursehouse c89a11213e DfsBlockCache#creditSpace: release clockLock in finally block
Enclose the call to getStat in a `try`, and release the previously
acquired lock in the `finally`. This prevents that the lock is left
unreleased in the case of an exception being raised in getStat.

Change-Id: I17b4cd134dae887e23a1165253be0ac2d4fd452c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 0ea73f6282 ArchiveTest: Check result of BufferedInputStream#skip
Change-Id: I5168c75ca2dda6d1e8c8a6c29043ac781fe9295a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 61790cb931 FS_POSIX: Fix boxing/unboxing of Boolean
Boolean is being abused to represent three possible states of atomic
file creation support (true/enabled, false/disabled, null/undefined).

Replace this with an enum of the three explicit states.

Change-Id: I2cd7fa6422311dc427823304b082ce8da50d2fbe
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 171f84a041 Use constants from StandardCharsets instead of hard-coded strings
Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and
"ISO-8859-1", use the corresponding constants from StandardCharsets.

UnsupportedEncodingException is not thrown when the StandardCharset
constants are used, so remove the now redundant handling.

Because the encoding names are no longer hard-coded strings, also
remove redundant $NON-NLS warning suppressions.

Also replace existing usages of the constants with static imports.

Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 5f94e44308 LfsServerText: Fix formatting of invalidPathInfo message
The message is formatted as:

  Invalid pathInfo '/abc' does not match '/{SHA-256}'

but should be:

  Invalid pathInfo: '/abc' does not match '/{SHA-256}'

(i.e. including a colon) to be consistent with other messages.

Change-Id: Ic38aa7d33dd02d7954b95c331a73919a90c69991
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse 899696ed08 InvalidLongObjectIdException: Fix formatting of message
The message is formatted as:

  Invalid id: : abcde...

but should be:

  Invalid id: abcde...

Change-Id: Ie15cacdcf2f168edaee262e6cf8061ebfe9d998d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse b433f2722f Move repeated LFS Gson handling to common class
LfsProtocolServlet and FileLfsServlet both implement the same
setup of the Gson object.

Factor it out to a common class and reuse it.

Change-Id: I5696404fad140cbff1b712ebb04a7e8bba60e4b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse cc3cf93eee DownloadTest: Use ExpectedException instead of try..catch..fail
Change-Id: I5547e64de6cea3d2200887b504245c69692f00b5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 20:02:59 +09:00
David Pursehouse a7e20cc77f FileLfsServlet: Do not pretty print json response
Pretty printing the response is useful for human readers, but most
(if not all) of the time, the response will be read by programs.

Remove it to avoid the additional overhead of the formatting and
extra bytes in the response. Adjust the test accordingly.

Note that LfsProtocolServlet already doesn't use pretty printing,
so this change makes FileLfsServlet's behavior consistent. In fact,
both classes now have duplicate Gson handling; this will be cleaned
up in a separate change.

Change-Id: I113a23403f9222f16e2c0ddf39461398b721d064
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-07 08:42:13 +09:00
David Pursehouse 34d56fd277 Merge branch 'stable-4.9'
* stable-4.9:
  LfsStore: Make inner class AppServer static
  DirCacheCheckout#processEntry: Fix typo in javadoc

Change-Id: Id8e4a3c4dc741e6e0182522e72ecb4b34ae419eb
2017-12-07 08:13:45 +09:00
Thomas Wolf 7f2ef4b6ba TransportHttp: retry on IOException with another mechanism
When a 401 occurs on POST and the server advertises Negotiate, we
may get an exception from GSSAPI if the client isn't configured
at all for Kerberos.

Add exception logic similar to the GET case: keep trying other
authentication mechanisms if this occurs.

Bug: 501167
Change-Id: Ic3a3368378d4b3408a35aec93e78ef425d54b3e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-12-06 17:12:48 -05:00
David Pursehouse 239043e59d LfsStore: Make inner class AppServer static
FindBugs reports:

  This class is an inner class, but does not use its embedded reference
  to the object which created it. This reference makes the instances
  of the class larger, and may keep the reference to the creator object
  alive longer than necessary. If possible, the class should be made
  static.

Change-Id: I245e44678166176de0cfb275e22ddd159f88e0bd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-06 23:27:52 +09:00
David Pursehouse 2f0a880bd6 DirCacheCheckout#processEntry: Fix typo in javadoc
Change-Id: Ie08a6275f4656ebb46abe8748b73b0fb52a98a35
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-06 20:00:21 +09:00
David Pursehouse df3a7c32a4 ConfigTest: Move pathToString to FileUtils
ConfigTest#pathToString is not visible to FileBasedConfigTest when
bulding with bazel.

Move it to FileUtils rather than messing about with the bazel build
rules to make it visible.

Change-Id: Idcfd4822699dac9dc4a426088a929a9cd31bf53f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-06 09:34:07 +09:00
Marc Strapetz 26d78902f8 FileBasedConfig: support for relative includes
Relative include.path are now resolved against the config's parent
directory. include.path starting with ~/ are resolved against the
user's home directory

Change-Id: I91911ef404126618b1ddd3589294824a0ad919e6
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-04 23:38:24 +01:00
Marc Strapetz b8f2577472 Config: extract method to read included config
Change-Id: I2c55af3e5fe29e0ce8e3d05e4b54ec961d3f863a
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2017-12-04 23:38:24 +01:00
Marc Strapetz e1adfee5f5 ConfigTest: fix on Windows
Change-Id: I37a2ef611aef97faf1b891a9660c1745435a915d
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-04 23:38:22 +01:00
David Pursehouse f8eff40cad Merge branch 'stable-4.9'
* stable-4.9:
  Prepare 4.9.2-SNAPSHOT builds
  JGit v4.9.1.201712030800-r
  Silence API warnings for reintroduced ResolveMerger#processEntry

Change-Id: I8a703815a9b3d8b7e06e308cc46a8c458731c013
2017-12-04 01:03:24 +01:00
Matthias Sohn 68c77a4d39 Prepare 4.9.2-SNAPSHOT builds
Change-Id: I5879ad4aee94ff6783b5589728912117f2495dd3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 14:17:43 +01:00
Matthias Sohn a3588cbb2a JGit v4.9.1.201712030800-r
Change-Id: I8bf477778c9dac41cb65233a9e7d590531a836b7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 13:59:36 +01:00
Matthias Sohn d2a23afb53 Silence API warnings for reintroduced ResolveMerger#processEntry
This was silenced before but suppression was unintentionally lost in
merge commit 6858339c1e.
This method was removed in 4.9.0 and reintroduced in 4.9.1 to avoid
breaking EMF compare versions which were built against older versions.

See: abf420302b
Change-Id: I152d58ac885e044bcab682b9423f6cc83b667989
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-12-03 13:58:34 +01:00
David Pursehouse 8b159b6235 Make local assert methods private in test classes
Change-Id: I1bed28a1eac3c7f84cc40841853b9540c72be265
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-01 10:56:18 +09:00
David Pursehouse abfb8eb6e6 Upgrade error_prone_core to 2.1.3
Change-Id: I45619f8089a3cd1390e57158c4bf808c84330a50
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-12-01 10:27:34 +09:00
Matthias Sohn 470629a237 Merge branch 'stable-4.9'
* stable-4.9:
  GC: Delete stale temporary packs and indexes

Change-Id: I49b37845ee8a465404b801a2d8de0205a2e7ba30
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-30 23:24:25 +09:00
Matthias Sohn b181f3eb63 Update Jetty to 9.4.8.v20171121
Change-Id: I4ea421758e3469a59fa4fbbf46ce86757dd267e9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-29 23:09:17 +01:00
Shawn Pearce 64ec65ee47 Merge "Break down DfsBlockCache stats by pack file extension." 2017-11-28 15:37:30 -05:00
Jonathan Nieder 04b8168beb RepoCommand: Skip RemoteReader when encountering a full SHA-1
There is no point in calling back to the RemoteReader to resolve a
40-digit hex SHA-1 to itself.  We already skip that call when not
ignoring remote failures; skip it when ignoring remote failures, too.

This should simplify RemoteReader implementations.

Reported-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I7566968ed1f39b1ad73574fa903faf3ee308eb87
2017-11-28 08:17:04 -08:00
Minh Thai 159da6dacc Break down DfsBlockCache stats by pack file extension.
Change-Id: Iaecf0580279b33e3e2439784528cae7b69fb28bc
Signed-off-by: Minh Thai <mthai@google.com>
2017-11-27 21:55:21 -08:00
David Pursehouse c213f0a4cd Upgrade spotbugs-maven-plugin to 3.1.0-RC8
Change-Id: I650d1a527063204aa9696fe99b4f59bfa7e5fb97
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-26 05:31:17 -05:00
Hector Caballero bac4d32d39 GC: Delete stale temporary packs and indexes
When a GC operation is interrupted, temporary packs and indexes can be
left on the pack folder. In big, busy repositories this can lead to
significant amounts of wasted disk space if this interruption is done
with a certain frequency.

Remove stale temporary packs and indexes at the end of the GC process so
they do not accumulate. To avoid interfering with a possible concurrent
JGit GC process in the same repository, only delete temporary files that
are older than one day.

Change-Id: If9b6c1e57fac8a6a0ecc0a703089634caba4caae
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
2017-11-24 05:13:24 -05:00
Stephen Lawson e3f19a5298 Performance improvement on writing a large index
The index header consists of a 4-byte version number. The current
supported version numbers are 2 and 3. The code checks if any entries
are extended. If it finds any entries that are extended it picks version
'3', otherwise it chooses version '2'.


DirCache.java
-Changed the 'extended' check to exit early when any entry is considered
'extended' in the index.

(Of course, I maybe missing a bitwise optimization that is made in
the Java bytecode.)

Change-Id: If70db9454befe683319b974ebd3774060be9445d
Signed-off-by: Stephen Lawson <slawson@ptc.com>
2017-11-23 20:07:41 -05:00
David Pursehouse ebee164043 Upgrade maven-compiler-plugin to 3.7.0
Change-Id: I9c4a591b809cee702b2d064d74c9809c023c78c8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-23 19:34:13 -05:00
Matthias Sohn ced98986f9 Fix discouraged access warnings in RefsUnreadableInMemoryRepository
Change-Id: I16e291f533625c80b1d39f7cbaf40efbdfcc2c3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:23:07 +01:00
Matthias Sohn f0c119de4f Merge branch 'stable-4.9'
* stable-4.9:
  Ignore warning for minor version change without API change
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Icc33d2e36f140e8714fce088379673a8834ae9de
2017-11-24 01:18:13 +01:00
Matthias Sohn 03abd1dff2 Ignore warning for minor version change without API change
- this is a new warning option in Eclipse 4.7 and higher
- we always change version of all bundles in a release to keep release
engineering simple

Change-Id: Ic7523d77b67b2802f1bab3bc70af250d712a034f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:12:14 +01:00
Matthias Sohn 6858339c1e Merge branch 'stable-4.8' into stable-4.9
* stable-4.8:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: I7cf2e48934195430b3945b6d74b092f93a3ccd36
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-24 01:08:23 +01:00
Matthias Sohn 4ac790fda8 Merge branch 'stable-4.7' into stable-4.8
* stable-4.7:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Iaa99ec84594baf733c993c2d6768281ff14f545a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 23:30:13 +01:00
Matthias Sohn 5f79ef0ba4 Merge branch 'stable-4.6' into stable-4.7
* stable-4.6:
  Silence boxing warning
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: I8f6bc09540727c6273d22775a9f9ca382a729c9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 23:07:01 +01:00
Matthias Sohn 182d3b3dad Silence boxing warning
Change-Id: I36c40eb91ce0c51f89b47911fa14beffcbc0a7cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 21:34:37 +01:00
Matthias Sohn 3ac44f1b38 Merge branch 'stable-4.5' into stable-4.6
* stable-4.5:
  Prepare 4.5.5-SNAPSHOT builds
  JGit v4.5.4.201711221230-r
  Fix LockFile semantics when running on NFS
  Honor trustFolderStats also when reading packed-refs
  Prepare 4.5.4-SNAPSHOT builds
  JGit v4.5.3.201708160445-r

Change-Id: Ie9c8e0d9172c8d53f075c284bf2a9677980d8dfb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 21:34:08 +01:00
Matthias Sohn 7608de5e5d Prepare 4.5.5-SNAPSHOT builds
Change-Id: I71f946f2875716670a2d74c21a8ab38a1f53a25c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 19:27:22 +01:00
Matthias Sohn 6372c60ab2 JGit v4.5.4.201711221230-r
Change-Id: Ia1079da239c5b3fde1ba8d2acc4e465a46297b4d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-22 18:31:24 +01:00
Christian Halstrick 10e65cb4fa Fix LockFile semantics when running on NFS
When running on NFS there was a chance that JGits LockFile
semantic is broken because File#createNewFile() may allow
multiple clients to create the same file in parallel. This
change provides a fix which is only used when the new config
option core.supportsAtomicCreateNewFile is set to false. The
default for this option is true. This option can only be set in the
global or the system config file. The repository config file is not
taken into account in this case.

If the config option core.supportsAtomicCreateNewFile is true
then File#createNewFile() is trusted and the behaviour doesn't
change.

But if core.supportsAtomicCreateNewFile is set to false then after
successful creation of the lock file a hardlink to that lock file is
created and the attribute nlink of the lock file is checked to be 2. If
multiple clients manage to create the same lock file nlink would be
greater than 2 showing the error.

This expensive workaround is described in
 https://www.time-travellers.org/shane/papers/NFS_considered_harmful.html
section III.d) "Exclusive File Creation"

Change-Id: I3d2cc48d8eb280d5f7039eb94da37804f903be6a
2017-11-22 18:15:11 +01:00
David Pursehouse bd052b94aa Merge branch 'stable-4.9'
* stable-4.9:
  Yet another work-around for a Jsch bug: timeouts

Change-Id: I7cf227c62a3c06f91cee1a6c61719b6fe50da883
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-22 11:31:29 +09:00
Christian Halstrick 218cf3403d Honor trustFolderStats also when reading packed-refs
Then list of packed refs was cached in RefDirectory based on mtime of
the packed-refs file. This may fail on NFS when attributes are cached.
A cached mtime of the packed-refs file could cause JGit to trust the
cached content of this file and to overlook that the file is modified.

Honor the config option trustFolderStats and always read the packed-refs
content if the option is false. By default this option is set to true
and this fix is not active.

Change-Id: I2b65cfaa8f4aba2efbf8a5e865d3f09f927e2eec
2017-11-21 21:21:22 +01:00
Thomas Wolf 5284cc1bf7 Yet another work-around for a Jsch bug: timeouts
Jsch 0.1.54 passes on the values from ~/.ssh/config for
"ServerAliveInterval" and "ConnectTimeout" as read from
the config file to java.net.Socket.setSoTimeout(). That
method expects milliseconds, but the values in the config
file are seconds!

The missing conversion in Jsch means that the timeout is
set way too low, and if the server doesn't respond within
that very short time frame, Jsch kills the connection and
then throws an exception with a message such as "session is
down" or "timeout in waiting for rekeying process".

As a work-around, do the conversion to milliseconds in the
Jsch-facing Config interface of OpenSshConfig. That way Jsch
already gets these values as milliseconds.

Bug: 526867
Change-Id: Ibc9b93f7722fffe10f3e770dfe7fdabfb3b97e74
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-11-20 22:44:23 +01:00
Dave Borowitz 8b3ab4343c Config: Handle leading/trailing single whitespaces
Change-Id: I468106acd2006d0a174c76dfd4bce231f1c7a6f8
2017-11-20 13:55:25 -05:00
David Pursehouse 3efea067a3 Merge branch 'stable-4.9'
* stable-4.9:
  Fix NPE in TransportGitSsh.ExtSession.exec()
  Add missing help text for rev-parse's --verify option
  Remove final modifier on args4j argument field in RevParse

Change-Id: I5ac9e2f185f2210ee76970501710b99b12e93e75
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-11-20 15:38:43 +09:00
Matthias Sohn 29c5f49f63 Fix NPE in TransportGitSsh.ExtSession.exec()
Bug: 527383
Change-Id: I965860b4e0e9aa1adf6a1bb0735940cbd22adced
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-11-20 00:35:37 -05:00