Commit Graph

4203 Commits

Author SHA1 Message Date
David Pursehouse b0ac5f9c89 LargePackedWholeObject: Refactor to open DfsReader in try-with-resource
Change-Id: Ia9557e6c1ab230dbe2e94e025a49e93159d8658c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-04-11 03:10:36 -04:00
Markus Duft aaf4b35557 Don't throw if a pre-push hook is ignored.
This breaks any scenario where native git (with LFS) clones a repository
(and thus installs the hook) and later on JGit is used to push changes.

Change-Id: I2a17753377265a0b612ba3451b9df63a577a1c38
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
2018-04-10 21:06:27 -04:00
Thomas Wolf a5db080af0 Silence API error for ObjectIdSerializer.readWithoutMarker()
Change-Id: I99111797253b04148c42c83e1ef3d2cc383cc627
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-03-27 22:15:42 +02:00
Matthias Sohn 10d03a77a4 Merge branch 'stable-4.11'
* stable-4.11:
  Remove package import for javax.servlet.http from org.eclipse.jgit
  Add missing @since tag and silence API error

Change-Id: I2783a15ead26ab19de31a8fb3bfb148ef19de91a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-26 16:35:30 -07:00
Matthias Sohn a263bb1173 Remove package import for javax.servlet.http from org.eclipse.jgit
This package import is unused.

Change-Id: I9f202bb3162736a1ef8054516f1a3145d3a7bb9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-27 00:36:20 +02:00
Matthias Sohn 4de2a9d57d Add missing @since tag and silence API error
These methods were introduced for 4.11.1 so we have to silence the API
error adding API in a service release raises.

Change-Id: Ic847cebbed439912d3979ec2ec1809f77a28f61e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-27 00:35:40 +02:00
Thomas Wolf 4bfc6c2ae9 Significantly speed up FileTreeIterator on Windows
Getting attributes of files on Windows is an expensive operation.
Windows stores file attributes in the directory, so they are
basically available "for free" when a directory is listed. The
implementation of Java's Files.walkFileTree() takes advantage of
that (at least in the OpenJDK implementation for Windows) and
provides the attributes from the directory to a FileVisitor.

Using Files.walkFileTree() with a maximum depth of 1 is thus a
good approach on Windows to get both the file names and the
attributes in one go.

In my tests, this gives a significant speed-up of FileTreeIterator
over the "normal" way: using File.listFiles() and then reading the
attributes of each file individually. The speed-up is hard to
quantify exactly, but in my tests I've observed consistently 30-40%
for staging 500 files one after another, each individually, and up
to 50% for individual TreeWalks with a FileTreeIterator.

On Unix, this technique is detrimental. Unix stores file attributes
differently, and getting attributes of individual files is not costly.
On Unix, the old way of doing a listFiles() and getting individual
attributes (both native operations) is about three times faster than
using walkFileTree, which is implemented in Java.

Therefore, move the operation to FS/FS_Win32 and call it from
FileTreeIterator, so that we can have different implementations
depending on the file system.

A little performance test program is included as a JUnit test (to be
run manually).

While this does speed up things on Windows, it doesn't solve the basic
problem of bug 532300: the iterator always gets the full directory
listing and the attributes of all files, and the more files there are
the longer that takes.

Bug: 532300
Change-Id: Ic5facb871c725256c2324b0d97b95e6efc33282a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-03-25 13:43:37 +02:00
David Pursehouse 0bc2020412 TransportHttp: Refactor to use try-with-resource and suppress resource warning
Change-Id: I130269e7c5e46aea2152dea6b02539529208eea2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-25 09:49:43 +01:00
Matthias Sohn d8234d310d Add missing @since tags for new API
These methods were added after 4.11 so strictly speaking they violate
semantic versioning since new API requires increasing the minor version
number. Hence pretend these methods were introduced in 5.0

Change-Id: I7793ead16577dc1f2ddea09ba6b055103c783555
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-25 00:56:53 +01:00
David Pursehouse 66165fef48 Remove unused API problem filters
Change-Id: I414ebba1db831686fd378dadcb7c4882eb323115
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 19:55:08 -04:00
David Pursehouse 84a6318eae DfsInserter#openStream: Suppress resource warning about DfsReader
DfsReader is not opened in a try-with-resource because in the case where
the method returns an ObjectStream.Filter, the DfsReader should only be
closed from within the Filter's close() method.

Suppress the resource warning.

Change-Id: Ifcaf5e4a326bd1d03c6331b476c645ca43943b34
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:32:59 +01:00
David Pursehouse b3a4ace695 CloneCommand: Suppress resource warning about Repository
Repository is not opened in try-with-resource because it is wrapped
in a Git instance which should be closed by the caller. On exeptions
during fetch, it is explicitly closed in the catch blocks.

Suppress the warning with an explanatory comment.

Change-Id: Ib32c74ce39bb810077ab84db33002bdde806f3b6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:15:45 +01:00
David Pursehouse a77ffdf080 SubmoduleSyncCommand: Refactor to open Repository in try-with-resource
Change-Id: I502904ff7dbe074f7bbcb2a56a17bf4729f4f4d3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:15:44 +01:00
David Pursehouse 63b3340126 SubmoduleUpdateCommand: Refactor to open Repository in try-with-resource
Change-Id: I1a303fdfdb6823043fa6751c43eaeaf678f2e64f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:15:43 +01:00
David Pursehouse 153f7258f9 TransportGitAnon: Suppress resource warning about Socket
Change-Id: Id1f5c0db4273324748a0ead2e6dac58d9114cf86
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:15:43 +01:00
David Pursehouse 76b4ed6a85 FS#runProcess: Fix OutputStream left unclosed after IOException
The runProcess method creates an OutputStream that is not managed by
a try-with-resource because it's manually closed and any IOException
raised by the close() method is explicitly ignored.

Suppress the resource warning with an explanatory comment.

Enclose the call to StreamGobbler#copy in an inner try-block, and move
the call to close() inside its finally block. This prevents the stream
from being left unclosed if StreamGobbler#copy raises IOException.

Change-Id: Idca9adfc4d87e0989d787ad8239c055c0c849814
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-24 22:15:43 +01:00
Jonathan Nieder 7719a406bf Merge changes from topic 'If72b4b422-cleanup'
* changes:
  UploadPackTest: Fix name hiding introduced by If72b4b422
  Suppress non-localized string warnings introduced by If72b4b422
2018-03-22 21:01:31 -04:00
Minh Thai bf8057058e scanPacks to return reftables even if no packs
An empty repository may have a dangling symref HEAD pointing to
refs/heads/master.  In this case, there will be a reftable even though
there are no packs yet.

Change-Id: Ib759ffbbfc490953481853e74263dd46d2592888
Signed-off-by: Minh Thai <mthai@google.com>
2018-03-22 17:11:45 -07:00
David Pursehouse 5b1c40fe90 Suppress non-localized string warnings introduced by If72b4b422
Change-Id: I2e4544d715b4284f544a26f504d01159c5843160
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-20 08:28:14 +09:00
David Pursehouse 901e663a66 PackWriter: Fix Eclipse errors about missing Javadoc
Change If72b4b422 added a new method filterAndAddObject with a
partial Javadoc, which causes errors in Eclipse.

Since it's a private method, Javadoc is not strictly necessary, so
just convert it to a standard comment block.

Bug: 532540
Change-Id: I06aa79211d1223dccf6c931451ca885ca6d39cbc
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-20 08:28:03 +09:00
David Pursehouse 7f69c7e93d ReflogWriter: Refactor to open FileOutputStream in try-with-resource
Change-Id: I028ced10eecc99214a4c4a8055c379af72193f13
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-16 12:06:52 +09:00
David Pursehouse 553da7540e PushCommand: Suppress resource warning for Transport in for loop
A list of Transport instances is provided by Transport.openAll, and
then iterated over in a for loop. Eclipse warns that the Transport
in the for-loop should be managed by try-with-resource.

The Transport is explicitly closed in the finally block, so just
suppress the warning.

Change-Id: I92b73cd90902637cf1ac1ab7b02b5ee5ed6bdb1e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-16 12:06:52 +09:00
David Pursehouse 7929653ff1 Merge branch 'stable-4.11'
* stable-4.11:
  ObjectIdSerializer: Support serialization of known non-null ObjectId

Change-Id: Ie430fa2c5d13ae698d884a37d0d03884ebbf25ec
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-16 11:59:41 +09:00
Jonathan Tan 4ac32e79b7 Teach UploadPack to support filtering by blob size
Teach UploadPack to advertise the filter capability and support a
"filter" line in the request, accepting blob sizes only, if the
configuration variable "uploadpack.allowfilter" is true. This feature is
currently in the "master" branch of Git, and as of the time of writing,
this feature is to be released in Git 2.17.

This is incomplete in that the filter-by-sparse-specification feature
also supported by Git is not included in this patch.

If a JGit server were to be patched with this commit, and a repository
on that server configured with RequestPolicy.ANY or
RequestPolicy.REACHABLE_COMMIT_TIP, a Git client built from the "master"
branch would be able to perform a partial clone.

Change-Id: If72b4b422c06ab432137e9e5272d353b14b73259
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2018-03-15 16:46:42 -04:00
David Pursehouse c6a2c58624 SubmoduleStatusCommand: Open Repository in try-with-resource
Change-Id: Id65c8a94d43bc01e76733eb2ff87635d0f8a8dc2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 10:28:14 +09:00
David Pursehouse 11ddaff6af RepositoryCache#registerRepository: Refactor to use try-with-resource
Change-Id: Iaad45b66cc10ac267f6aed7999cc8dc8c07f92e6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 10:28:14 +09:00
David Pursehouse 61e4f16652 ObjectIdSerializer: Support serialization of known non-null ObjectId
The implementation of ObjectIdSerializer, added in change I7599cf8bd,
is not equivalent to the original implementation in Gerrit [1].

The Gerrit implementation provides separate methods to (de)serialize
instances of ObjectId that are known to be non-null. In these methods,
no "marker" is written to the stream. Replacing Gerrit's implementation
with ObjectIdSerializer [2] broke persistent caches because it started
writing markers where they were not expected [3].

Since ObjectIdSerializer is included in JGit 4.11 we can't change the
existing #write and #read methods. Keep those as-is, but extend the
Javadoc to clarify that they support possibly null ObjectId instances.

Add new methods #writeWithoutMarker and #readWithoutMarker to support
the cases where the ObjectId is known to be non-null and the marker
should not be written to the serialization stream.

Also:

- Replace the hard-coded `0` and `1` markers with constants that can
  be linked from the Javadocs.

- Include the marker value in the "Invalid flag before ObjectId"
  exception message.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/9792
[2] https://gerrit-review.googlesource.com/c/gerrit/+/165851
[3] https://gerrit-review.googlesource.com/c/gerrit/+/165952

Change-Id: Iaf84c3ec32ecf83efffb306fdb4940cc85740f3f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 09:39:43 +09:00
David Pursehouse a9bbbc0965 DfsPackCompactor: Open PackWriter in try-with-resource
Change-Id: I2c7b3fdf6a51471c35434a6176865d622b13653d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 07:57:02 +09:00
David Pursehouse 280b2bfe52 DfsPackFile: Refactor getBitmapIndex to open ReadableChannel in try-with-resource
Refactor getBitmapIndex to open ReadableChannel in try-with-resource
instead of closing the channel in the finally block.

The same cannot be done in copyPackThroughCache, so just suppress the
warning with an explanatory comment.

Change-Id: I9b95373d350728e85a159423d5ca80e8b215914d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 07:57:02 +09:00
David Pursehouse 5639639b06 IndexDiff: Refactor to open Repository in try-with-resource
Change-Id: I667d685ccedc730e786f1e52323fdeba4b91af3e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 07:57:02 +09:00
David Pursehouse 7bd2a4a7c6 DirCache: Suppress resource warning related to TemporaryBuffer
In #writeTo, the TemporaryBuffer can't be opened in try-with-resource
because it's referenced in the finally block. Instead it is explicitly
closed within the try block. Suppress the warning with an explanatory
comment.

Change-Id: I02009f77f9630d5d55afc34eb86d304ff103b8b0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 07:57:02 +09:00
David Pursehouse 3e1066d0a4 Add SilentFileInputStream to allow ignoring exceptions raised by close()
There are several cases where a FileInputStream is opened outside of
a try-with-resource because we want to explicitly close it and ignore
any IOException that is raised by the close() method.

Introduce a helper class, SilentFileInputStream, that overrides the
close method and ignores the exceptions. This allows to open the stream
in a try-with-resource block and remove the explicit handling of the
close method.

Change-Id: I8612f948a1a5b3d1031344922ad75ce4492cfc61
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-15 06:39:57 +09:00
David Pursehouse aa563091d5 WorkingTreeIterator#idSubmodule: Refactor to open Repository in try-with-resource
Change-Id: I991f0096c833da721b98c1e0423a8dadc67cd64f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse 67df4986ce UploadPack: Suppress resource warning about PackWriter
PackWriter is auto-closeable and should be opened in try-with-resource,
but this is not possible since the variable is being referenced in the
finally block before being explicitly closed there.

Suppress the warning and add an explanatory comment.

Change-Id: I161923f35142132234fd951c0146d3cb30920b7b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse 6344e7a071 SubmoduleWalk#forIndex: Suppress resource warning and update Javadoc
SubmoduleWalk is auto-closeable, and Eclipse warns that is is not
managed by try-with-resource. However in this case the resource should
not be closed, because the caller needs to use it. Instead, it is the
responsibility of the caller to close it after use.

Update the Javadoc to clarify this, and suppress the warning.

Change-Id: Ib7ba349353bfd3341bdcbe4bb19abaeb9f3aeba5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse 4775a394c0 SubmoduleWalk: Open Repository in try-with-resource
Change-Id: Ib29aaf26b49aa94a5a7cc0b0264a0a93ecff0c16
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse fe6c97bc4c NetRC: Open BufferedReader in try-with-resource
Change-Id: Ie0688b0ce19f657834a23438f112569a86491b3b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse 3e437da6ba PackParser: Refactor to open InputStream in try-with-resource
Change-Id: I8d002ccc8f168f5891492a4c5742c82f8cb7a0b6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse bac3efbe67 DfsInserter: Refactor writePackIndex to use try-with-resource
Refactor to allow the temporary buffer to be opened in try-with-resource.

Change-Id: Id913e6c3ed3913fd5d79d66238b779e0c225b47d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-14 10:44:23 +09:00
David Pursehouse 6eb4cd276c AmazonS3: Refactor error method to avoid 'should be managed by try-with-resource' warning
Change-Id: I205fc1c77777870b0a036e52fa9954de5d9f60b5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-13 22:21:36 +01:00
David Pursehouse 7392d3b304 Transport: Simplify scan method using try-with-resource
The IOExceptions caught in the nested try blocks are all ignored,
so we can just wrap them all up into a single try-with-resource
block.

Change-Id: I536d682f1017c5088b94ff9f98ffa2b7c783d8bf
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-13 22:21:35 +01:00
David Pursehouse 5c70be0085 Open auto-closeable resources in try-with-resource
When an auto-closeable resources is not opened in try-with-resource,
the warning "should be managed by try-with-resource" is emitted by
Eclipse.

Fix the ones that can be silenced simply by moving the declaration of
the variable into a try-with-resource.

In cases where we explicitly call the close() method, for example in
tests where we are testing specific behavior caused by the close(),
suppress the warning.

Leave the ones that will require more significant refcactoring to fix.
They can be done in separate commits that can be reviewed and tested
in isolation.

Change-Id: I9682cd20fb15167d3c7f9027cecdc82bc50b83c4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-13 22:16:06 +01:00
Matthias Sohn e23b09ad6e Implement --force option in FetchCommand and CLI fetch command
Change-Id: I42cdb57b8fb54ce466d1958391f12f911045327f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-13 02:20:29 +01:00
Matthias Sohn 6f82de449f Merge "Introduce SshSupport to centralize SSH related utility code" 2018-03-12 19:52:43 -04:00
Markus Duft 4ca9bb5ab9 Introduce SshSupport to centralize SSH related utility code
As discussed with Thomas here:
https://git.eclipse.org/r/#/c/83506/31/org.eclipse.jgit.lfs/src/org/eclipse/jgit/lfs/SmudgeFilter.java@349

Move the code from ConfigureGerritAfterCloneTask to SshSupport and
eliminate the slightly modified copy of the code from
LfsConnectionFactory. Separate EGit commit will eliminate the code from
ConfigureGerritAfterCloneTask.

Change-Id: Ifb5adb1342e0fc1f2a70cddf693408d4e0ef7906
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
2018-03-12 07:53:35 +01:00
David Pursehouse d88d5b94c6 UploadPack: Remove redundant suppression of deprecation warning
Change-Id: I9a7f46e1da42fd86adedc18b3394d5f755722624
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-12 10:52:03 +09:00
David Pursehouse f07b60239c Consistently use Constants.CHARSET rather than StandardCharsets.UTF_8
Change-Id: I6714fc3666e1bced22abba94ceb700477349586e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-11 22:46:52 +01:00
David Pursehouse 62bc48df60 Constants: Initialize CHARSET and CHARACTER_ENCODING from StandardCharsets.UTF_8
Change-Id: Ib4e0a35429892fa1581db09a9b77717120d03275
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-03-11 22:28:30 +01:00
Matthias Sohn bfe89332a4 Remove deprecated StreamCopyThread#flush
StreamCopyThread: Do not let flush interrupt a write.
flush calls interrupt() to interrupt a pending read and trigger a
flush.  Unfortunately that interrupt() call can also interrupt a
pending write, putting Jsch in a bad state and triggering "Short read
of block" errors.

Change-Id: I11f8a014fd72df06617cc8731d992eb14cc32a67
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-10 19:41:31 +01:00
Matthias Sohn 50436cc829 Remove deprecated SafeBufferedOutputStream
Use Java 8 BufferedOutputStream instead. Java 8 fixed the silent flush
during close issue by FilterOutputStream (base class of
BufferedOutputStream) using try-with-resources to close the stream,
getting a behavior matching what JGit's SafeBufferedOutputStream
was doing

Change-Id: Ieeab59f49b44519585abda213d287b19c7863b17
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-03-10 19:41:30 +01:00