Commit Graph

5756 Commits

Author SHA1 Message Date
David Pursehouse 5ef028ff4a IgnoreNodeTest: Suppress deprecation warnings in testEmptyIgnoreNode
Change-Id: I930c9408cc702af911419f2a7b3d03f652a2d4e3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-25 17:36:38 +09:00
David Pursehouse c9e3921fbb Fix warnings about boxing/unboxing Boolean
Change-Id: I9d81d510282e9181267750fe3f9c571c35b61407
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-25 17:35:12 +09:00
David Pursehouse e8c69fa5fd WorkingTreeIterator: Fix warnings about variable hiding
Change-Id: I78ed3ae7aa30a7e7f146d0bd2e9feff74554da78
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-25 17:32:33 +09:00
David Pursehouse 6ea21b454c WorkingTreeIterator: Add NON-NLS suppressions
Change-Id: I369b8f68912134fc3880c162e9a2c5a1669bb4ac
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-25 17:30:16 +09:00
Matthias Sohn 3e3554c9eb Merge "Fix processing of gitignore negations" 2018-02-23 19:42:03 -05:00
Marc Strapetz 78420b7d0a Fix processing of gitignore negations
Processing of negated rules, like !bin/ was not working correctly: they
were interpreted too broad, resulting in unexpected untracked files
which should actually be ignored

Bug: 409664
Change-Id: I0a422fd6607941461bf2175c9105a0311612efa0
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2018-02-23 13:36:48 +01:00
Michael Keppler beacbe3eee Update tycho to 1.1.0
The upgrade was already done 3 weeks ago, this version number was
missing, however.

Change-Id: Ibc794ba8035c46ea2160925b344ded5714b7c987
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2018-02-23 09:14:05 +01:00
Matthias Sohn 152d5e2a14 Configure Semaphore used in TestRequestLog to be fair
Bug: 528187
Change-Id: I26c94ad7f802fc1dbe72d63a7ea346dc6e19ff94
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-22 23:16:42 +01:00
Matthias Sohn 520f3af7b4 Fix indentation in TestRequestLog
We use tabs to indent.

Change-Id: I19b00e06becfb92fbabb99b5699385506fd85214
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-22 23:16:42 +01:00
Dave Borowitz 879372eb72 DfsReaderIoStats: Clarify that "blocks" are really cache misses
The previous wording was ambiguous as to whether these were blocks
requested from the cache (hits + misses) or read from underlying storage
(misses only).

They are in fact recording only misses:
Accumulator#{readBlock,readBlockBytes,readBlockMicros} are only
incremented from BlockBasedFile#readOneBlock, which is only called from
the cache miss path in DfsBlockCache#getOrLoad (line 391).

Change-Id: I0135cd1e76d09c1e28e0f1833b34c312511c66ce
2018-02-22 16:27:09 +09:00
Marc Strapetz 49cb6ba5dd PathMatcher: fix handling of **/
**/ should match only directories, but not files

Change-Id: I885c83e5912cac5bff338ba657faf6bb9ec94064
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2018-02-22 14:39:23 +09:00
David Pursehouse 0e20df710a Merge changes Id616611e,Ib6509e81,I52f5d3f2
* changes:
  RepoCommand: generate relative submodule URLs from absolute URLs.
  RepoCommand: don't record new commit if tree did not change
  RepoCommand: persist unreadable submodules in .gitmodules
2018-02-22 00:37:48 -05:00
Marc Strapetz 372e04dcf3 CGitIgnoreTest: also test untracked files
Change-Id: I21a4ebd63eaaa85aa2e68f99ef58c141189bdab4
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
2018-02-21 14:50:50 -05:00
Shawn Pearce d0fbaf502e dfs: Try to avoid searching UNREACHABLE_GARBAGE during packing
If an object can be found in a non-garbage pack, favor that pack over
paging in the garbage pack's idx and pack content.

Only fall back to garbage packs if an object cannot be found and there
are garbage packs present in the repository.  This fallback is
required to correct race conditions during GC.

Change-Id: Ia7c123975bc069b8e6e713eda2d357303b71e329
2018-02-21 14:46:57 -05:00
Han-Wen Nienhuys 6a420613f7 RepoCommand: generate relative submodule URLs from absolute URLs.
If a manifest file specifies an absolute URL on the same host on which
the superproject resides, rewrite the URLs to be relative.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id616611e5195998fb665c8e7806539a3a02e219a
2018-02-21 13:41:50 +01:00
Han-Wen Nienhuys 3214171dec RepoCommand: don't record new commit if tree did not change
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Ib6509e816684256d723558d5e8f3c37de58a2ff8
2018-02-19 20:24:07 +09:00
Han-Wen Nienhuys 446a7096ef RepoCommand: persist unreadable submodules in .gitmodules
In cases where a manifest file mixes different remotes, a Gerrit
server process may not have access to all remotes, and won't be able
to produce a full submodule tree.

Preserving this information in .gitmodules will let downstream clients
reconstruct the full tree.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I52f5d3f288e771dca0af2b4dd3f3fa0f940dcf15
2018-02-19 20:24:07 +09:00
Han-Wen Nienhuys 35bb7ccc4c Declare LFS dependency on GSON for Bazel build
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Ia017dd5651756c3b80f18b908fb28ba56b6cbf14
2018-02-19 19:06:38 +09:00
Matthias Sohn 44f0a01f24 Silence API error for new constant in ConfigConstants
Change-Id: Ic4bd10b67caf6b2a2cc0c23bb4e7c717e4ccd6d5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-18 22:23:00 +01:00
Matthias Sohn 33d5c0fe4a Silence API errors raised for new public fields in PackStatistics
Change-Id: I8e118282a833361b676aa3cc90cbe774ccd42ed0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-18 22:08:47 +01:00
Markus Duft 94bcde663c LFS: Add remote download to SmudgeFilter
Transfer data in chunks of 8k Transferring data byte per byte is slow,
running checkout with CleanFilter on a 2.9MB file takes 20 seconds.
Using a buffer of 8k shrinks this time to 70ms.

Also register the filter commands in a way that the native GIT LFS can
be used alongside with JGit.

Implements auto-discovery of LFS server URL when cloning from a Gerrit
LFS server.

Change-Id: I452a5aa177dcb346d92af08b27c2e35200f246fd
Also-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
2018-02-16 18:27:25 +01:00
David Pursehouse 9bebb1eae7 RepoCommand: Don't leave Git open
When the command is run on a non-bare repository, an instance of
Git is created to execute the commit, and is left open when the
command has finished.

Refactor to not use a class scope Git instance, and make sure it
gets closed before returning.

Change-Id: Ic623ae0fd8b9e264b5dfd434da0de6bb4f910984
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-15 16:59:42 +09:00
Matthias Sohn 282dc592da Merge "Merge branch 'stable-4.10'" 2018-02-14 18:18:43 -05:00
David Pursehouse 184ddc5afa Merge "InitCommand: Don't leave Repository open after Git is closed" 2018-02-14 18:18:35 -05:00
Matthias Sohn d2d22e1471 Merge branch 'stable-4.10'
* stable-4.10:
  Fix ssh host name handling for Jsch
  Jsch overrides the port in the URI with the one in ~/.ssh/config

Change-Id: I860fc61ceb12ae792b1ee7421046ecd32373b9f8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-14 23:28:28 +01:00
Matthias Sohn 95fc2b210f Don't subclass ThreadLocal to avoid memory leak in NLS
These problem usually occur when you subclass ThreadLocal (usually to
implement initialValue). Those classes reference the webapp's
classloader. The ThreadLocal subclass in turn is referenced by each
Thread instance (that's how ThreadLocals are implemented, they have a
"helper-Map" in each Thread instance, so the leak is actually not a tiny
Random instance but the whole webapp's classloader with a bunch of class
definitions and statically referenced parts of the webapp.

Bug: 449321
Change-Id: Ie7a8b0b90e40229e2471202f2a12637b9e0b1d11
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-14 21:39:25 +01:00
Matthias Sohn 95f868f246 Set context classloader to null in WorkQueue
If threads created by the executor have a reference to the context
classloader this may cause a leak.

See: https://wiki.apache.org/tomcat/MemoryLeakProtection#cclThreadSpawnedByWebApp
Bug: 517107
Change-Id: If79c1ebd3f9cf733fd88a5fce38fd46ff0d08d10
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-14 21:39:24 +01:00
Matthias Sohn 6e0f8bacd4 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  Fix ssh host name handling for Jsch
  Jsch overrides the port in the URI with the one in ~/.ssh/config

Change-Id: Iff9076f65e767bbe8df016337b631bdaeb40ad98
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-14 21:25:16 +01:00
Matthias Sohn 105bfd9b56 Merge "TransportLocal should reuse FS from source repo" 2018-02-14 15:02:22 -05:00
Thomas Wolf 185e53bce4 Fix ssh host name handling for Jsch
If we give Jsch access to the ssh config file, we must _not_ resolve
the host name from the alias. Instead we must give the alias (i.e.,
the host name as is in the URI) to Jsch, so that it finds the same
ssh config entry.

Otherwise if the hostname in the URI, which is taken as an alias in
ssh config ("Host" line), is unequal to the "Hostname" line, and
there happens to be another ssh config entry with that translated
host name as alias, Jsch will pick up that second entry, and we end
up with a strange mixture of both.

Add tests for this case.

Bug: 531118
Change-Id: I249d8c073b0190ed110a69dca5b9be2a749822c3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-02-14 13:39:28 +01:00
Thomas Wolf 891103609d Jsch overrides the port in the URI with the one in ~/.ssh/config
Jsch unconditionally overwrites the port from the ssh config
file (if a port is specified there), even if the URI explicitly does
give a different port.

Fix this, and add tests.

Change-Id: I7b014543c7ece26270e366db39d7647f82d64f0d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2018-02-14 13:32:37 +01:00
David Pursehouse f527d4548f InitCommand: Don't leave Repository open after Git is closed
The InitCommand returns a Git that is instantiated with the newly
created Repository, but the Repository is not closed with the Git
resulting in resource leaks.

Create the Git with `closeRepo` set to true, such that the Repository
is also closed when the Git is closed.

Adjust the tests to use try-with-resource on the Git instance.

Change-Id: Ib26e7428c7d8840956d1edb09e53b93e23e6fe5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2018-02-14 14:09:40 +09:00
Matthias Sohn 19355ce123 URIish: add a test for equals and hashcode methods
Change-Id: I5727ac1757dca0d63631401bcb074ee8f58ce082
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-13 19:00:40 -05:00
Terry Parker 9530c10192 Add a minimum negotiation feature for fetch
Android an Chrome have several repos with >300k refs. We sometimes see
negotiations of >100k rounds. This change provides a "minimal negotiation"
feature on the client side that limits how many "have" lines the client
sends. The client extracts the current SHA-1 values for the refs in its
wants set, and terminates negotiation early when all of those values have
been sent as haves. If a new branch is being fetched then that set will
be empty and the client will terminate after current default minimum
of two rounds.

This feature is gated behind a "fetch.useminimalnegotiation" configuration
flag, which defaults to false.

Change-Id: Ib12b095cac76a59da6e8f72773c4129e3b32ff2b
Signed-off-by: Terry Parker <tparker@google.com>
2018-02-12 14:03:11 -08:00
Terry Parker 302596cc67 Add negotiation statistics to PackStatistics
Add fetch statistics for the counts of advertised refs, wants and haves.
Also add the duration in milliseconds for the negotiation phase. For
non-bidirectional transports like HTTP, this is the time for the final
round that sends the pack back to the user.

Change-Id: I1af7ffd3cb7b62182340682e2a243691ea24ec2e
Signed-off-by: Terry Parker <tparker@google.com>
2018-02-12 13:59:42 -08:00
Minh Thai 5efc3dcc6c MergedReftable to skip shadowed refs in same reftable
This would allow compact and GC process to clean up duplicate ref names in the reftables.

Change-Id: I2b9df0bf72dba63cc3525e374982e60559a776c2
Signed-off-by: Minh Thai <mthai@google.com>
2018-02-09 09:49:24 -08:00
Christian Halstrick 4d5231f001 Fix CleanCommand not to throw FileNotFoundExceptions
When CleanCommand is collecting the files and folders to be deleted
it may happen that the list of directories contains obsolete entries.
E.g. a folder and its parent folder may be in the list. Only the
parent folder would be sufficient.

This was a reason for hitting FileNotFoundExceptions when finally
trying to delete the files and folders. Improve CleanCommand
to ignore files to be deleted which are already gone.

Bug: 514434
Change-Id: I10caa01bfb9cec5967dfdaea50c6e4a713eeeabd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-08 10:27:21 +01:00
Hector Caballero 93654f75a2 GC: Remove empty references folders
After packaging references, the folders containing these references are
not deleted. In a busy repository, this causes operations to slow down
as traversing the references tree becomes longer.

Delete empty reference folders after the loose references have been
packed.
To avoid deleting a folder that was just created by another concurrent
operation, only delete folders that were not modified in the last 30
seconds.

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ie79447d6121271cf5e25171be377ea396c7028e0
2018-02-07 18:37:12 -05:00
David Ostrovsky ab1b97234d Bazel: Support building with Java 9
Bazel@HEAD supports Java 9.

The current code has one single issue with Java 9 compliance: the usage
of javax.xml.bind.DatatypeConverter class for printHexBinary() method.
This class is not available on Java 9. One alternative is to use guava
library. Something similar was done here: [1]. But unlike the case with
checkstyle library, JGit currently doesn't use guava. Instead, we add
java.xml.bind module with --add-modules compiler option.

To build (or test) with Java 9, build custom bazel version and issue:

  $ bazel --host_javabase=/usr/lib64/jvm/java-9-openjdk build \
    --javacopt='--release 9' \
    --java_toolchain=@bazel_tools//tools/jdk:toolchain_jdk9 :all

The Java 9 support is backwards compatible.

* [1] https://github.com/checkstyle/checkstyle/issues/5027

Change-Id: I2c5203fc4e65885ce7b210f824fda85ba6d6c51d
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2018-02-07 17:51:35 -05:00
Hector Caballero 6766fc85d0 GC: Merge if statement with the enclosing one
Reported by Sonar.

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ib6fe6c67d6f7697e6d60693d5cee7e9c6a227767
2018-02-06 08:41:41 -05:00
Hector Caballero 4644d15bc5 GC: Replace Files methods with File alternatives
The Files.exists method has noticeably poor performance in JDK 8 and can
slow an application significantly when used to check files that do not
actually exist. The same goes for Files.notExists, Files.isDirectory and
Files.isRegularFile [1].

Replace them with their File counterpart.

[1] https://rules.sonarsource.com/java/tag/performance/RSPEC-3725

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: I89d23b9cc74bec8e05f6b7f3e49bfd967dbb6373
2018-02-06 08:39:43 -05:00
Hector Caballero 66127d4284 GC: Replace anonymous classes with lambdas
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ic9725bdb1a281c80055a5438a02d961b1d1875f4
2018-02-06 06:29:29 -05:00
Hector Caballero 54053389ef GC: Simplify lambda
filter() stream operation also accepts a lambda expression.

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ie0b7fe43d39593172969fd16c3e44b49bac27306
2018-02-06 06:28:10 -05:00
Hector Caballero e00a12f5a2 GC: Simplify reference equals() method
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: I5b2e12e012d5423e9b84c1c66a478960121ea212
2018-02-06 06:26:14 -05:00
Hector Caballero f58c3fce90 GC: Simplify boolean expression
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: I0e7a365c26b2333627fcc26b9ea9f4fcd65e8746
2018-02-06 06:25:57 -05:00
Hector Caballero 6ca4c79063 GC: Simplify expression
Avoid converting path to file to then reconvert it to path.

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: I6a8c3ca9b83bf9b0eead9506938f5d68b27a76f5
2018-02-06 06:25:32 -05:00
Hector Caballero 049417d504 GC: Remove redundant type arguments
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Id3281a744b08f9d7b2b60f69b4e461b053ac5d02
2018-02-06 06:25:17 -05:00
Hector Caballero e3b73d889c GC: Remove unnecessary continue statement
continue is unnecessary when it is the last statement in a loop

Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: I12af9f9a0bb2fd7fc0239f1f3b59fb8e64e1f351
2018-02-06 06:24:40 -05:00
Matthias Sohn f7244b81cc Close RevWalk in BranchTrackingStatus.of()
Bug: 530757
Change-Id: I985ee5395e1468981c28ae0a80797732e2f528d2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-06 01:10:26 +01:00
Matthias Sohn dce5bc2498 Update tycho to 1.1.0
Change-Id: I8a72c59c1453a9252e56d68c8a162341c690622f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-02-05 00:44:33 +01:00