Commit Graph

8583 Commits

Author SHA1 Message Date
Matthias Sohn 035e0e23f2 UploadPack: don't prematurely terminate timer in case of error
In uploadWithExceptionPropagation don't prematurely terminate timer in
case of error to enable reporting it to the client. Expose a close
method so that callers can terminate it at the appropriate time.

If the timer is already terminated when trying to report it to the
client this failed with the error java.lang.IllegalStateException:
"Timer already terminated".

Bug: 579670
Change-Id: I95827442ccb0f9b1ede83630cf7c51cf619c399a
2022-06-30 14:45:31 +02:00
Matthias Sohn ca6b518432 Merge "Do not create reflog for remote tracking branches during clone" into stable-5.13 2022-06-26 15:36:03 -04:00
Luca Milanesio 4bb4693633 Do not create reflog for remote tracking branches during clone
When using JGit on a non-bare repository, the CloneCommand
it previously created local reflogs for all branches including remote
tracking ones, causing the generation of a potentially large
number of files on the local filesystem.

The creation of the remote-tracking branches (refs/remotes/*) during
clone is not an issue for the local filesystem because all of them are
stored in a single packed-refs file. However, the creation of a large
number of ref logs on a local filesystem IS an issue because it
may not be tuned or initialised in term of inodes to contain a very
large number of files.

When a user (or a CI system) performs the CloneCommand against
a potentially large repository (e.g., millions of branches), it is
interested in working or validating a single branch or tag and is
unlikely to work with all the remote-tracking branches.
The eager creation of a reflogs for all the remote-tracking branches is
not just a performance issue but may also compromise the ability to
use JGit for cloning a large repository.

The behaviour implemented in this change is also consistent with the
optimisation done in the C code-base [1].

We differentiate between clone and fetch commands using --branch
<initialBranch> option, that is only available in clone command,
and is set as HEAD per default.

[1] 58f233ce1e

Bug: 579805
Change-Id: I58d0d36a8a4ce42e0f59b8bf063747c4b81bd859
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2022-06-25 12:09:01 +01:00
Luca Milanesio 66ace4b9af UploadPack: do not check reachability of visible SHA1s
When JGit needs to serve a Git client requesting SHA1s
during the want phase, it needs to make a full reachability
check from the advertised refs to the ones requested to
keep all objects in the correct scope of confidentiality
allowed by the avertised refs.

The check is also performed when the SHA1 corresponds to
one of the tips of the advertised refs which is a waste of
resources.

Example:

fetch> ref-prefix refs/heads/foo
fetch< 900505eb8ce8ced2a1757906da1b25c357b9654e refs/heads/foo
fetch< 0000
fetch> command=fetch
fetch> 0001
fetch> thin-pack
fetch> ofs-delta
fetch> want 900505eb8ce8ced2a1757906da1b25c357b9654e

The SHA1 in the want is the tip of refs/heads/foo and therefore
the full reachability check can be shortened and resolved more
quickly.

Change-Id: I49bd9e2464e0bd3bca2abf14c6e9df550d07383b
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2022-06-25 06:45:05 -04:00
Matthias Sohn f4cbf31ae4 Add missing package import javax.management to org.eclipse.jgit
Class org.eclipse.jgit.util.Monitoring uses JMX hence we need this
import otherwise OSGi applications can face ClassNotFoundException.

Bug: 577018
Change-Id: Ifd75337b87c7faec95d333b771bb0a2f3e46a418
2022-06-17 13:49:59 +02:00
Matthias Sohn db074a1352 Prepare 5.13.2-SNAPSHOT builds
Change-Id: I4862e5d80a7d95a1a119d06306e3f6927445d1d3
2022-06-14 00:41:18 +02:00
Matthias Sohn b34961a493 JGit v5.13.1.202206130422-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ife74d64e8171c68dbf08271492c0ac852a6dc51c
2022-06-13 10:22:43 +02:00
eric.steele e9a5430c25 AmazonS3: Add support for AWS API signature version 4
Updating the AmazonS3 class to support AWS Signature version 4 because
version 2 is no longer supported in all AWS regions. The version can be
selected with the new 'aws.api.signature.version' property (defaults to
2 for backwards compatibility). When set to '4', the user must also
specify the AWS region via the 'region' property. The 'region' property
must match the region that the 'domain' property resolves to.

Bug: 579907
Change-Id: If289dbc6d0f57323cfeaac2624c4eb5028f78d13
2022-06-13 09:44:23 +02:00
Matthias Sohn 5efd32e91d Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Fix connection leak for smart http connections

Change-Id: Id34f29c1b27a80c2b56c911cad7e3f64ef63af48
2022-06-07 11:34:25 +02:00
Matthias Sohn 8bb17e518f Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Fix connection leak for smart http connections

Change-Id: I6caabf4774ccf34706cef846c1087710f67e2ecd
2022-06-07 11:33:38 +02:00
Matthias Sohn c7335f32e9 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Fix connection leak for smart http connections

Change-Id: I3885c6114caed897f762f5ce523d3b27288205b2
2022-06-07 10:53:24 +02:00
Matthias Sohn 85011b8b07 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Fix connection leak for smart http connections

Change-Id: I5e7144b2f5cd850978220c476947001ae2debb8e
2022-06-07 10:42:22 +02:00
Saša Živkov 011c26ff36 Fix connection leak for smart http connections
SmartHttpPushConnection: close InputStream and OutputStream after
processing. Wrap IOExceptions which aren't TransportExceptions already
as a TransportException.

Also-By: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8e11d899672fc470c390a455dc86367e92ef9076
2022-06-06 08:14:18 +02:00
James Z.M. Gao d67ac798f1 Remove stray files (probes or lock files) created by background threads
NOTE: port back from master branch.

On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.

Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.

Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.

Bug: 579445
Change-Id: Iaee2301eb14e6201406398a90228ad10cfea6098
2022-05-27 01:20:16 +02:00
Darius Jokilehto 78c9b9260a Stop initCause throwing in readAdvertisedRefs
BasePackConnection::readAdvertisedRefsImpl was creating an exception by
calling `noRepository`, and then blindly calling `initCause` on it. As
`noRepository` can be overridden, it's not guaranteed to be missing a
cause.

BasePackPushConnection overrides `noRepository` and initiates a fetch,
which may throw a `NoRemoteRepositoryException` with a cause.

In this case calling `initCause` threw an `IllegalStateException`.

In order to throw the correct exception, we now return the
BasePackPushConnection exception and suppress the one thrown by
BasePackConnection

Bug: 578511
Change-Id: Ic1018b214be1e83d895979ee6c7cbce3f6765f6f
2022-02-08 09:52:03 +00:00
Matthias Sohn 2cc0009737 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: Ib43068c32d9cb8effe4b873396391dc3c9197a6e
2022-01-18 17:51:14 +01:00
Matthias Sohn 1e59cabc08 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: I5db432bd416cfa8d3dd295bdce63e31d5f160a8a
2022-01-18 17:49:03 +01:00
Marcin Czech 78d4fb1ca0 UploadPack v2 protocol: Stop negotiation for orphan refs
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).

To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.

Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
2022-01-18 08:00:03 -04:00
Matthias Sohn 486afbc08d Use slf4j-simple instead of log4j for logging
JGit uses slf4j-api as logging API.

The libraries
- org.eclipse.jgit.http.test
- org.eclipse.jgit.pgm
- org.eclipse.jgit.ssh.apache.test
- org.eclipse.jgit.test
used the outdated log4j 1.2.15 which is EOL since years.

Since both jgit command line and also the tests don't need sophisticated
logging features replace log4j with the much simpler slf4j-simple log
implementation. The org.slf4j.binding.simple 1.7.30 archive has only
25kB instead of 429kB for log4j 1.2.15

Applications using jgit are free to choose any other log implementation
supporting slf4j API.

Change-Id: I89e85cd3c76e954c3434622510975ce65dc227d4
2021-12-31 01:09:52 +01:00
Matthias Sohn e06eeb0167 Update orbit to R20211213173813
and update
- com.google.gson to 2.8.8.v20211029-0838
- javaewah to 1.1.13.v20211029-0839
- net.i2p.crypto.eddsa to 0.3.0.v20210923-1401
- org.apache.ant to 1.10.12.v20211102-1452
- org.apache.commons.compress to 1.21.0.v20211103-2100
- org.bouncycastle.bcprov to 1.69.0.v20210923-1401
- org.junit to 4.13.2.v20211018-1956

Change-Id: I3ac39fc8a5df571d2e290241a03668f1e60880b4
2021-12-31 01:09:51 +01:00
Matthias Sohn c8ab1392d1 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I6a0266cbcaaf18d0d60f0abecb5434fd919c44b7
2021-12-31 00:29:40 +01:00
Matthias Sohn 44bad3d98e Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I4e241860c2ca50750e22c2761c515c9895688c55
2021-12-31 00:28:53 +01:00
Matthias Sohn 8a4b983767 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: Ie3f2d05aeb1aa04af707cfafef5780349be4d981
2021-12-31 00:26:24 +01:00
Matthias Sohn 95bf9bbfe8 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I4f954c48ad6e8ff18826fdc72d225bff3e3ae2d9
2021-12-31 00:05:40 +01:00
Matthias Sohn c6b0ee04e4 Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: Ic97d38fc85daa00297abbfa186f83b779966e7ef
2021-12-30 23:58:41 +01:00
Matthias Sohn 725e77a517 Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: If9cc2f2bae5dbead7a38218828da461540be942e
2021-12-30 23:56:32 +01:00
Matthias Sohn 1343dd2de7 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I274d46d73cc896dcfde6e24c69c71f33aaa78d20
2021-12-30 23:53:54 +01:00
Matthias Sohn a461472bd5 Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: If904289feecd1e0d8466c1fb998f160f14d54b61
2021-12-30 23:51:41 +01:00
Matthias Sohn d87e5e07a5 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I84e11bdaa9306e23212dac9d8670557a18d40107
2021-12-30 23:41:54 +01:00
Matthias Sohn 7f51f8acd8 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I3d8eb2fa721e1a791db47a2342acc690ced01715
2021-12-30 23:40:21 +01:00
Matthias Sohn 62c2159b62 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: Ib79c310c5b632e845ba69ce65e739ae0146103ca
2021-12-30 23:33:06 +01:00
Matthias Sohn f33ae743ad Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: I17ede8876a0cf231c38cb9652c7bf51553b1e90e
2021-12-30 23:18:21 +01:00
Luca Milanesio fed0ab9baa Use FileSnapshot without using configs for FileBasedConfig
FileBasedConfig should not rely on auto-detection of
the file-snapshot attribute computation based on config.

The check was already performed when a new FileBasedConfig
is created at L158:

// don't use config in this snapshot to avoid endless recursion
newSnapshot = FileSnapshot.saveNoConfig(getFile());

The check was missing though when the FileBasedConfig is saved
to disk and the new snapshot is obtained from the associated
LockFile.

This change fixes the issue by keeping a non-config based
FileSnapshot also after a FileBasedConfig is saved.

Bug: 577983
Change-Id: Id1e410ba687e683ff2b2643af31e1110b103b356
2021-12-30 11:14:47 +01:00
Thomas Wolf 41406e278f Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I6576872cc0f5dd452252fa6e4526086cdee65c28
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 16:03:20 +01:00
Thomas Wolf f86d82fc30 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: Ib80336a42e22da729b9db1e573772504cc0a3e77
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 16:02:35 +01:00
Thomas Wolf 5f8c593bb6 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I9e79ea2a0c554a184e4ce3b13e375eac8b7a4ac5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 16:01:49 +01:00
Thomas Wolf 8a9b4972b4 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I2a84c838a886d1d6383c34f50b418baa743c57b0
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 16:00:48 +01:00
Thomas Wolf 95ea1c1123 Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I88a629e571fec5a9820114ebf5765b5d94a276bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:58:28 +01:00
Thomas Wolf 9fd632b0e4 Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: Ied786ab5e3c0dd05f701705fce2d4ad85502c4d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:57:42 +01:00
Thomas Wolf d295294dbe Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I454622dae6eb95aedbd858e3b12da72282d36673
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:56:33 +01:00
Thomas Wolf 1c689d18c4 Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I2622f1d384a88a556ba9d88f0d08a37af69e530c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:55:32 +01:00
Thomas Wolf fd612a1282 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: Ia1665dd92ccc3811a6116f41421a05aca10fc6eb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:54:12 +01:00
Thomas Wolf 4cc21ccd3d Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I52a57a17abe60e30e3d7615f8cb4d0c5e6aebd9b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:53:17 +01:00
Thomas Wolf 7edb062d8c Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: Id37f47a5ef2e3c8329eca30c171941f7e5606a85
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:52:03 +01:00
Thomas Wolf f2d4783c52 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"

Change-Id: I625667c2718ab31ae7df907c3dd6024a933913b8
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:50:42 +01:00
Thomas Wolf 7828ef349c Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation"
This reverts commit f829f5f838.

Using MISSING_FILEKEY as indicator for a non-existing file doesn't work
on Windows.

Bug: 577954
Change-Id: I92102a3d259f6cc0f367096a3213cfa794466817
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-12-26 15:38:26 +01:00
kylezhao 3d7351ee50 TreeRevFilter: fix wrong stop when the given path disappears
When chgs[i] == adds[i], it indicated that a commit added some files
that pList[i] did not have, but didn't mean pList[i] is "empty tree
root".

Follow the example below:

.                           .
└── src                     └── src
    └── d1          ==>          └── d1
        └─ file1                    ├─  file1
                                    └── file2
   c.parents[i]                   c

The variable chg[i] equals to variable add[i],
but commit c.parents[i] is not "empty tree root".

We should add an additional check for no paths matching the filter.

Bug: 577227
Change-Id: I834e9ddd0de86b108b280a1139519ea962913b38
Signed-off-by: kylezhao <kylezhao@tencent.com>
2021-12-17 00:29:11 +01:00
Matthias Sohn 35713588fe Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Add missing @since tags
  Add missing @since tag
  Add missing @since tags
  Remove unused import in ApacheSshTest
  Update maven plugins
  Ignore missing javadoc in test bundles
  storage: file: De-duplicate File.exists()+File.isFile()
  RefDirectory.scanRef: Re-use file existence check done in snapshot creation
  FileSnapshot: Lazy load file store attributes cache
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix p2 repository URLs

Change-Id: I9c85b3a774480c4f4b56ecdf5015916349a0ddc0
2021-12-04 21:06:33 +01:00
Matthias Sohn 1cffba4385 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Add missing @since tags
  Add missing @since tag
  Add missing @since tags
  Remove unused import in ApacheSshTest
  Update maven plugins
  Ignore missing javadoc in test bundles
  storage: file: De-duplicate File.exists()+File.isFile()
  RefDirectory.scanRef: Re-use file existence check done in snapshot creation
  FileSnapshot: Lazy load file store attributes cache
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix p2 repository URLs

Change-Id: I72f39fbe82dd578d71aa2e3ceccebb39a25c4e00
2021-12-04 20:42:56 +01:00
Matthias Sohn e14b25f240 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Add missing @since tags
  Add missing @since tag
  Add missing @since tags
  Remove unused import in ApacheSshTest
  Update maven plugins
  Ignore missing javadoc in test bundles
  storage: file: De-duplicate File.exists()+File.isFile()
  RefDirectory.scanRef: Re-use file existence check done in snapshot creation
  FileSnapshot: Lazy load file store attributes cache
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix p2 repository URLs

Change-Id: I84c73e11381c31566a12033db94e9afdcef5b705
2021-12-04 20:28:53 +01:00