Commit Graph

8869 Commits

Author SHA1 Message Date
Matthias Sohn 97ad9bdae6 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Silence API errors
  Silence API warnings

Change-Id: I2b8336652e60dec97666582cf9331c8505729473
2022-11-20 20:08:42 +01:00
Matthias Sohn 41b33a16b8 Silence API errors
Change-Id: Ie112b2099ea2125bc85863524e56f09ba4907373
2022-11-20 19:55:22 +01:00
Matthias Sohn 12f48276bd Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Silence API warnings

Change-Id: If5ab988a0e177c37b125e0b10625e506eeb2a74f
2022-11-20 19:54:44 +01:00
Matthias Sohn aa9f736c33 Silence API warnings
introduced by
- addition of configurable SHA1 implementation in 5.13.2
- 3-digit @since 5.9.1 annotations on GitServlet methods

Change-Id: If19853fcc5e3677e5b18e8e3fbbcd2773378dffc
2022-11-20 19:45:54 +01:00
Matthias Sohn 77e2f4bd27 Merge "Merge branch 'stable-6.0' into stable-6.1" into stable-6.1 2022-11-16 04:10:43 -05:00
Matthias Sohn 7f36943d0c Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
2022-11-16 09:54:28 +01:00
Matthias Sohn f1909615d3 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc
2022-11-16 00:15:17 +01:00
Matthias Sohn 48316309b1 [benchmarks] Remove profiler configuration
Profiler configuration can be added when required. It was commented out
in most benchmarks.

Change-Id: I725f98757f7d4d2ba2589658e34e2fd6fbbbedee
2022-11-15 23:08:20 +01:00
Matthias Sohn c20e9676c4 Add SHA1 benchmark
Results on a Mac M1 max:

    size     SHA1Native SHA1Java    SHA1Java
                        without     with
                        collision   collision
                        detection   detection
    [kB]     [us/op]    [us/op]     [us/op]
---------------------------------------------
      1       3.662       4.200       4.707
      2       7.053       7.868       8.928
      4      13.883      15.149      17.608
      8      27.225      30.049      35.237
     16      54.014      59.655      70.867
     32     106.457     118.022     140.403
     64     212.712     237.702     281.522
   1024    3469.519    3868.883    4637.287
 131072  445011.724  501751.992  604061.308
1048576 3581702.104 4008087.854 4831023.563

The last 3 sizes (1, 128, 1024 MB) weren't committed
here to limit the total runtime.

Bug: 580310
Change-Id: I7d0382fd4aa4c4734806b12e96b671bee37d26e3
2022-11-15 23:08:19 +01:00
Matthias Sohn 4fac796588 [benchmarks] Set version of maven-compiler-plugin to 3.8.1
Change-Id: Ib14db133c76a55358ea79663ef38d9fb47a67f45
2022-11-15 23:08:19 +01:00
Matthias Sohn 8d53a1433d Fix running JMH benchmarks
Without this I sometimes hit the error:

$ java -jar target/benchmarks.jar
Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to
find the resource: /META-INF/BenchmarkList
	at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:98)
	at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:124)
	at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:253)
	at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
	at org.openjdk.jmh.Main.main(Main.java:71)

Change-Id: Iea9431d5f332f799d55a8a2d178c79a2ef0da22b
2022-11-15 23:08:19 +01:00
Matthias Sohn 59029aec30 Add option to allow using JDK's SHA1 implementation
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.

When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.

Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e
2022-11-15 23:08:13 +01:00
Sven Selberg 5bf7472e0c Extract Exception -> HTTP status code mapping for reuse
Extract private static method UploadPackServlet#statusCodeForThrowable
to a public static method in the UploadPackErrorHandler interface so
that implementers of this interface can reuse the default mapping.

Change-Id: Ie4a0a006b0148d5b828d610c55d19ce407aab055
2022-11-10 19:59:50 +01:00
Sven Selberg ac127a7932 Don't handle internal git errors as an HTTP error
The fix that fixed the propagation of error-codes:
  8984e1f66 HTTP Smart: set correct HTTP status on error [1]
made some faulty assumptions.

"Wants not valid", can be an intermittent git error and the HTTP
response should be 200 and not 400 since the request isn't necessary
faulty.

[1] https://git.eclipse.org/r/c/jgit/jgit/+/192677

Bug: 579676
Change-Id: I461bc78ff6e450636811ece50d21c57a2a7f2ae3
2022-11-10 04:35:28 -04:00
Matthias Sohn 924491d4df Ignore IllegalStateException if JVM is already shutting down
Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.

Bug: 580953
Change-Id: I8fc6fdd5585837c81ad0ebd6944430856556d90e
2022-10-27 20:31:58 +02:00
Saša Živkov cb90ed0852 Allow to perform PackedBatchRefUpdate without locking loose refs
Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.

This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.

Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9
2022-10-26 16:51:08 +02:00
Matthias Sohn d01376106a Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: I08734ee2c8f3296d908da6a29d53ed87c4b48eb2
2022-07-06 17:00:42 +02:00
Matthias Sohn 9f7d77b608 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: I6db0a4d74399fde892eeec62efd2946f97547a5d
2022-07-06 16:59:30 +02:00
Matthias Sohn 035e0e23f2 UploadPack: don't prematurely terminate timer in case of error
In uploadWithExceptionPropagation don't prematurely terminate timer in
case of error to enable reporting it to the client. Expose a close
method so that callers can terminate it at the appropriate time.

If the timer is already terminated when trying to report it to the
client this failed with the error java.lang.IllegalStateException:
"Timer already terminated".

Bug: 579670
Change-Id: I95827442ccb0f9b1ede83630cf7c51cf619c399a
2022-06-30 14:45:31 +02:00
Matthias Sohn ca6b518432 Merge "Do not create reflog for remote tracking branches during clone" into stable-5.13 2022-06-26 15:36:03 -04:00
Luca Milanesio 4bb4693633 Do not create reflog for remote tracking branches during clone
When using JGit on a non-bare repository, the CloneCommand
it previously created local reflogs for all branches including remote
tracking ones, causing the generation of a potentially large
number of files on the local filesystem.

The creation of the remote-tracking branches (refs/remotes/*) during
clone is not an issue for the local filesystem because all of them are
stored in a single packed-refs file. However, the creation of a large
number of ref logs on a local filesystem IS an issue because it
may not be tuned or initialised in term of inodes to contain a very
large number of files.

When a user (or a CI system) performs the CloneCommand against
a potentially large repository (e.g., millions of branches), it is
interested in working or validating a single branch or tag and is
unlikely to work with all the remote-tracking branches.
The eager creation of a reflogs for all the remote-tracking branches is
not just a performance issue but may also compromise the ability to
use JGit for cloning a large repository.

The behaviour implemented in this change is also consistent with the
optimisation done in the C code-base [1].

We differentiate between clone and fetch commands using --branch
<initialBranch> option, that is only available in clone command,
and is set as HEAD per default.

[1] 58f233ce1e

Bug: 579805
Change-Id: I58d0d36a8a4ce42e0f59b8bf063747c4b81bd859
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2022-06-25 12:09:01 +01:00
Luca Milanesio 66ace4b9af UploadPack: do not check reachability of visible SHA1s
When JGit needs to serve a Git client requesting SHA1s
during the want phase, it needs to make a full reachability
check from the advertised refs to the ones requested to
keep all objects in the correct scope of confidentiality
allowed by the avertised refs.

The check is also performed when the SHA1 corresponds to
one of the tips of the advertised refs which is a waste of
resources.

Example:

fetch> ref-prefix refs/heads/foo
fetch< 900505eb8ce8ced2a1757906da1b25c357b9654e refs/heads/foo
fetch< 0000
fetch> command=fetch
fetch> 0001
fetch> thin-pack
fetch> ofs-delta
fetch> want 900505eb8ce8ced2a1757906da1b25c357b9654e

The SHA1 in the want is the tip of refs/heads/foo and therefore
the full reachability check can be shortened and resolved more
quickly.

Change-Id: I49bd9e2464e0bd3bca2abf14c6e9df550d07383b
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2022-06-25 06:45:05 -04:00
Matthias Sohn f4cbf31ae4 Add missing package import javax.management to org.eclipse.jgit
Class org.eclipse.jgit.util.Monitoring uses JMX hence we need this
import otherwise OSGi applications can face ClassNotFoundException.

Bug: 577018
Change-Id: Ifd75337b87c7faec95d333b771bb0a2f3e46a418
2022-06-17 13:49:59 +02:00
Matthias Sohn d0bc2b544a Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: Ie9c38ab8033fe1283e8b444b6acd3f4298062bf3
2022-06-15 16:32:08 +02:00
Matthias Sohn d961bb6502 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: Ibd663a1d874d1aac274abc3dd44354fd99f64c39
2022-06-15 16:31:38 +02:00
Matthias Sohn db074a1352 Prepare 5.13.2-SNAPSHOT builds
Change-Id: I4862e5d80a7d95a1a119d06306e3f6927445d1d3
2022-06-14 00:41:18 +02:00
Matthias Sohn b34961a493 JGit v5.13.1.202206130422-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ife74d64e8171c68dbf08271492c0ac852a6dc51c
2022-06-13 10:22:43 +02:00
eric.steele e9a5430c25 AmazonS3: Add support for AWS API signature version 4
Updating the AmazonS3 class to support AWS Signature version 4 because
version 2 is no longer supported in all AWS regions. The version can be
selected with the new 'aws.api.signature.version' property (defaults to
2 for backwards compatibility). When set to '4', the user must also
specify the AWS region via the 'region' property. The 'region' property
must match the region that the 'domain' property resolves to.

Bug: 579907
Change-Id: If289dbc6d0f57323cfeaac2624c4eb5028f78d13
2022-06-13 09:44:23 +02:00
Matthias Sohn 1bd087a7f5 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Fix connection leak for smart http connections

Change-Id: I8b7e643b87c0f788cdafb060a2c9e8b4ef6f7375
2022-06-07 11:36:11 +02:00
Matthias Sohn a96645a5f3 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Fix connection leak for smart http connections

Change-Id: Ic851f2c4660ed761f5527e405b116b54da42fb7c
2022-06-07 11:35:13 +02:00
Matthias Sohn 5efd32e91d Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Fix connection leak for smart http connections

Change-Id: Id34f29c1b27a80c2b56c911cad7e3f64ef63af48
2022-06-07 11:34:25 +02:00
Matthias Sohn 8bb17e518f Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Fix connection leak for smart http connections

Change-Id: I6caabf4774ccf34706cef846c1087710f67e2ecd
2022-06-07 11:33:38 +02:00
Matthias Sohn c7335f32e9 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Fix connection leak for smart http connections

Change-Id: I3885c6114caed897f762f5ce523d3b27288205b2
2022-06-07 10:53:24 +02:00
Matthias Sohn 85011b8b07 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Fix connection leak for smart http connections

Change-Id: I5e7144b2f5cd850978220c476947001ae2debb8e
2022-06-07 10:42:22 +02:00
Saša Živkov 011c26ff36 Fix connection leak for smart http connections
SmartHttpPushConnection: close InputStream and OutputStream after
processing. Wrap IOExceptions which aren't TransportExceptions already
as a TransportException.

Also-By: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8e11d899672fc470c390a455dc86367e92ef9076
2022-06-06 08:14:18 +02:00
Matthias Sohn 29dce2e7be Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Remove stray files (probes or lock files) created by background threads

Change-Id: Iab21b9776c2ebecd4cad5d015028cadd4b6541db
2022-05-27 16:22:09 +02:00
Matthias Sohn 9612aae885 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Remove stray files (probes or lock files) created by background threads

Change-Id: I7af1355a77f14995118145162f6bb8a4f1755f2b
2022-05-27 16:20:28 +02:00
James Z.M. Gao d67ac798f1 Remove stray files (probes or lock files) created by background threads
NOTE: port back from master branch.

On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.

Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.

Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.

Bug: 579445
Change-Id: Iaee2301eb14e6201406398a90228ad10cfea6098
2022-05-27 01:20:16 +02:00
Sven Selberg 8984e1f663 HTTP Smart: set correct HTTP status on error
Previous behavior was that status code was automatically set to 200
regardless of reported status and according to HTTP Smart protocol[1]:

  If there is no repository at $GIT_URL, or the resource pointed to by
  a location matching $GIT_URL does not exist, the server MUST NOT
  respond with 200 OK response. A server SHOULD respond with
  404 Not Found, 410 Gone, or any other suitable HTTP status code which
  does not imply the resource exists as requested.

Since the jgit HTTP client isn't able to handle reading content from a
response reporting an error (calling HttpURLConnection#getInputStream
on a "failed" connection throws an exception and the internal interface
HttpConnection does not expose HttpURLConnection#getErrorStream) the
SmartClientSmartServerTest needed to be rewritten to expect the generic
response messages.

[1] https://git-scm.com/docs/http-protocol#_general_request_processing

Bug: 579676
Change-Id: Ibb942d02124a0bc279df09600b091354019ce064
2022-05-09 22:06:38 +02:00
Matthias Sohn 8f7ef245f0 Prepare 6.1.1-SNAPSHOT builds
Change-Id: Ifc80355025d8459245843be1c24dc5a286913e77
2022-03-08 17:04:10 +01:00
Matthias Sohn e982de3fcb JGit v6.1.0.202203080745-r
Change-Id: I8766ed400020c9571f321bbbfe34b0688af0107d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2022-03-08 13:45:56 +01:00
Thomas Wolf f26ab4ebee [checkout] Use .gitattributes from the commit to be checked out
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.

These attributes must be used to determine whether the working tree is
dirty.

But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.

Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.

Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.

The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)

Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.

Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.

Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?

Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-07 18:45:25 +01:00
Matthias Sohn 72bba7bd53 Don't use final for method parameters
See https://wiki.eclipse.org/EGit/Contributor_Guide#Use_of_the_.22final.22_modifier

Change-Id: Idc6ed85483e381689e5085c4a1bacd75d26f5489
2022-03-06 18:52:59 +01:00
Thomas Wolf 8a2c769417 [push] support the "matching" RefSpecs ":" and "+:"
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.

This RefSpec can be expanded only once the advertisement from the
remote has been received.

Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.

Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 17:30:01 +01:00
Thomas Wolf 90df7c123e [push] Call the pre-push hook later in the push process
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.

Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 17:30:01 +01:00
Thomas Wolf 72ae234e79 IndexDiff: use tree filter also for SubmoduleWalk
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.

Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 11:11:26 +01:00
Matthias Sohn 46d58b84c9 Run license check with option -Ddash.projectId=technology.jgit
This ensures that works-with-dependencies are attributed properly.

Change-Id: Ie41733cdecc83ae0b532fb4bd9f692bc09e16107
2022-03-04 07:56:50 +01:00
Matthias Sohn c77284ab7f Exclude transitive dependencies of sshd-sftp
We don't need the dependencies of sshd-sftp to sshd-common and
sshd-core since they are contained in sshd-osgi.
Excluding them helps the dash IP log tool to not list them as required
dependencies of jgit.

Change-Id: I85bce4dca1b99ba880cf3e909fac9669c75e7854
2022-03-04 00:30:36 +01:00
Matthias Sohn aed354a1ed Update DEPENDENCIES for 6.1.0 release
Change-Id: I9ea925bd444952dc204e2538b9d16fe631202e0c
2022-03-03 22:11:06 +01:00
Matthias Sohn 996e651c14 Add dependency to dash-licenses
This is required to run the license check using the license tool [1]
required by the Eclipse project handbook [2]:

mvn org.eclipse.dash:license-tool-plugin:license-check -Ddash.summary=DEPENDENCIES

Note: the tool still requires Java 11 hence it needs to be run in a
separate build step and is not yet integrated in the build which runs
on Java 8.

[1] https://github.com/eclipse/dash-licenses
[2] https://www.eclipse.org/projects/handbook/#ip-license-tool

Change-Id: Ib41d54de246c3c9499cc3be9f026294c39fdfd99
2022-03-03 22:00:28 +01:00