Commit Graph

6179 Commits

Author SHA1 Message Date
Thomas Wolf 1c886d92f6 RawText.isBinary(): handle complete buffer correctly
Make sure we always get consistent results, whether or not we have the
full data in the buffer.

Change-Id: Ieb379a0c375ad3dd352e63ac2f23bda6ef16c215
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-11-16 15:17:19 -05:00
Matthias Sohn 0fb9d26eff Merge branch 'stable-6.3'
* stable-6.3:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Fix API breakage caused by extracting WorkTreeUpdater
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Ignore IllegalStateException if JVM is already shutting down
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Ib58879be292c54a2a7f4936ac0986997985c822b
2022-11-16 10:15:30 +01:00
Matthias Sohn 1cd9a1f804 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: I562be0802efa231023c5f10e6461339b2d7fbacf
2022-11-16 10:14:13 +01:00
Matthias Sohn a24b22632f Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Icb321779184d20f3871e236fda1a3acba605a6da
2022-11-16 10:13:20 +01:00
Matthias Sohn 77e2f4bd27 Merge "Merge branch 'stable-6.0' into stable-6.1" into stable-6.1 2022-11-16 04:10:43 -05:00
Matthias Sohn f3e0e9d5a3 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa
2022-11-16 09:56:08 +01:00
Matthias Sohn d588c2c9ad Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286
2022-11-16 09:55:22 +01:00
Matthias Sohn 7f36943d0c Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
2022-11-16 09:54:28 +01:00
Matthias Sohn f1909615d3 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc
2022-11-16 00:15:17 +01:00
Matthias Sohn 59029aec30 Add option to allow using JDK's SHA1 implementation
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.

When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.

Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e
2022-11-15 23:08:13 +01:00
Matthias Sohn 4d9e004f57 Fix API breakage caused by extracting WorkTreeUpdater
448052dc2e made WorkTreeUpdater package visible which breaks API for
subclasses of ResolveMerger since they cannot access WorkTreeUpdater.

Fix this by moving WorkTreeUpdater into ResolveMerger as a nested class
and mark it protected so that subclasses can use it.

Bug: 581049
Change-Id: I5a2c4953f8514dc0a1b8041c8e069d28370bb2eb
2022-11-15 15:18:38 +01:00
Eryk Szymanski bd2aced4a3 Use replace instead of replaceAll in toCleanString
This is from SonarLint (rule.java:S4348)
Regex patterns should not be created needlessly:

When String::replaceAll is used, the first argument should be a real
regular expression. If it’s not the case, String::replace does exactly
the same thing as String::replaceAll without the performance drawback of
the regex.

Change-Id: I00ba967ff4a27eeeb6fccf9373f6df2c94ecd823
2022-11-10 20:08:25 +01:00
Josh Brown fe9aeb02e6 UploadPack: Receive and parse client session-id
Before this change JGit did not support the session-id capability
implemented by native Git in UploadPack. This change implements
advertising the capability from the server and parsing the session-id
received from the client during an UploadPack operation.

Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the UploadPack object.

This change does not add the capability to send a session-id from the
JGit client.

https://git-scm.com/docs/gitprotocol-capabilities#_session_idsession_id

Change-Id: Ib1b6929ff1b3a4528e767925b5e5c44b5d18182f
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 16:13:22 -04:00
Josh Brown 7b0a71a5e9 TransferConfig: Move reading advertisesid setting into TransferConfig
The config setting to enable advertising the session-id capability is
currently read in the ReceivePack class. This change moves it to a
common location in the TransferConfig class so that it can be reused
in other places like UploadPack. TransferConfig is also a more logical
place for the setting as it resides in the `transfer` config section.

Set the transfer.advertisesid setting to true to send the session-id
capability to the client.

Change-Id: If68ecb5e68b59f5c452a7992d02e3688b0a86747
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 16:13:08 -04:00
Josh Brown e8068188f1 FirstWant: Parse client session-id if received.
In protocol V0 the client capabilities are appended to the first line.
Parsing session-id is currently only supported during a ReceivePack
operation. This change will parse the client session-id capability if
it has been sent by the client.

If the server sends the session-id capability to the client. The client
may respond with a session ID of its own. FirstWant.fromLine will now
parse the ID and make it available via the getClientSID method.

This change does not add support to send the session-id capability from
the server. The change is necessary to support session-id in UploadPack.

Change-Id: Id3fe44fdf9a72984ee3de9cf40cc4e71d434df4a
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 20:12:03 +00:00
Josh Brown 93097f0018 ReceivePack: Receive and parse client session-id.
Before this change JGit did not support the session-id capability
implemented by native Git. This change implements advertising the
capability from the server and parsing the session-id received from
the client during a ReceivePack operation.

Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the ReceivePack object. All capabilities
in the form `capability=value` are now split into key value pairs at the
first `=` character. This change replaces specific handling for the 
agent capability.

This change does not add advertisement or parsing to UploadPack. This
change also does not add the ability to send a session ID from the JGit
client.

https://git-scm.com/docs/protocol-v2/2.33.0#_session_idsession_id

Change-Id: I56fb115e843b11b27e128c4ac427b05d5ec129d0
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-10-27 16:17:50 -04:00
Matthias Sohn 924491d4df Ignore IllegalStateException if JVM is already shutting down
Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.

Bug: 580953
Change-Id: I8fc6fdd5585837c81ad0ebd6944430856556d90e
2022-10-27 20:31:58 +02:00
Saša Živkov cb90ed0852 Allow to perform PackedBatchRefUpdate without locking loose refs
Add another newBatchUpdate method in the RefDirectory where we can
control if the created PackedBatchRefUpdate will lock the loose refs or
not.

This can be useful in cases when we run programs which have exclusive
access to a Git repository and we know that locking loose refs is
unnecessary and just a performance loss.

Change-Id: I7d0932eb1598a3871a2281b1a049021380234df9
2022-10-26 16:51:08 +02:00
kylezhao ad9c217f49 PushCommand: allow users to disable use of bitmaps for push
Reachability bitmaps are designed to speed up the "counting objects"
phase of generating a pack during a clone or fetch. They are not
optimized for Git clients sending a small topic branch via "git push".
In some cases (see [1]), using reachability bitmaps during "git push"
can cause significant performance regressions.

Add PushCommand#setUseBitmaps(boolean) to allow users to tell "git push"
not to use bitmaps.

[1]: https://lore.kernel.org/git/87zhoz8b9o.fsf@evledraar.gmail.com/

Change-Id: I7fb7d26084ec63ddfa7249cf58abb85929b30e56
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-10-21 08:11:33 +02:00
Thomas Wolf 71af0d6a5c I/O redirection for the pre-push hook
Fix and complete the implementation of calling the pre-push hook.
Add the missing error stream redirect, and add the missing setters
in Transport and in PushCommand. In Transport, delay setting up a
PrePushHook such that it happens only on a push. Previously, the
hook was set up also for fetches.

Bug: 549246
Change-Id: I64a576dfc6b139426f05d9ea6654027ab805734e
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-10-20 23:34:56 +02:00
Ivan Frade 96236fdcb5 PackParser: populate full size of the PackedObjectInfos
We need the full size of the objects to populate the object-size index
of a pack. This size is not always the one encoded in the object header
in the pack (e.g. for deltas).

Populate the full size of PackedObjectInfos in the PackParser, which is
invoked when receiving a pack e.g. in a push.

Change-Id: I102c20901aefb5e85047e2e526c0d733f82ff74b
2022-10-18 11:19:21 -07:00
Ivan Frade 60206ea95f PackedObjectInfo: add the full size to the description
So we can create a size index later.

Change-Id: I9db47ced929fbf045fc37bead6449bbf5484d308
2022-10-18 11:19:21 -07:00
Ivan Frade b58ea5c6c9 ObjectReader: New #isNotLargerThan method
Partial clones filter the objects to send by size calling
ObjectReader#getObjectSize per object. This method reads the object from
storage to get the size, which can be expensive.

Offer a #isNotLargerThan method. The default implementation reads the
object, but subclasses can override it with more efficient lookups (e.g.
adding an index).

isNotLargerThan gives implementors more options to optimize than
getObjectIndex (e.g. can be implemented storing only object over certain
size).

Change-Id: Iefd4b1370cb9144f15cc0391286aeeb365e6ea87
2022-10-18 11:18:46 -07:00
Matthias Sohn 656e32e9f2 Remove unused API filters
Change-Id: Ief32f935c701842f03b173ee48465d7dcf7dfabd
2022-09-22 16:34:36 +02:00
Jörg Kubitz 2021ce3423 ObjectDirectory: avoid using File.getCanonicalPath()
On java 17 + Windows OS java.io.File.getCanonicalPath is a very slow
system call which uses most time during clone.

That is since JDK 12 the result of File.getCanonicalPath is not cached
anymore by default:
https://bugs.openjdk.java.net/browse/JDK-8207005

* Use toRealPath() to follow symbolic links also on windows.
* Cache the result.

Bug: 580568
Change-Id: I95f4f5b2babefd7210ee4740646230225ebf3788
2022-09-22 10:48:50 +02:00
Jörg Kubitz eb5124c74f AutoCRLFOutputStream: use BufferedOutputStream
This should improve performance of autocrlf checkout.

Bug: 580651
Change-Id: I2e2fe0273ac3c71fad50a575278234804ee28306
2022-09-20 10:00:17 +02:00
Thomas Wolf f71fcbf36b CloneCommand: set HEAD also when not checking out
CloneCommand, when setNoCheckout(true) was set, did not set HEAD.
With C git, "git clone --no-checkout" does.

Change-Id: Ief3df7e904ce90829a6345a6c3e9ee6a68486ab0
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-18 19:43:40 +02:00
Thomas Wolf 4f4204914c Pass on shallowSince only if not null
FetchCommand.setShallowSince() and Transport.setDeepenSince() require
a non-null argument.

Change-Id: I1c3a20be518374e380a4e90787ed834438da40ee
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-18 19:32:31 +02:00
Thomas Wolf ccad49a28c Fix wrong @since tag
PatchApplier was merged only in JGit 6.4.

Change-Id: Ica84d8184c1d3a1acce8beba9b076f6c32e749d7
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-18 17:02:51 +02:00
Han-Wen NIenhuys 21a497843c Merge "Split out ApplyCommand logic to PatchApplier class" 2022-09-15 04:16:48 -04:00
Nitzan Gur-Furman acde6c8f5b Split out ApplyCommand logic to PatchApplier class
PatchApplier now routes updates through the index. This has two
results:

* we can now execute patches in-memory.

* the JGit apply command will now always update the
index to match the working tree.

Change-Id: Id60a88232f05d0367787d038d2518c670cdb543f
Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>
Co-authored-by: Nitzan Gur-Furman <nitzan@google.com>
2022-09-15 09:15:55 +02:00
Matthias Sohn 85182df267 Prepare 6.4.0-SNAPSHOT builds
Change-Id: I47ca5f1d0263caa0bfc7c303042360c6c5ac4dec
2022-09-14 13:56:40 +02:00
kylezhao ac3bf7896e Fix typo in FetchV2Request javadoc
Change-Id: I905dda15e1b7f5e431816d3fbb01a1672e5fc786
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-09-14 15:49:39 +08:00
Matthias Sohn fb377b09eb Prepare 6.3.1-SNAPSHOT builds
Change-Id: I44e159eca4131880d74d3078060e7e20f9b5ce76
2022-09-12 10:09:10 +02:00
Matthias Sohn 68e8ecc91b JGit v6.3.0.202209071007-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Iea3fae9f6f6c5fb0a79f7684334a3e0059738c4f
2022-09-07 16:07:11 +02:00
Matthias Sohn f8104e25f1 JGit v6.3.0.2022009070944-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I3cc78dbcf8c7970e80bf1499751611110ec2b30b
2022-09-07 15:39:48 +02:00
Thomas Wolf a8e683fef6 [merge] Fix merge conflicts with symlinks
Previous code would do a content merge on symlinks, and write the merge
result to the working tree as a file. C git doesn't do this; it leaves
a symlink in the working tree unchanged, or in a delete-modify conflict
it would check out "theirs".

Moreover, previous code would write the merge result to the link target,
not to the link. This would overwrite an existing link target, or fail
if the link pointed to a directory.

In link/file conflicts or file/link conflicts, C git always puts the
file into the working tree.

Change conflict handling accordingly. Add tests for all the conflict
cases.

Bug: 580347
Change-Id: I3cffcb4bcf8e336a85186031fff23f0c4b6ee19d
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-07 15:02:02 +02:00
Matthias Sohn 370d1170e8 Merge branch 'master' into stable-6.3
* master:
  Move WorkTreeUpdater to merge package
  WorkTreeUpdater: use DirCacheCheckout#StreamSupplier
  DirCacheCheckout#getContent: also take InputStream supplier
  WorkTreeUpdater: remove safeWrite option

Change-Id: I8be570dbc4ad0d0b46046b85cbda24c3adcba170
2022-09-06 15:58:47 +02:00
Matthias Sohn 57087e2b92 Merge changes I888f313f,I98de155c,I2efa9a6d,Ifa79dac2
* changes:
  Move WorkTreeUpdater to merge package
  WorkTreeUpdater: use DirCacheCheckout#StreamSupplier
  DirCacheCheckout#getContent: also take InputStream supplier
  WorkTreeUpdater: remove safeWrite option
2022-09-06 09:42:14 -04:00
Han-Wen Nienhuys 448052dc2e Move WorkTreeUpdater to merge package
This avoids making it public with the associated costs for backward
compatibility guarantees.

Change-Id: I888f313f3854deace8d4cd92f354a6ef0d3b5460
2022-09-06 14:48:13 +02:00
Han-Wen Nienhuys 6ed4c074ca WorkTreeUpdater: use DirCacheCheckout#StreamSupplier
This avoids having to introduce the StreamLoader bridging class.

Change-Id: I98de155c458745236df24d6323eabed5061e7f8c
2022-09-06 14:48:13 +02:00
Han-Wen Nienhuys 6d2bcb6043 DirCacheCheckout#getContent: also take InputStream supplier
This lets us use DirCacheCheckout for routines that want to write
files in the worktree that aren't available as a git object.

DirCacheCheckout#getContent takes a InputStream supplier rather than
InputStream: if filtering fails with IOException, the data is placed
unfiltered in the checkout. This means that the stream has to be read
again, from the start.

Use it in this way in ApplyCommand. This use is incorrect, though: the
same InputStream is returned twice, so if the read to be retried, the
stream will return 0 bytes. It doesn't really matter, because in
either case, the SHA1 will not match up, and the patch fails.

Change-Id: I2efa9a6da06806ff79b155032fe4b34be8fec09e
2022-09-06 14:38:41 +02:00
Matthias Sohn 7c4a5421cc Revert "Adds FilteredRevCommit that can overwrites its parents in the
DAG."

This reverts commit 6297491e8a.

This is done as a quick fix for the failure of egit tests caused by  the
introduction of FilteredRevCommit.

Bug: 580690
Change-Id: Ia6b651dd11b0a4b02d5e52247eb4bf13adf94e27
2022-09-06 10:40:26 +02:00
Matthias Sohn ee6334bccf Revert "Option to pass start RevCommit to be blamed on to the
BlameGenerator."

This reverts commit 5747bba48b.

This is done as a quick fix for the failure of egit tests caused by  the
introduction of FilteredRevCommit.

Bug: 580690
Change-Id: Ia0178bc2de4fc825a81207bbd7979bf3a386c955
2022-09-06 10:40:26 +02:00
Han-Wen Nienhuys 2778bb2cc9 WorkTreeUpdater: remove safeWrite option
This was added in Ideaefd5178 to anticipate on writing files for
ApplyCommand, but we are keeping WorkTreeUpdater private to the merge
package for now.

Change-Id: Ifa79dac245e60eb7a77eaea4cc1249222e347d38
2022-09-06 10:01:44 +02:00
Ivan Frade ca166a0c62 BaseSuperprojectWriter: report invalid paths as manifest errors
An invalid path in the manifest (e.g. '.') is reported by DirCache in a
runtime exception. In server context this becomes a 500 instead of a user error.

Wrap the runtime invalid path exception into a checked ManifestErrorException that
caller can handle.

Change-Id: I61a2104922765506ae232334891057bb06141d97
2022-08-23 12:10:27 -07:00
Han-Wen Nienhuys 25aceffdc5 ApplyCommand: fix ApplyResult#updatedFiles
On executing a copy, mark the destination as updated.

On executing a rename, mark both source and destination as updated.

Change-Id: Ied5b9b0e5a14eac59a06cdd0961e25e143f50ff0
2022-08-23 09:23:23 -04:00
Han-Wen Nienhuys 0887111ba5 WorkTreeUpdater: rename metadata maps
Change-Id: I8ff3803da8fc13377d11c2dc5523e9e32d0650cb
2022-08-23 05:13:43 -04:00
Han-Wen Nienhuys ddb1a0c3a2 WorkTreeUpdater#Result: hide data members
This is the standard across JGit.

Change-Id: Ie52ad7000d8725657b33dd4f3adcc05ab9666875
2022-08-23 04:24:37 -04:00
Han-Wen NIenhuys d718127a7e Merge "WorkTreeUpdater: re-format and clean-up" 2022-08-18 11:22:46 -04:00