Commit Graph

3299 Commits

Author SHA1 Message Date
kylezhao 7016e2ddae CommitGraph: add core.commitGraph config
Change-Id: I3b5e735ebafba09ca18fd83da479c7950fa3ea8d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 10:21:09 -05:00
Ivan Frade 6ea36794d1 Merge "Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering" 2022-12-16 08:20:24 -05:00
kylezhao 7b0f633b67 CommitGraph: implement commit-graph read
Git introduced a new file storing the topology and some metadata of
the commits in the repo (commitGraph). With this data, git can browse
commit history without parsing the pack, speeding up e.g.
reachability checks.

This change teaches JGit to read commit-graph-format file, following
the upstream format([1]).

JGit can read a commit-graph file from a buffered stream, which means
that we can provide this feature for both FileRepository and
DfsRepository.

[1] https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: Ib5c0d6678cb242870a0f5841bd413ad3885e95f6
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 06:57:06 -05:00
Anna Papitto 5c6c374ff6 Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering
Deleting orphan files depends on .pack and .keep being reverse-sorted
to before the corresponding index files that could be orphans. The new
reverse index file extension (.rev) will break that frail dependency.

Rewrite Gc#deleteOrphans to avoid that dependence by tracking which pack
names have a .pack or .keep file and then deleting any index files that
without a corresponding one. This approach takes linear time instead of
the O(n logn) time needed for sorting.

Change-Id: If83c378ea070b8871d4b01ae008e7bf8270de763
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-15 11:54:11 -08:00
Matthias Sohn ebc1f7d65c commitgraph package: fix exports/imports, add @since tag for new API
Change-Id: I9175b1d796f91f5ba4e21d3418550ae451c054b0
2022-12-08 02:00:58 +01:00
kylezhao cf70e7cbe4 CommitGraph: implement commit-graph writer
Teach JGit to write a commit-graph formatted file by walking commit
graph from specified commit objects.

See: https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: I34f9f28f8729080c275f86215ebf30b2d05af41d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-06 20:34:46 +08:00
Matthias Sohn 339b38340f Prepare 6.4.1-SNAPSHOT builds
Change-Id: I860bfde113c05015c41304c4a77c44c224bd0923
2022-11-30 15:41:41 +01:00
Matthias Sohn acd079b372 JGit v6.4.0.202211300538-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: If4001b255a209849b4acabd2083164d0794f00c4
2022-11-30 11:38:12 +01:00
Dmitrii Filippov cb9f058f9b Fix crashes on rare combination of file names
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.

The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.

The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.

The fix processes a situation when some trees reached the end, while
others are still point to an entry.

bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86
2022-11-29 10:49:27 +01:00
Han-Wen NIenhuys 1d5a6c77a6 Merge "Fix crashes on rare combination of file names" 2022-11-28 09:34:46 -05:00
Matthias Sohn 2e28f27c26 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I4238b6181e96e22e540cf34802a332f868cb6dfb
2022-11-23 19:09:33 +01:00
Matthias Sohn ce0c739f21 Prepare 6.4.0-SNAPSHOT build
Change-Id: I41c4f73472bb47d8f9d2d117d17e11bba4802928
2022-11-23 17:10:33 +01:00
Matthias Sohn a3a726be6a JGit v6.4.0.202211231055-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ia34696d07568b298544ee2cdc6f4b6746774bb82
2022-11-23 16:56:09 +01:00
Matthias Sohn bde09c185f UploadPackTest: ensure UploadPack is closed to fix resource leak
Change-Id: I4c8cf6041b4011934d338138d4531d190fdd6abb
2022-11-21 00:08:42 +01:00
Matthias Sohn cddff2b7fd Fix warnings in PatchApplierTest
- don't use final for method parameters
- fix hiding member warnigns

Change-Id: I73c386f669918d3291ee3380024c018483aa3c97
2022-11-20 20:41:20 +01:00
Matthias Sohn 143188e831 Fix boxing warnings in TransportTest
Change-Id: I7e6dc845b89899cff262fab77c3977dbef5eea02
2022-11-20 20:34:57 +01:00
Matthias Sohn f849b8d0d9 Silence warnings about unclosed BasePackPushConnection
Change-Id: If52e8462e6222dd58d1004dd5ac174a27d96d098
2022-11-20 20:32:37 +01:00
Anna Papitto accacc27a1 DfsStreamKey: Replace ForReverseIndex to separate metrics.
Keys used for identifying reverse indexes in the DfsBlockCache use a
custom subclass ForReverseIndex because there was no PackExt for them.
This conflates BlockCacheMetrics for reverse indexes with those for
packs, since the key falls back onto 0 when there is no extension.

Replace the custom ForReverseIndex with a DfsStreamKey usage to bring
keys for the new REVERSE_INDEX extension in line with INDEX and BITMAP
and separate reverse index and pack BlockCacheMetrics.

Change-Id: I305e2c16d2a8cb2a824855ea92e0c9a9b188fce5
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-11-17 12:38:36 -05:00
Thomas Wolf 1c886d92f6 RawText.isBinary(): handle complete buffer correctly
Make sure we always get consistent results, whether or not we have the
full data in the buffer.

Change-Id: Ieb379a0c375ad3dd352e63ac2f23bda6ef16c215
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-11-16 15:17:19 -05:00
Matthias Sohn 0fb9d26eff Merge branch 'stable-6.3'
* stable-6.3:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Fix API breakage caused by extracting WorkTreeUpdater
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Ignore IllegalStateException if JVM is already shutting down
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Ib58879be292c54a2a7f4936ac0986997985c822b
2022-11-16 10:15:30 +01:00
Matthias Sohn f3e0e9d5a3 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa
2022-11-16 09:56:08 +01:00
Matthias Sohn d588c2c9ad Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286
2022-11-16 09:55:22 +01:00
Matthias Sohn 7f36943d0c Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
2022-11-16 09:54:28 +01:00
Matthias Sohn f1909615d3 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I40105336f0b9e593a8a2c242a9557f854c274fdc
2022-11-16 00:15:17 +01:00
Matthias Sohn 59029aec30 Add option to allow using JDK's SHA1 implementation
The change If6da9833 moved the computation of SHA1 from the JVM's
JCE to a pure Java implementation with collision detection.
The extra security for public sites comes with a cost of slower
SHA1 processing compared to the native implementation in the JDK.

When JGit is used internally and not exposed to any traffic from
external or untrusted users, the extra cost of the pure Java SHA1
implementation can be avoided, falling back to the previous
native MessageDigest implementation.

Bug: 580310
Change-Id: Ic24c0ba1cb0fb6282b8ca3025ffbffa84035565e
2022-11-15 23:08:13 +01:00
Matthias Sohn f288de7490 Update Orbit to S20221109014815
and update
- com.sun.jna to 5.12.1.v20221103-2317
- com.sun.jna.platform to 5.12.1.v20221103-2317
- org.bouncycastle.bcpg to 1.72.0.v20221013-1810
- org.bouncycastle.bcpkix to 1.72.0.v20221013-1810
- org.bouncycastle.bcprov to 1.72.0.v20221013-1810
- org.bouncycastle.bcutil to 1.72.0.v20221013-1810
- org.mockito.mockito-core to 4.8.1.v20221103-2317
- org.objenesis to 3.3.0.v20221103-2317

Change-Id: If00094d23e51d5f66928f83c1334aa6b18b98dfe
2022-11-14 17:12:20 +01:00
Dmitrii Filippov 1e04046a6d Fix crashes on rare combination of file names
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.

The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.

The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.

The fix processes a situation when some trees reached the end, while
others are still point to an entry.

bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86
2022-11-03 14:09:56 -04:00
Josh Brown fe9aeb02e6 UploadPack: Receive and parse client session-id
Before this change JGit did not support the session-id capability
implemented by native Git in UploadPack. This change implements
advertising the capability from the server and parsing the session-id
received from the client during an UploadPack operation.

Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the UploadPack object.

This change does not add the capability to send a session-id from the
JGit client.

https://git-scm.com/docs/gitprotocol-capabilities#_session_idsession_id

Change-Id: Ib1b6929ff1b3a4528e767925b5e5c44b5d18182f
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 16:13:22 -04:00
Josh Brown 7b0a71a5e9 TransferConfig: Move reading advertisesid setting into TransferConfig
The config setting to enable advertising the session-id capability is
currently read in the ReceivePack class. This change moves it to a
common location in the TransferConfig class so that it can be reused
in other places like UploadPack. TransferConfig is also a more logical
place for the setting as it resides in the `transfer` config section.

Set the transfer.advertisesid setting to true to send the session-id
capability to the client.

Change-Id: If68ecb5e68b59f5c452a7992d02e3688b0a86747
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 16:13:08 -04:00
Josh Brown e8068188f1 FirstWant: Parse client session-id if received.
In protocol V0 the client capabilities are appended to the first line.
Parsing session-id is currently only supported during a ReceivePack
operation. This change will parse the client session-id capability if
it has been sent by the client.

If the server sends the session-id capability to the client. The client
may respond with a session ID of its own. FirstWant.fromLine will now
parse the ID and make it available via the getClientSID method.

This change does not add support to send the session-id capability from
the server. The change is necessary to support session-id in UploadPack.

Change-Id: Id3fe44fdf9a72984ee3de9cf40cc4e71d434df4a
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-11-02 20:12:03 +00:00
Josh Brown 93097f0018 ReceivePack: Receive and parse client session-id.
Before this change JGit did not support the session-id capability
implemented by native Git. This change implements advertising the
capability from the server and parsing the session-id received from
the client during a ReceivePack operation.

Enable the transfer.advertisesid config setting to advertise the
capability from the server. The client may send a session-id capability
in response. If received, the value from this is parsed and available
via the getClientSID method on the ReceivePack object. All capabilities
in the form `capability=value` are now split into key value pairs at the
first `=` character. This change replaces specific handling for the 
agent capability.

This change does not add advertisement or parsing to UploadPack. This
change also does not add the ability to send a session ID from the JGit
client.

https://git-scm.com/docs/protocol-v2/2.33.0#_session_idsession_id

Change-Id: I56fb115e843b11b27e128c4ac427b05d5ec129d0
Signed-off-by: Josh Brown <sjoshbrown@google.com>
2022-10-27 16:17:50 -04:00
kylezhao ad9c217f49 PushCommand: allow users to disable use of bitmaps for push
Reachability bitmaps are designed to speed up the "counting objects"
phase of generating a pack during a clone or fetch. They are not
optimized for Git clients sending a small topic branch via "git push".
In some cases (see [1]), using reachability bitmaps during "git push"
can cause significant performance regressions.

Add PushCommand#setUseBitmaps(boolean) to allow users to tell "git push"
not to use bitmaps.

[1]: https://lore.kernel.org/git/87zhoz8b9o.fsf@evledraar.gmail.com/

Change-Id: I7fb7d26084ec63ddfa7249cf58abb85929b30e56
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-10-21 08:11:33 +02:00
Thomas Wolf 71af0d6a5c I/O redirection for the pre-push hook
Fix and complete the implementation of calling the pre-push hook.
Add the missing error stream redirect, and add the missing setters
in Transport and in PushCommand. In Transport, delay setting up a
PrePushHook such that it happens only on a push. Previously, the
hook was set up also for fetches.

Bug: 549246
Change-Id: I64a576dfc6b139426f05d9ea6654027ab805734e
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-10-20 23:34:56 +02:00
Ivan Frade 96236fdcb5 PackParser: populate full size of the PackedObjectInfos
We need the full size of the objects to populate the object-size index
of a pack. This size is not always the one encoded in the object header
in the pack (e.g. for deltas).

Populate the full size of PackedObjectInfos in the PackParser, which is
invoked when receiving a pack e.g. in a push.

Change-Id: I102c20901aefb5e85047e2e526c0d733f82ff74b
2022-10-18 11:19:21 -07:00
Thomas Wolf f71fcbf36b CloneCommand: set HEAD also when not checking out
CloneCommand, when setNoCheckout(true) was set, did not set HEAD.
With C git, "git clone --no-checkout" does.

Change-Id: Ief3df7e904ce90829a6345a6c3e9ee6a68486ab0
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-18 19:43:40 +02:00
Han-Wen NIenhuys 21a497843c Merge "Split out ApplyCommand logic to PatchApplier class" 2022-09-15 04:16:48 -04:00
Nitzan Gur-Furman acde6c8f5b Split out ApplyCommand logic to PatchApplier class
PatchApplier now routes updates through the index. This has two
results:

* we can now execute patches in-memory.

* the JGit apply command will now always update the
index to match the working tree.

Change-Id: Id60a88232f05d0367787d038d2518c670cdb543f
Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>
Co-authored-by: Nitzan Gur-Furman <nitzan@google.com>
2022-09-15 09:15:55 +02:00
Matthias Sohn 85182df267 Prepare 6.4.0-SNAPSHOT builds
Change-Id: I47ca5f1d0263caa0bfc7c303042360c6c5ac4dec
2022-09-14 13:56:40 +02:00
Matthias Sohn c7df1addf6 Merge branch 'stable-6.3'
* stable-6.3:
  Prepare 6.3.1-SNAPSHOT builds
  JGit v6.3.0.202209071007-r
  JGit v6.3.0.2022009070944-r
  [merge] Fix merge conflicts with symlinks
  Update DEPENDENCIES for 6.3.0
  Update tycho to 2.7.5
  Revert "Adds FilteredRevCommit that can overwrites its parents in the DAG."
  Revert "Option to pass start RevCommit to be blamed on to the BlameGenerator."
  Prepare 6.3.0-SNAPSHOT builds
  JGit v6.3.0.202208161710-m3

Change-Id: Ia9430fb516dca795e25064a190704b70689af364
2022-09-12 10:51:37 +02:00
Matthias Sohn fb377b09eb Prepare 6.3.1-SNAPSHOT builds
Change-Id: I44e159eca4131880d74d3078060e7e20f9b5ce76
2022-09-12 10:09:10 +02:00
yunjieli e7bffdfc48 DfsBundleWriter: Add test case about GC_REST pack.
Add a test case to make sure that the bundle writer writes objects in
GC_REST packs as well.

Signed-off-by: Yunjie Li <yunjieli@google.com>
Change-Id: Iba4d88c573aa1cda4505afbe2b83581a09a343df
2022-09-07 10:18:16 -07:00
Matthias Sohn 68e8ecc91b JGit v6.3.0.202209071007-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Iea3fae9f6f6c5fb0a79f7684334a3e0059738c4f
2022-09-07 16:07:11 +02:00
Matthias Sohn f8104e25f1 JGit v6.3.0.2022009070944-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I3cc78dbcf8c7970e80bf1499751611110ec2b30b
2022-09-07 15:39:48 +02:00
Thomas Wolf a8e683fef6 [merge] Fix merge conflicts with symlinks
Previous code would do a content merge on symlinks, and write the merge
result to the working tree as a file. C git doesn't do this; it leaves
a symlink in the working tree unchanged, or in a delete-modify conflict
it would check out "theirs".

Moreover, previous code would write the merge result to the link target,
not to the link. This would overwrite an existing link target, or fail
if the link pointed to a directory.

In link/file conflicts or file/link conflicts, C git always puts the
file into the working tree.

Change conflict handling accordingly. Add tests for all the conflict
cases.

Bug: 580347
Change-Id: I3cffcb4bcf8e336a85186031fff23f0c4b6ee19d
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-09-07 15:02:02 +02:00
Matthias Sohn 7c4a5421cc Revert "Adds FilteredRevCommit that can overwrites its parents in the
DAG."

This reverts commit 6297491e8a.

This is done as a quick fix for the failure of egit tests caused by  the
introduction of FilteredRevCommit.

Bug: 580690
Change-Id: Ia6b651dd11b0a4b02d5e52247eb4bf13adf94e27
2022-09-06 10:40:26 +02:00
Matthias Sohn ee6334bccf Revert "Option to pass start RevCommit to be blamed on to the
BlameGenerator."

This reverts commit 5747bba48b.

This is done as a quick fix for the failure of egit tests caused by  the
introduction of FilteredRevCommit.

Bug: 580690
Change-Id: Ia0178bc2de4fc825a81207bbd7979bf3a386c955
2022-09-06 10:40:26 +02:00
Ivan Frade ca166a0c62 BaseSuperprojectWriter: report invalid paths as manifest errors
An invalid path in the manifest (e.g. '.') is reported by DirCache in a
runtime exception. In server context this becomes a 500 instead of a user error.

Wrap the runtime invalid path exception into a checked ManifestErrorException that
caller can handle.

Change-Id: I61a2104922765506ae232334891057bb06141d97
2022-08-23 12:10:27 -07:00
Han-Wen Nienhuys 25aceffdc5 ApplyCommand: fix ApplyResult#updatedFiles
On executing a copy, mark the destination as updated.

On executing a rename, mark both source and destination as updated.

Change-Id: Ied5b9b0e5a14eac59a06cdd0961e25e143f50ff0
2022-08-23 09:23:23 -04:00
Ronald Bhuleskar 5747bba48b Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.

Change-Id: Ie127cb710d004079e9f53a5802130afdb49a7de1
2022-08-17 13:50:34 -04:00
Ronald Bhuleskar 6297491e8a Adds FilteredRevCommit that can overwrites its parents in the DAG.
Change-Id: I1ea63a3b56074099688fc45d6a22943a8ae3c2ae
2022-08-16 21:32:25 -07:00
Thomas Wolf b255eb0fb6 DirCacheCheckout: load WorkingTreeOptions only once
Previous code loaded the WorkingTreeOptions afresh for every single
file being checked out. This checked the git config (all three files,
repo, user and system config) for having been modified every time.

These checks can be costly, for instance on Windows, or if one of the
three config files is not on a local disk, or on an otherwise slow
storage.

Improve this by loading the options and thus checking the git config
only once before the checkout.

Bug: 579715
Change-Id: I21cd5a808f9d90b5ca2d022f91f0eeb8ca26091c
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-08-14 21:33:19 +02:00
Thomas Wolf d9c0629ffe Fix adding symlinks to the index when core.symlinks=false
With core.symlinks=false, symlinks are checked out as plain files.
When such a file is re-added to the index, and the index already
contains a symlink there, add the file as a symlink. Previous code
changed the index entry to a regular file.

Bug: 580412
Change-Id: I5497bedc3da89c8b10120b8077c56bc5b67cb791
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-08-13 15:14:15 +02:00
Han-Wen Nienhuys 3b4a636183 ResolveMerger: add coverage for inCore file => directory transition
Change-Id: I6c3473932eb418a036f5943c78e619184559ef3a
2022-08-08 18:38:23 +02:00
Terry Parker 1c7b4a580f Merge "Revert "Option to pass start RevCommit to be blamed on to the BlameGenerator."" 2022-08-03 18:38:05 -04:00
Terry Parker 7f89b0fa16 Merge "Revert "Adds FilteredRevCommit that can overwrites its parents in the DAG."" 2022-08-03 18:37:57 -04:00
Ronald Bhuleskar e5e46bc536 Revert "Adds FilteredRevCommit that can overwrites its parents in the DAG."
This reverts commit ceb51a5e0e.

Reason for revert: The change in https://git.eclipse.org/r/c/jgit/jgit/+/194354 broke the egit test [1]. Calling c.getShortMessage() causes an NPE.

[1] https://ci.eclipse.org/egit/job/egit.gerrit/2711/

Change-Id: I411565b6eaa0bbb562cc1c8a355942ff09fd29bc
2022-08-03 17:16:34 -04:00
Ronald Bhuleskar 3d2f087932 Revert "Option to pass start RevCommit to be blamed on to the BlameGenerator."
This reverts commit 59e8bec6e7.

Reason for revert: The change in https://git.eclipse.org/r/c/jgit/jgit/+/194354 broke the egit test [1]. Calling c.getShortMessage() causes an NPE.

[1] https://ci.eclipse.org/egit/job/egit.gerrit/2711/

Change-Id: Iaf5feb35f4bb4c3487b04be15d1fe11376975523
2022-08-03 17:14:18 -04:00
Thomas Wolf 8184683f7e CleanCommand: fix prefix matching
String.startsWith() is not a valid test for file path prefixes:
directory "a" is _not_ a prefix of a file "ab", only of "a/b".

Add a proper Paths.isEqualOrPrefix() method and use it in CleanCommand.

Bug: 580478
Change-Id: I6863e6ba94a8ffba6561835cc57044a0945d2770
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-08-03 07:42:27 +02:00
Ronald Bhuleskar 59e8bec6e7 Option to pass start RevCommit to be blamed on to the BlameGenerator.
This can allow passing a FilteredRevCommit which is the filtered list of
commit graph making it easier for Blame to work on. This can
significantly improve blame performance since blame can skip expensive
RevWalk.

Change-Id: I5dab25301d6aef7df6a0bc25a4c553c730199272
2022-08-02 21:02:33 -04:00
Ronald Bhuleskar ceb51a5e0e Adds FilteredRevCommit that can overwrites its parents in the DAG.
Change-Id: I2df9843dde0f589f5fea6cedaaff52e313eea6de
2022-08-02 20:25:58 -04:00
Ronald Bhuleskar 61b4d105e4 Add the ability to override parents on RevCommit.
This makes RevCommit extensible to allow having different structure of
child-parent relationship. This change is a pre-requsite for having a
FilteredRevCommit that overrides parents from the RevCommit. That then
provides a cheaper way to walk over a subset of RevCommits instead of
an expensive way that applies filters while walking over selected
commits. Useful with Blame which works on a single file and that can be
made performant, if we know all the commits needed by the Blame
algorithm. So Blame algorithm can avoid walking over finding what
commits to blame on.

This change makes parents field on RevCommit private and exposes it
thrrough overrideable methods such as getParents, getParent at index,
getParentCount and setParents. All other files other than RevCommit are
updating the usages of accessing RevCommits parents.

Change-Id: I2d13b001c599cc4ebc92d1ab6e07b07acb3b7fe5
2022-08-02 10:50:16 -07:00
Dmitrii Filippov 5af93628d4 Fix BUILD rules for FilteredRenameDetectorTest
The FilteredRenameDetectorTest was added in [1], but bazel build rules
were not updated.

[1] https://git.eclipse.org/r/c/jgit/jgit/+/194200

Change-Id: I7fc713e19b4768176eb84e9768137431b33a805e
2022-08-01 13:31:13 -04:00
Han-Wen NIenhuys 688c83d93a Merge "NameConflictTreeWalk: respect git order on multi-tree iteration" 2022-08-01 11:25:10 -04:00
Robin Müller 207dd4c938 Fetch: add support for shallow
This adds support for shallow cloning. The CloneCommand and the
FetchCommand now have the new methods setDepth, setShallowSince and
addShallowExclude to tell the server that the client doesn't want to
download the complete history.

Bug: 475615
Change-Id: Ic80fb6efb5474543ae59be590ebe385bec21cc0d
2022-07-31 14:08:47 +02:00
Dmitrii Filippov 8584ac7048 NameConflictTreeWalk: respect git order on multi-tree iteration
The NameConflictTreeWalk class is used in 3-way merge for iterating over
entries in 3 different commits. The class provides information about a
current entry and a state of the entry in commits (e.g entry is file,
entry is directory, entry is missing). In rare cases, the tree walker
can mix information about entries with different name.

The problem appears, because git uses unusual sorting order for
files. Example (this is a simplified real-life example):
Commit 1:
* gradle.properties - file
* gradle - directory (with nested files)
*   gradle/file - file in gradle directory
Commit 2:
* gradle.properties - file
* no entry with the name gradle
Commit 3:
* gradle.properties - file
* gradle - file
Here the names are ordered like this:
"gradle" file <"gradle.properties" file < "gradle/file" file.

NameConflictTreeWalk iterator already have code for processing
git sorting order, however in the example above the code doesn't
work correctly. Before the fix, NameConflictTreeWalk returns:
#next()
"gradle - directory" | "gradle.properties" | "gradle - file" - which is
wrong. The expected result is
#next()
"gradle - directory | MISSED_FILE | "gradle - file"
#next()
"gradle.properties"|"gradle.properties"|"gradle.properties"

Ensure that the "matches" field of tree iterators (which contains the
current path) is kept in sync in the case above.

Change-Id: Ief5aa06d80b358f4080043c8694aa0fd7c60045b
Signed-off-by: Dmitrii Filippov <dmfilippov@google.com>
2022-07-29 19:35:22 +02:00
Matthias Sohn 911b4e0d82 Update Orbit to S20220726152247 and bouncycastle to 1.71
Change-Id: I036b9029d575e66ab67839468e5d2a8a7dd85c83
2022-07-27 09:52:43 +02:00
Thomas Wolf a1ce9063fb Fix the handling of .git/info/exclude and core.excludesFile
The RootIgnoreNode in a WorkingTreeIterator must _not_ add the rules
from .git/info/exclude or from the file designated by git config
core.excludesFile to the list of rules read from the root .gitignore.
These really must be separate nodes in a hierarchy, otherwise the
precedence rules from [1] are violated and the outcome is not the
same as in C git.

[1] https://git-scm.com/docs/gitignore

Bug: 580381
Change-Id: I57802ba7bbbe4f183504c882b6c77a78cc3a9b99
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-07-15 21:00:16 +02:00
Andrey Loskutov ffcb951385 Remove version constraints from org.tukaani.xz package
Fixes compilation errors in org.eclipse.jgit.test with 4.25 platform

Bug: 580370
Change-Id: I9b7cb21bcbb87d14176fc0675bf03d20f81fa009
2022-07-13 09:46:11 +02:00
Matthias Sohn f6935d8cd2 Merge branch 'stable-6.2'
* stable-6.2:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: Ia2280b848cea7bdfa5bc48356484e040d159845d
2022-07-06 17:02:58 +02:00
Matthias Sohn 34600e3ca2 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: Ib3ff6a038e1b92a931a7f2624c19bbefd8cb95fd
2022-07-06 17:02:04 +02:00
Matthias Sohn d01376106a Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: I08734ee2c8f3296d908da6a29d53ed87c4b48eb2
2022-07-06 17:00:42 +02:00
Matthias Sohn 9f7d77b608 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  UploadPack: don't prematurely terminate timer in case of error
  Do not create reflog for remote tracking branches during clone
  UploadPack: do not check reachability of visible SHA1s
  Add missing package import javax.management to org.eclipse.jgit

Change-Id: I6db0a4d74399fde892eeec62efd2946f97547a5d
2022-07-06 16:59:30 +02:00
Simeon Andreev 1a364c49ec JGit blame very slow for large merge commits that rename files
Adjusted BlameGenerator to filter rename detection with the blame path.
This reduces the running time of the blame computation significantly,
for repositories with massive commits involving renames.

The filtered rename detection is made (internally) available with:
org.eclipse.jgit.internal.diff.FilteredRenameDetector

Bug: 578900
Change-Id: I6580004e81102d685081b8180da1587a35073d36
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
2022-07-01 16:10:13 +02:00
Luca Milanesio 4bb4693633 Do not create reflog for remote tracking branches during clone
When using JGit on a non-bare repository, the CloneCommand
it previously created local reflogs for all branches including remote
tracking ones, causing the generation of a potentially large
number of files on the local filesystem.

The creation of the remote-tracking branches (refs/remotes/*) during
clone is not an issue for the local filesystem because all of them are
stored in a single packed-refs file. However, the creation of a large
number of ref logs on a local filesystem IS an issue because it
may not be tuned or initialised in term of inodes to contain a very
large number of files.

When a user (or a CI system) performs the CloneCommand against
a potentially large repository (e.g., millions of branches), it is
interested in working or validating a single branch or tag and is
unlikely to work with all the remote-tracking branches.
The eager creation of a reflogs for all the remote-tracking branches is
not just a performance issue but may also compromise the ability to
use JGit for cloning a large repository.

The behaviour implemented in this change is also consistent with the
optimisation done in the C code-base [1].

We differentiate between clone and fetch commands using --branch
<initialBranch> option, that is only available in clone command,
and is set as HEAD per default.

[1] 58f233ce1e

Bug: 579805
Change-Id: I58d0d36a8a4ce42e0f59b8bf063747c4b81bd859
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2022-06-25 12:09:01 +01:00
David Ostrovsky 87391ccee9 Fix DefaultCharset bug pattern flagged by error prone
See more details in: [1].

[1] https://errorprone.info/bugpattern/DefaultCharset
Change-Id: I3de0be57a2d74490a5b4e66801e9767b38f13bf9
2022-06-17 09:44:14 +02:00
Matthias Sohn c67d951a0a Merge branch 'stable-6.2'
* stable-6.2:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: I13514595f3a710b0d6e3c2de62f6508b950d22f9
2022-06-15 17:40:31 +02:00
Matthias Sohn 58f5302e1d Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: Id4965aacd4e2ea1e8575a2c1bd4845729db6049a
2022-06-15 17:39:52 +02:00
Matthias Sohn d0bc2b544a Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: Ie9c38ab8033fe1283e8b444b6acd3f4298062bf3
2022-06-15 16:32:08 +02:00
Matthias Sohn d961bb6502 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Prepare 5.13.2-SNAPSHOT builds
  JGit v5.13.1.202206130422-r
  AmazonS3: Add support for AWS API signature version 4

Change-Id: Ibd663a1d874d1aac274abc3dd44354fd99f64c39
2022-06-15 16:31:38 +02:00
Matthias Sohn db074a1352 Prepare 5.13.2-SNAPSHOT builds
Change-Id: I4862e5d80a7d95a1a119d06306e3f6927445d1d3
2022-06-14 00:41:18 +02:00
Matthias Sohn b34961a493 JGit v5.13.1.202206130422-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ife74d64e8171c68dbf08271492c0ac852a6dc51c
2022-06-13 10:22:43 +02:00
eric.steele e9a5430c25 AmazonS3: Add support for AWS API signature version 4
Updating the AmazonS3 class to support AWS Signature version 4 because
version 2 is no longer supported in all AWS regions. The version can be
selected with the new 'aws.api.signature.version' property (defaults to
2 for backwards compatibility). When set to '4', the user must also
specify the AWS region via the 'region' property. The 'region' property
must match the region that the 'domain' property resolves to.

Bug: 579907
Change-Id: If289dbc6d0f57323cfeaac2624c4eb5028f78d13
2022-06-13 09:44:23 +02:00
Matthias Sohn a9f535e104 Prepare 6.2.1-SNAPSHOT builds
Change-Id: I4a0f3919ff43a3b9fafa85b8ecec2d760b7eb161
2022-06-07 22:31:32 +02:00
Matthias Sohn 236b45f95b JGit v6.2.0.202206071550-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ib4ae85a0cabcc9cb867f2c85034d72fb676a500a
2022-06-07 21:49:30 +02:00
Matthias Sohn 8137e5e746 Prepare 6.3.0-SNAPSHOT builds
Change-Id: I092fdd2c35d85bf35e3ef700aa7078e6d304d977
2022-06-07 01:02:10 +02:00
Matthias Sohn cc862f843a Prepare 6.2.0-SNAPSHOT builds
Change-Id: Id2ad8d5b561620723b2c52c86909321d628de12f
2022-06-06 22:08:59 +02:00
Andrey Loskutov 2ef2a3562e Merge branch 'master' into stable-6.2
* master:
  Adapt diff- and merge tool code for PGM and EGit usage
  Teach JGit to handle external diff/merge tools defined in
.gitattributes

Change-Id: I3aefc14160caaac859bd3548460dd755ebe42fc5
2022-06-03 15:51:54 +02:00
Andre Bossert c32694e5ae Teach JGit to handle external diff/merge tools defined in .gitattributes
Adds API that allows UI to find (and handle) diff/merge tools, specific
for the given path. The assumption is that user can specify file type
specific diff/merge tools via gitattributes.

Bug: 552840
Change-Id: I1daa091e9afa542a9ebb5417853dff0452ed52dd
Signed-off-by: Mykola Zakharchuk <zakharchuk.vn@gmail.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2022-06-02 10:36:39 +02:00
Andre Bossert ff77d412a9 Adapt diff- and merge tool code for PGM and EGit usage
see: https://git-scm.com/docs/git-mergetool

* DiffTools and MergeTools
  * store FS, gitDir and workTree for usage without
git repository (for EGit preferences)
  * add getUserDefinedToolNames() and getPredefinedToolNames()
  * replace getToolNames() with getAllToolNames() that combines the two
lists and put default tool name (diff.tool or merge.tool) as first
element (for EGit preferences)
* FileElement: refactoring of getFile() and friends to have midName
(LOCAL, REMOTE etc.) always added to the temp file name (also for EGit)
* FileElement: added directory attribute that is used in getFile() to
return path with workDir as parent
* DiffTool and MergeTool
  * added errw.flush(), because sometimes stderr is not printed in case
of die()
  * print e.getMessage() always to stderr
* Moved toolname and prompt logic into managers
* Exported internal packages required for egit.ui

Bug: 356832
Change-Id: I71e7f4dc362169a7612ca4f6546a021bc4b2b5f4
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Tim Neumann <Tim.Neumann@advantest.com>
2022-06-02 10:36:31 +02:00
Matthias Sohn 7b1ade51d2 JGit v6.2.0.202206011217-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Id9639b466583d8c373ef700402cb685ce4a8ee15
2022-06-01 18:17:31 +02:00
Andre Bossert 973e955ead Add availability check of pre-defined tools
see: https://git-scm.com/docs/git-difftool
see: https://git-scm.com/docs/git-mergetool

* now all available tools are printed with "--tool-help"
* if no diff.tool or merge.tool is defined the first available
pre-defined tool is used

TODO:
- add mergetools to difftools --> extra change or merge to this
- return the exit-code of the tool to jgit / java runtime

Bug: 356832
Change-Id: I20fb04e71ced981f5625020f461bbac24e6cec70
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2022-06-01 14:23:48 +02:00
Andre Bossert e81085944f Add filtering with help of DirCacheCheckout.getContent()
see: https://git-scm.com/docs/git-mergetool

* refactoring of content (FileElement) handling
* now the temporary files are already filled with filtered content in
the calling classes (PGM), that can be used with EGit content too

TODO:
 * keep the temporaries when no change detected and the user answers no
to the question if the merge was successful

Bug: 356832
Change-Id: I86a0a052d059957d4d152c1bb94c262902c377d2
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2022-05-30 13:28:32 +02:00
Matthias Sohn 29dce2e7be Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Remove stray files (probes or lock files) created by background threads

Change-Id: Iab21b9776c2ebecd4cad5d015028cadd4b6541db
2022-05-27 16:22:09 +02:00
Matthias Sohn 9612aae885 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Remove stray files (probes or lock files) created by background threads

Change-Id: I7af1355a77f14995118145162f6bb8a4f1755f2b
2022-05-27 16:20:28 +02:00
James Z.M. Gao d67ac798f1 Remove stray files (probes or lock files) created by background threads
NOTE: port back from master branch.

On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.

Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.

Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.

Bug: 579445
Change-Id: Iaee2301eb14e6201406398a90228ad10cfea6098
2022-05-27 01:20:16 +02:00
Andre Bossert 8573435635 Add command line support for "git mergetool"
see: https://git-scm.com/docs/git-mergetool
see: https://git-scm.com/docs/git-config

* add command line support for "git mergetool"
  * add option handling for "--tool-help", "--tool=<mytool>",
"--[no-]prompt",  "--[no-]gui"
  * handle prompt
  * add MergeTools
  * add pre-defined mergetools
  * print merge actions --> no execute, will be done later

Bug: 356832
Change-Id: I6e505ffc3d03f75ecf4bba452a25d25dfcf5793f
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2022-05-25 13:52:04 +02:00
Andre Bossert 24171b05f0 Add config reader for user-defined mergetools
see: https://git-scm.com/docs/git-mergetool
see: https://git-scm.com/docs/git-config

* add config reader for user-defined mergetools
  * merge.tool
  * merge.guitool
  * mergetool.prompt
  * mergetool.keepBackup
  * mergetool.keepTemporaries
  * mergetool.writeToTemp
  * mergetool.<tool>.path
  * mergetool.<tool>.cmd
  * mergetool.<tool>.trustExitCode

Bug: 356832
Change-Id: Ic4f07376630713d8e06cbad284d9c72c9ecc0405
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
2022-05-25 13:50:50 +02:00
Andre Bossert b63c2f39a1 Add difftool compare feature (execute external tool)
see: http://git-scm.com/docs/git-difftool

* add CommandExecutor that handles tool execution with help of "jgit.FS"
  * it handles tool execution with temporary created "command file" -->
for for all "command interpreters" and parameters with spaces etc.
  * using of external bash.exe at Windows (MinGW) if shell-script is
used as difftool command. It can be enabled with parameter
"jgit.usemsys2bash=auto" that checks if command contains ".sh" or
enabled / disabled with "jgit.usemsys2bash=true|false"
* added special handling for empty files (e.g. deleted, added etc.) that
are named "/dev/null"
* added creation and deletion of temporary files needed for compare
* added own Exception class for reporting to pgm / command line / EGit
* added prompt option handling before executing difftool
* reworked trustExitCode option for specific difftool and override for
all difftools from config and command line
* tested with command line options "--[no]-trust-exit-code",
"--tool=<toolname>", "--[no]-gui", --[no]-prompt
* ContentSource
  * added close() methods to close / cleanup used resources
(like ObjectReader TreeWalk etc.)
  * added isWorkingTreeSource() methods to check if file can be used
from working tree instead of copy from "ObjectLoader / ObjectReader" to
temporary file (fixes "difftool <commit> <commit>")

Bug: 356832
Change-Id: I5462fb6dbe4ecfd9da7c74117fce4070bbfd4d7a
Signed-off-by: Andre Bossert <andre.bossert@siemens.com>
Signed-off-by: Simeon Andreev <simeon.danailov.andreev@gmail.com>
2022-05-25 13:23:50 +02:00
James Z.M. Gao 88d5f51e61 Remove stray files (probes or lock files) created by background threads
On process exit, it was possible that the filesystem timestamp
resolution measurement left behind .probe files or even a lock file
for the jgit.config.

Ensure the SAVE_RUNNER is shut down when the process exits (via
System.exit() or otherwise). Move lf.lock() into the try-finally
block when saving the config file.

Delete .probe files on JVM shutdown -- they are created in daemon
threads that may terminate abruptly, not executing the "finally"
clause that normally removes these files.

Bug: 579445
Change-Id: Ie27aca1fdaddfa487ebe072c3913d78355910df8
2022-05-15 19:41:24 +02:00
yunjieli eca101fc05 Fetch: Introduce negative refspecs.
Implement negative refspecs in JGit fetch, following C Git. Git
supports negative refspecs in source only while this change supports
them in both source and destination.

If one branch is equal to any branch or matches any pattern in the
negative refspecs collection, the branch will not be fetched even if
it's in the toFetch collection.

With this feature, users can express more complex patterns during fetch.

Change-Id: Iaa1cd4de5c08c273e198b72e12e3dadae7be709f
Sign-off-by: Yunjie Li<yunjieli@google.com>
2022-04-13 10:21:20 -07:00