Commit Graph

8863 Commits

Author SHA1 Message Date
Thomas Wolf 513c7318de CommitCommand: commit message cleanup
Use CommitConfig.CleanupMode to implement git commit --cleanup. Add
setters for the clean-up mode, the comment character, and for the
default default clean-up mode.

Behavior of existing client code is unchanged as the default clean-up
mode is set to "verbatim". To use git config defaults, one can call
setCleanupMode(CleanupMode.DEFAULT). The default comment character
is hard-coded as '#' for now, as in other parts of JGit. Implementing
full support for core.commentChar shall be done in a separate change.

Bug: 553065
Change-Id: I470785e464a762d3f409f163f1cbdbb98dd81aaf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-31 00:42:35 +01:00
Thomas Wolf 318a25f0e6 Provide git config commit.cleanup
Add an enumeration for the possible values, and a method to resolve the
"default" value. Give CommitConfig a static method to process a text
according to a given clean-up mode and comment character.

(The core.commentChar is not yet handled by JGit; it's hard-coded as #.)

Bug: 553065
Change-Id: If6e384522275f73b713fbc29ffcaa1753c239dea
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-31 00:42:35 +01:00
Nail Samatov 2b01ac3389 [test] Fix closing of test repositories
Fix tests failing on Windows because Repository instance is created but
not closed on tear down.

Fix repositories closed twice, except in tests that test this behavior
explicitly.

Name the temporary directories the tests run in after the test method;
that makes it easier to figure out in which tests repositories are
closed twice if it should occur again in the future.

Bug: 550111
Change-Id: I9398b58f0f36d2c29236d2a9a8599117d9083980
Signed-off-by: Nail Samatov <sanail@yandex.ru>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:16:02 -05:00
Thomas Wolf 8633ea4f07 [test] DirCacheCheckoutTest: fix test expectation for eol=native
With eol=native, we expect LF on Unixes, and CR-LF on Windows. One test
didn't account for this and always expected LF, and thus failed on
Windows.

Bug: 550111
Change-Id: I69354ac691c464d1b6003812ddb4510c5ab5e77a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 22:47:38 +01:00
Thomas Wolf 4bb87a957f ObjectWalk: close ObjectReader on close() if needed
If the walk is created via ObjectWalk(Repository), it creates a new
ObjectReader. This reader was closed only on dispose(). If such an
ObjectWalk was used in a try-with-resource statement the reader might
not get closed.

Bug: 578458
Change-Id: I1be31829dc466530f23006a53c29b657fd5fb410
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 22:18:30 +01:00
Thomas Wolf d4d30bc716 [test] Fix a Windows-only test in CheckoutCommandTest
Test.txt should not be in the "removed" list if it can't be deleted
but only in the "not deleted" list. The test was wrong.

Bug: 550111
Change-Id: I3ecede4278014c15015c8c24089647fa3db3742f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 21:43:13 +01:00
Thomas Wolf f8eb530711 [test] Fix ApplyCommandTest for Windows
Some tests checked whether or not a file is executable via
java.io.File.canExecute(). But that always returns true on Windows.

Use FS.DETECTED.canExecute() instead, and guard all such assertions to
run only of file systems that do support the "execute" flag.

Bug: 550111
Change-Id: Iacb9e414b612359fcecb61312c3dfb830801cd36
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 21:28:03 +01:00
Thomas Wolf 002e13f0f6 [test] Fix OpenSshConfigFileTest for Windows
The tests assumed that a path like "/tmp" was an absolute path, and
also compared against strings with forward slashes. On Windows, "/tmp"
is not an absolute path and thus resolved against the current directory,
and the separator is a backslash.

Change the tests to use ~/ notation, and test paths resolved against
the (mocked) user home directory. That way, the tests are independent
of the file system used.

Bug: 550111
Change-Id: I1c31608ca83c8d8586256d1586a792e4a33cfaa4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 20:59:26 +01:00
Thomas Wolf 8bca5245e0 BinaryHunkInputStream: accept CR-LF
Let's be lenient and accept hunk lines terminated by CR-LF, too, not
just lines terminated by LF.

Bug: 550111
Change-Id: I7f796df666300ab56cc6c07f22eda45fbf4c941e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:58:34 +01:00
Thomas Wolf 0fb5f47d2e sshd: Add README.md for SSH agents
Explain SSH agent protocols, what transports are available and how to
choose them in ~/.ssh/config. For Windows, add some information on
which commonly used SSH agents can be used.

Change-Id: I0b08a95654fd76643512606edb1ed74d9980aa85
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:47 +01:00
Thomas Wolf 8e9a42b7c0 sshd: support the ConnectTimeout ssh config
Parse the value from the ssh config and if set use it when connecting.

Change-Id: I85b44c9468a5027602375706612c46ea7a99b2bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:47 +01:00
Thomas Wolf f41929708e sshd: Skip unknown keys from the SSH agent
An SSH agent might contain keys that Apache MINA sshd cannot handle.
Pageant for instance can contain ed448 keys, which are not implemented
in OpenSSH or in Apache MINA sshd.

When an agent delivers such keys, simply skip (and log) them. That way,
we can work with the remaining keys. Otherwise a single unknown key in
the agent would break pubkey authentication.

Change-Id: I3945d932c7e64b628465004cfbaf10f4dc05f3e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:46 +01:00
Thomas Wolf b73548bc4c sshd: support the AddKeysToAgent ssh config
Add parsing of the config. Implement the SSH agent protocol for adding
a key. In the pubkey authentication, add keys to the agent as soon as
they've been loaded successfully, before even attempting to use them
for authentication. OpenSSH does the same.

Bug: 577052
Change-Id: Id1c08d9676a74652256b22281c2f8fa0b6508fa6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:46 +01:00
Thomas Wolf 68bd2c1462 sshd: handle "IdentityAgent SSH_AUTH_SOCK" in ssh config
OpenSSH has (for legacy reasons?) the option of specifying the default
environment variable directly, instead of using ${SSH_AUTH_SOCK}. Make
sure the plain variable name is not taken as a relative path name.

Bug: 577053
Change-Id: If8f550dffc43887254f71aa0b487c50fa14d0627
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:46 +01:00
Thomas Wolf e0281c5adb sshd: Connector for the Win32-OpenSSH SSH agent
Win32-OpenSSH uses a named Windows pipe for communication. Implement
a connector for this mechanism using JNA. Choose the appropriate
connector based on the setting of the 'identityAgent' parameter.

Bug: 577053
Change-Id: I205f07fb33654aa18ca5db92706e65544ce38641
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:46 +01:00
Thomas Wolf 071084818c sshd: handle IdentitiesOnly with an SSH agent
If an SSH agent is used but "IdentitiesOnly yes" is set, only those
keys from the agent that correspond to one of the keys explicitly given
via an IdentityFile directive are to be used.

Implement this by filtering the list of keys obtained from the agent
against the list of IdentityFiles, each entry suffixed with ".pub".
Load the public keys from these files, and ignore all other keys from
the agent. Keys without ".pub" file are also ignored.

Apache MINA sshd has no operation to load only the public key from a
private key file, so we have to rely on *.pub files.

Bug: 577053
Change-Id: I75c2c0b3ce35781c933ec2944bd6da1b94f4caf9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:45 +01:00
Thomas Wolf 4efc6a396a sshd: support IdentityAgent config
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.

Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:44 +01:00
Nail Samatov ad098b3b85 LFS: Fix error occurring during delete branch
Fix TransportException occurring when deleting
a branch and push that change to remote repository
if BuiltinLFS is used to work with repository.
When finding LFS pointers in RemoteRepUpdate,
LfsPrePushHook fails to open ObjectReader
with new object id equal to ObjectId.zeroId().
If update is a deleting update (new object id is zero id),
we can assume that this update doesn't contain LFS Pointer
and we can skip step with extracting LFS pointer for that
RemoteRefUpdate.

Bug: 578313
Change-Id: Ic4367978338b8234d39d9af0d9674490f79fc22d
Signed-off-by: Nail Samatov <sanail@yandex.ru>
2022-01-27 20:55:58 +01:00
Ivan Frade 27e554e465 Merge "PackOutputStream: Extract cancellation and digest to superclass" 2022-01-27 11:42:54 -05:00
kylezhao 1a86c1044d PackOutputStream: Extract cancellation and digest to superclass
Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.

Pull up that functionality to a superclass, so it can be reused.

Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-01-27 19:21:31 +08:00
David Ostrovsky 75e7d08480 Bazel: Include bazel resource configuration file for RBE build
In I988f61e34 tools/remote-bazelrc was added, but this resource file
wasn't imported from main resource file workspace/.bazelrc.

Change-Id: Ie0c2f128fbccaa6f9e99ad29a3a76ac294f3cd29
2022-01-22 20:43:56 +01:00
Ivan Frade 076ecf8ded Merge "DFS block cache: report index load and evict stats" 2022-01-21 14:59:30 -05:00
Alina Djamankulova b536dbdb9b DFS block cache: report index load and evict stats
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.

Add a custom consumer to report loading events and eviction events when
enabled.

Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
2022-01-21 11:27:31 -08:00
Antonio Barone 788f439c0e Fix warning: The value of the parameter otp is not used
Silence warning by removing unused argument to the beginCopyAsIs()
method.

Change-Id: I94e7ff1c61cf8b03752de2974baa24b9c061c163
2022-01-20 12:37:16 +01:00
David Ostrovsky 969601c742 Bazel: Add RBE support
The Bazel build can be used with Google's Remote Build Execution.

This needs the following setup steps:

  $ gcloud auth application-default login
  $ gcloud services enable remotebuildexecution.googleapis.com  --project=${PROJECT}

Create a worker pool. The instances should have at least 4 CPUs each
for adequate performance.

  $ gcloud alpha remote-build-execution worker-pools create default \
    --project=${PROJECT} \
    --instance=default_instance \
    --worker-count=50 \
    --machine-type=e2-standard-4 \
    --disk-size=200

To use RBE, execute

  $ bazel test --config=remote \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    //...

Change-Id: I988f61e342dab2136d8752ace945a4ed91a4189a
2022-01-20 06:43:13 -04:00
David Ostrovsky be732c16ab Bazel: Switch to using toolchain resolution for java rules
Bump Bazel version to release 5.0.0. In this new Bazel release,
--incompatible_use_toolchain_resolution_for_java_rules is flipped, that
means that the build must be adapted to toolchain resolution.

Specification toolchain resolution for java rules is here: [1]. Main
tracking Bazel issue is here: [2].

Given that new Bazel release also added support for remote JDK 17, add
experimental support for building with remote JDK 17 to produce major
byte code version 61.

Test Plan:

To build with remote JDK 11 run:

  $ bazel build :all

To build with remote JDK 17 run:

  $ bazel build --config java17 :all

[1] https://docs.google.com/document/d/1MVbBxbKVKRJJY7DnkptHpvz7ROhyAYy4a-TZ-n7Q0r4/edit?usp=sharing
[2] https://github.com/bazelbuild/bazel/issues/7849

Change-Id: I8d26dff722c6677ea7642913b61e416b23ea9041
2022-01-20 06:40:51 -04:00
David Ostrovsky c812947962 Bazel: Simplify java 11 toolchain definition
Also remove unused JDK9_JVM_OPTS constant.

Change-Id: I0f45ca8312a2a8c3aee3855e3ef3afa95808b4ad
2022-01-20 00:54:15 -05:00
David Ostrovsky bf3298478a Bazel: Format build files with buildifier
Change-Id: I1ff38237058d11a3f61acc881c504382ff4b7e08
2022-01-20 00:54:01 -05:00
David Ostrovsky 9a16e9a20a Bazel: Remove JDK 15 toolchain definition
Java 15 is outdated and the recent Bazel releases added support to
JDK 17.

Change-Id: I68c1f5b5132d844f07bfefc22f81ecf0a19df910
2022-01-20 00:53:51 -05:00
David Ostrovsky 8227098d7d Bazel: Remove version check
Recent Bazel releases perform Bazel version check, based on
.bazelversion file.

Change-Id: I92d5daaef8ecbd011517d314564c0f5492c8ea75
2022-01-20 00:53:42 -05:00
David Ostrovsky 132ba05c82 ExternalToolTest: Rename class name to usual name for test cases
Change-Id: I00e4a5e73f530b7ac671d09b9b447bb24d6419fa
2022-01-20 06:52:13 +01:00
David Ostrovsky f8a96e1a0b Bazel: Add missing java packages to error_prone package group
Change-Id: I709a3cd2639e9209b2c92c700e04ab4c90ed0524
2022-01-20 06:49:25 +01:00
David Ostrovsky 51b909d9e0 Adapt junit_tests invocation to removal of resource_jars attribute
In upcoming Bazel release 5.0 resource_jars attribute was removed: [1].
Replace resource_jars with runtime_deps as recommended in the release
notes.

[1] https://github.com/bazelbuild/bazel/issues/13221

Change-Id: Iab834d647cf63259ca182adc1f862b6001db0cce
2022-01-19 18:39:32 +01:00
Matthias Sohn 56f45e36dc Update orbit to I20220111151929
and update
- com.google.gson to 2.8.9.v20220111-1409
- org.bouncycastle.bcpg to 1.70.0.v20220105-1522
- org.bouncycastle.bcpkix to 1.70.0.v20220105-1522
- org.bouncycastle.bcprov to 1.70.0.v20220105-1522
- org.bouncycastle.bcutil to 1.70.0.v20220105-1522

Change-Id: I7d9159e0e8524459a11da84cb2db709cb1f0823d
2022-01-19 09:54:37 +01:00
Matthias Sohn 685cc54045 Update org.apache.maven.wagon:wagon-ssh to 3.5.1
Change-Id: I90807ef1ae065a8a8559e445c0aa7e2abdd2b9a8
2022-01-19 09:54:37 +01:00
Matthias Sohn 32cc4c56a6 Update org.eclipse.jdt:ecj to 3.28.0
Change-Id: I7573d8cfd07038bca4a7ed9e122e5dcf666632f8
2022-01-19 09:54:37 +01:00
Matthias Sohn 73c398a95f Update org.osgi:org.osgi.core to 6.0.0
Change-Id: If93e0e8769fcbb92befa1f159f48b0ac6b6f3d24
2022-01-19 09:54:37 +01:00
Matthias Sohn 4aca4e0a31 [errorprone] Fix implicit use of platform default charset
Change-Id: I42f893c4248d8b00bf4aaf59cd528c43940f99e2
2022-01-19 09:54:37 +01:00
Matthias Sohn 8e306e7a62 [bazel] Fix build of ExternalDiffToolTest
Change-Id: Ida4721142f49c86aa3f05aeffc6ca0aef40939f9
2022-01-19 09:54:37 +01:00
Matthias Sohn 9a57ae4707 Let org.eclipse.jgit.pgm use BooleanTriState
Change-Id: I575cd5e1ecd326514a8bb4c36a86a2dc71ba5457
2022-01-19 09:54:37 +01:00
Matthias Sohn 5f556588a9 Merge branch 'stable-6.0'
* stable-6.0:
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  [errorprone] Fix InfiniteRecursion error in RecordingLogger
  [errorprone] Suppress Finally error in ObjectDownloadListener
  [errorprone] Fix implicit use of default charset in FileBasedConfigTest
  [errorprone] Suppress FutureReturnValueIgnored in FileRepository#autoGc

Change-Id: I08d58c8f3f04e3a920da43b5fb252b1654c2b33c
2022-01-19 09:53:21 +01:00
Matthias Sohn 9b28f43cf1 Merge changes I6a22f37f,I092389e4,I20af1d8d,I83332efc into stable-6.0
* changes:
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  [errorprone] Fix InfiniteRecursion error in RecordingLogger
  [errorprone] Suppress Finally error in ObjectDownloadListener
  [errorprone] Fix implicit use of default charset in FileBasedConfigTest
2022-01-19 03:50:59 -05:00
Matthias Sohn b55c224ef3 Merge "[errorprone] Suppress FutureReturnValueIgnored in FileRepository#autoGc" into stable-6.0 2022-01-19 03:43:27 -05:00
Matthias Sohn d017a655df Merge branch 'stable-6.0'
* stable-6.0:
  UploadPack v2 protocol: Stop negotiation for orphan refs
  Complete update to servlet api 4.0.0

Change-Id: I55ab6e8fd4a76e4313e37b12f9fc5d5e4b84a681
2022-01-18 18:09:03 +01:00
Matthias Sohn de1abd3237 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: I6a9ed8338ffbf5363e48d640a2c4209e4e503549
2022-01-18 18:07:59 +01:00
Matthias Sohn 2cc0009737 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: Ib43068c32d9cb8effe4b873396391dc3c9197a6e
2022-01-18 17:51:14 +01:00
Matthias Sohn 1e59cabc08 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: I5db432bd416cfa8d3dd295bdce63e31d5f160a8a
2022-01-18 17:49:03 +01:00
Matthias Sohn b33133497f [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.

FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.

Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
2022-01-18 16:11:58 +01:00
Matthias Sohn a268faf729 [errorprone] Fix InfiniteRecursion error in RecordingLogger
Change-Id: I092389e428232a4fe7613d846c288d285ae9102c
2022-01-18 16:11:57 +01:00
Matthias Sohn 6fde4d3a6c [errorprone] Suppress Finally error in ObjectDownloadListener
Change-Id: I20af1d8d931608e93fbc52e127f1b7bafd2f917c
2022-01-18 16:11:57 +01:00