Commit Graph

8748 Commits

Author SHA1 Message Date
Thomas Wolf e0281c5adb sshd: Connector for the Win32-OpenSSH SSH agent
Win32-OpenSSH uses a named Windows pipe for communication. Implement
a connector for this mechanism using JNA. Choose the appropriate
connector based on the setting of the 'identityAgent' parameter.

Bug: 577053
Change-Id: I205f07fb33654aa18ca5db92706e65544ce38641
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:46 +01:00
Thomas Wolf 071084818c sshd: handle IdentitiesOnly with an SSH agent
If an SSH agent is used but "IdentitiesOnly yes" is set, only those
keys from the agent that correspond to one of the keys explicitly given
via an IdentityFile directive are to be used.

Implement this by filtering the list of keys obtained from the agent
against the list of IdentityFiles, each entry suffixed with ".pub".
Load the public keys from these files, and ignore all other keys from
the agent. Keys without ".pub" file are also ignored.

Apache MINA sshd has no operation to load only the public key from a
private key file, so we have to rely on *.pub files.

Bug: 577053
Change-Id: I75c2c0b3ce35781c933ec2944bd6da1b94f4caf9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:45 +01:00
Thomas Wolf 4efc6a396a sshd: support IdentityAgent config
Handle the 'none' value, and change the value to select Pageant to
something that looks like an absolute UNC path name to avoid it's
handled as an relative path name.

Bug: 577053
Change-Id: I4ccf047abbc1def50e2782319e4fa7c744069401
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-30 17:13:44 +01:00
Nail Samatov ad098b3b85 LFS: Fix error occurring during delete branch
Fix TransportException occurring when deleting
a branch and push that change to remote repository
if BuiltinLFS is used to work with repository.
When finding LFS pointers in RemoteRepUpdate,
LfsPrePushHook fails to open ObjectReader
with new object id equal to ObjectId.zeroId().
If update is a deleting update (new object id is zero id),
we can assume that this update doesn't contain LFS Pointer
and we can skip step with extracting LFS pointer for that
RemoteRefUpdate.

Bug: 578313
Change-Id: Ic4367978338b8234d39d9af0d9674490f79fc22d
Signed-off-by: Nail Samatov <sanail@yandex.ru>
2022-01-27 20:55:58 +01:00
Ivan Frade 27e554e465 Merge "PackOutputStream: Extract cancellation and digest to superclass" 2022-01-27 11:42:54 -05:00
kylezhao 1a86c1044d PackOutputStream: Extract cancellation and digest to superclass
Checking the cancelled status and keeping a digest of the written data
is useful for other output streams. e.g. to write commit-graphs.

Pull up that functionality to a superclass, so it can be reused.

Change-Id: I177b50be09c4ea631e7a144cc6127085ec2ca411
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-01-27 19:21:31 +08:00
David Ostrovsky 75e7d08480 Bazel: Include bazel resource configuration file for RBE build
In I988f61e34 tools/remote-bazelrc was added, but this resource file
wasn't imported from main resource file workspace/.bazelrc.

Change-Id: Ie0c2f128fbccaa6f9e99ad29a3a76ac294f3cd29
2022-01-22 20:43:56 +01:00
Ivan Frade 076ecf8ded Merge "DFS block cache: report index load and evict stats" 2022-01-21 14:59:30 -05:00
Alina Djamankulova b536dbdb9b DFS block cache: report index load and evict stats
Enhance cache performance monitoring for large data such as pack and
bitmap indexes. Provide details about what is loaded and evicted from
cache like total number of cache hits, time in cache before eviction.

Add a custom consumer to report loading events and eviction events when
enabled.

Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I5739325db7ff7ec370e4defd8f7e46f1c3f5d2dd
2022-01-21 11:27:31 -08:00
David Ostrovsky 969601c742 Bazel: Add RBE support
The Bazel build can be used with Google's Remote Build Execution.

This needs the following setup steps:

  $ gcloud auth application-default login
  $ gcloud services enable remotebuildexecution.googleapis.com  --project=${PROJECT}

Create a worker pool. The instances should have at least 4 CPUs each
for adequate performance.

  $ gcloud alpha remote-build-execution worker-pools create default \
    --project=${PROJECT} \
    --instance=default_instance \
    --worker-count=50 \
    --machine-type=e2-standard-4 \
    --disk-size=200

To use RBE, execute

  $ bazel test --config=remote \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    //...

Change-Id: I988f61e342dab2136d8752ace945a4ed91a4189a
2022-01-20 06:43:13 -04:00
David Ostrovsky be732c16ab Bazel: Switch to using toolchain resolution for java rules
Bump Bazel version to release 5.0.0. In this new Bazel release,
--incompatible_use_toolchain_resolution_for_java_rules is flipped, that
means that the build must be adapted to toolchain resolution.

Specification toolchain resolution for java rules is here: [1]. Main
tracking Bazel issue is here: [2].

Given that new Bazel release also added support for remote JDK 17, add
experimental support for building with remote JDK 17 to produce major
byte code version 61.

Test Plan:

To build with remote JDK 11 run:

  $ bazel build :all

To build with remote JDK 17 run:

  $ bazel build --config java17 :all

[1] https://docs.google.com/document/d/1MVbBxbKVKRJJY7DnkptHpvz7ROhyAYy4a-TZ-n7Q0r4/edit?usp=sharing
[2] https://github.com/bazelbuild/bazel/issues/7849

Change-Id: I8d26dff722c6677ea7642913b61e416b23ea9041
2022-01-20 06:40:51 -04:00
David Ostrovsky c812947962 Bazel: Simplify java 11 toolchain definition
Also remove unused JDK9_JVM_OPTS constant.

Change-Id: I0f45ca8312a2a8c3aee3855e3ef3afa95808b4ad
2022-01-20 00:54:15 -05:00
David Ostrovsky bf3298478a Bazel: Format build files with buildifier
Change-Id: I1ff38237058d11a3f61acc881c504382ff4b7e08
2022-01-20 00:54:01 -05:00
David Ostrovsky 9a16e9a20a Bazel: Remove JDK 15 toolchain definition
Java 15 is outdated and the recent Bazel releases added support to
JDK 17.

Change-Id: I68c1f5b5132d844f07bfefc22f81ecf0a19df910
2022-01-20 00:53:51 -05:00
David Ostrovsky 8227098d7d Bazel: Remove version check
Recent Bazel releases perform Bazel version check, based on
.bazelversion file.

Change-Id: I92d5daaef8ecbd011517d314564c0f5492c8ea75
2022-01-20 00:53:42 -05:00
David Ostrovsky 132ba05c82 ExternalToolTest: Rename class name to usual name for test cases
Change-Id: I00e4a5e73f530b7ac671d09b9b447bb24d6419fa
2022-01-20 06:52:13 +01:00
David Ostrovsky f8a96e1a0b Bazel: Add missing java packages to error_prone package group
Change-Id: I709a3cd2639e9209b2c92c700e04ab4c90ed0524
2022-01-20 06:49:25 +01:00
David Ostrovsky 51b909d9e0 Adapt junit_tests invocation to removal of resource_jars attribute
In upcoming Bazel release 5.0 resource_jars attribute was removed: [1].
Replace resource_jars with runtime_deps as recommended in the release
notes.

[1] https://github.com/bazelbuild/bazel/issues/13221

Change-Id: Iab834d647cf63259ca182adc1f862b6001db0cce
2022-01-19 18:39:32 +01:00
Matthias Sohn 56f45e36dc Update orbit to I20220111151929
and update
- com.google.gson to 2.8.9.v20220111-1409
- org.bouncycastle.bcpg to 1.70.0.v20220105-1522
- org.bouncycastle.bcpkix to 1.70.0.v20220105-1522
- org.bouncycastle.bcprov to 1.70.0.v20220105-1522
- org.bouncycastle.bcutil to 1.70.0.v20220105-1522

Change-Id: I7d9159e0e8524459a11da84cb2db709cb1f0823d
2022-01-19 09:54:37 +01:00
Matthias Sohn 685cc54045 Update org.apache.maven.wagon:wagon-ssh to 3.5.1
Change-Id: I90807ef1ae065a8a8559e445c0aa7e2abdd2b9a8
2022-01-19 09:54:37 +01:00
Matthias Sohn 32cc4c56a6 Update org.eclipse.jdt:ecj to 3.28.0
Change-Id: I7573d8cfd07038bca4a7ed9e122e5dcf666632f8
2022-01-19 09:54:37 +01:00
Matthias Sohn 73c398a95f Update org.osgi:org.osgi.core to 6.0.0
Change-Id: If93e0e8769fcbb92befa1f159f48b0ac6b6f3d24
2022-01-19 09:54:37 +01:00
Matthias Sohn 4aca4e0a31 [errorprone] Fix implicit use of platform default charset
Change-Id: I42f893c4248d8b00bf4aaf59cd528c43940f99e2
2022-01-19 09:54:37 +01:00
Matthias Sohn 8e306e7a62 [bazel] Fix build of ExternalDiffToolTest
Change-Id: Ida4721142f49c86aa3f05aeffc6ca0aef40939f9
2022-01-19 09:54:37 +01:00
Matthias Sohn 9a57ae4707 Let org.eclipse.jgit.pgm use BooleanTriState
Change-Id: I575cd5e1ecd326514a8bb4c36a86a2dc71ba5457
2022-01-19 09:54:37 +01:00
Matthias Sohn 5f556588a9 Merge branch 'stable-6.0'
* stable-6.0:
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  [errorprone] Fix InfiniteRecursion error in RecordingLogger
  [errorprone] Suppress Finally error in ObjectDownloadListener
  [errorprone] Fix implicit use of default charset in FileBasedConfigTest
  [errorprone] Suppress FutureReturnValueIgnored in FileRepository#autoGc

Change-Id: I08d58c8f3f04e3a920da43b5fb252b1654c2b33c
2022-01-19 09:53:21 +01:00
Matthias Sohn 9b28f43cf1 Merge changes I6a22f37f,I092389e4,I20af1d8d,I83332efc into stable-6.0
* changes:
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  [errorprone] Fix InfiniteRecursion error in RecordingLogger
  [errorprone] Suppress Finally error in ObjectDownloadListener
  [errorprone] Fix implicit use of default charset in FileBasedConfigTest
2022-01-19 03:50:59 -05:00
Matthias Sohn b55c224ef3 Merge "[errorprone] Suppress FutureReturnValueIgnored in FileRepository#autoGc" into stable-6.0 2022-01-19 03:43:27 -05:00
Matthias Sohn d017a655df Merge branch 'stable-6.0'
* stable-6.0:
  UploadPack v2 protocol: Stop negotiation for orphan refs
  Complete update to servlet api 4.0.0

Change-Id: I55ab6e8fd4a76e4313e37b12f9fc5d5e4b84a681
2022-01-18 18:09:03 +01:00
Matthias Sohn de1abd3237 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: I6a9ed8338ffbf5363e48d640a2c4209e4e503549
2022-01-18 18:07:59 +01:00
Matthias Sohn 2cc0009737 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: Ib43068c32d9cb8effe4b873396391dc3c9197a6e
2022-01-18 17:51:14 +01:00
Matthias Sohn 1e59cabc08 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  UploadPack v2 protocol: Stop negotiation for orphan refs

Change-Id: I5db432bd416cfa8d3dd295bdce63e31d5f160a8a
2022-01-18 17:49:03 +01:00
Matthias Sohn b33133497f [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.

FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.

Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
2022-01-18 16:11:58 +01:00
Matthias Sohn a268faf729 [errorprone] Fix InfiniteRecursion error in RecordingLogger
Change-Id: I092389e428232a4fe7613d846c288d285ae9102c
2022-01-18 16:11:57 +01:00
Matthias Sohn 6fde4d3a6c [errorprone] Suppress Finally error in ObjectDownloadListener
Change-Id: I20af1d8d931608e93fbc52e127f1b7bafd2f917c
2022-01-18 16:11:57 +01:00
Matthias Sohn 8b00cb9324 [errorprone] Fix implicit use of default charset in FileBasedConfigTest
Change-Id: I83332efc498a5bce242915a1eec2346e6e1f58fd
2022-01-18 16:11:56 +01:00
Matthias Sohn 2fed62528a [errorprone] Suppress FutureReturnValueIgnored in FileRepository#autoGc
Ignore the FutureReturnValueIgnored warning for the unused return value
of #gc.

Change-Id: I4e7a2f85d404962c01726f9a1d079fe4a6430a1b
2022-01-18 16:11:50 +01:00
Marcin Czech 78d4fb1ca0 UploadPack v2 protocol: Stop negotiation for orphan refs
The fetch of a single orphan ref (for example Gerrit meta ref:
refs/changes/21/21/meta) did not stop the negotiation so client
had to advertise all refs. This impacts the fetch performance
on repositories with a large number of refs (for example on
Gerrit repository it takes 20 seconds to fetch meta ref
comparing to 1.2 second to fetch ref with parent).

To avoid this issue UploadPack, used on the server side,
now checks if all `want` refs have parents, if not this
means that client doesn't need any extra objects, hence
the server responds with `ready` and finishes the
negotiation phase.

Bug: 577937
Change-Id: Ia3001b400b415d5cf6aae45e72345ca08d3af058
2022-01-18 08:00:03 -04:00
Matthias Sohn 1e932c2e52 Complete update to servlet api 4.0.0
Ibd0240cf7ad updated servlet-api to 4.0.0 only partially for the
osgi-based build in Eclipse.

Complete this by updating dependencies also in maven and bazel build.

Change-Id: Ic4c3eb78c538007ca2177f6109d415147e58eabe
2022-01-12 18:42:35 +01:00
Thomas Wolf d30447e269 Switch to Apache MINA sshd 2.8.0
Update version in root pom.xml, adapt code & manifests. Bump the
dependency in the bazel build.

Update Orbit to I20220105095044 to get Apache MINA sshd 2.8.0 and
regenerate all target platforms.

Bug: 577542
Change-Id: Iefc02ceda8a9b0683f49aa8059999a5486d1f322
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-07 02:15:24 +01:00
Thomas Wolf df4fa808b3 Merge changes Ia744cd4a,Id95350c7,I746b7fb7
* changes:
  sshd: backport upstream fix for SSHD-1231
  [releng] bump japicmp base version and configure sshd bundles
  Merge branch 'stable-6.0'
2022-01-06 14:05:00 -05:00
Thomas Wolf 4c555f0742 sshd: backport upstream fix for SSHD-1231
SSHD-1231[1] may lead to exceptions when trying to authenticate first
with an RSA key that is rejected by the server. The upstream fix is a
one-liner but unfortunately didn't make it into Apache MINA sshd 2.8.0.

Incorporate the upstream fix in JGitPublicKeyAuthentication, and add
a test case for this.

[1] https://issues.apache.org/jira/browse/SSHD-1231

Bug: 577545
Change-Id: Ia744cd4aa569bccd937c855f3bb45c0116915bad
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-06 19:33:44 +01:00
Sebastian Schuberth e4f6e0af4a Add myself to .mailmap
Signed-off-by: Sebastian Schuberth <sebastian.schuberth@bosch.io>
Change-Id: I46e7bccb6c0c1e4dc21450220279b0546723773e
2022-01-06 09:05:38 +01:00
Thomas Wolf 709087c582 [releng] bump japicmp base version and configure sshd bundles
Bump the japicmp base version to 6.0.0.202111291000-r and configure
the o.e.j.ssh.apache and o.e.j.ssh.apache.agent bundles to ignore
internal classes.

Change-Id: Id95350c73b9141e1583f9de5fb6ab2496c7407d9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-05 17:24:35 +01:00
Thomas Wolf 126839b8b1 Merge branch 'stable-6.0'
* stable-6.0:
  Use slf4j-simple instead of log4j for logging
  Update orbit to R20211213173813

Change-Id: I746b7fb71571020ce49f7b50fd675c9864327719
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-05 17:22:50 +01:00
Thomas Wolf f7662a756e Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Use slf4j-simple instead of log4j for logging
  Update orbit to R20211213173813

Change-Id: I219ef3901c1d908b91bf9c8f00431b22686ff7a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-01-05 15:59:09 +01:00
Matthias Sohn 486afbc08d Use slf4j-simple instead of log4j for logging
JGit uses slf4j-api as logging API.

The libraries
- org.eclipse.jgit.http.test
- org.eclipse.jgit.pgm
- org.eclipse.jgit.ssh.apache.test
- org.eclipse.jgit.test
used the outdated log4j 1.2.15 which is EOL since years.

Since both jgit command line and also the tests don't need sophisticated
logging features replace log4j with the much simpler slf4j-simple log
implementation. The org.slf4j.binding.simple 1.7.30 archive has only
25kB instead of 429kB for log4j 1.2.15

Applications using jgit are free to choose any other log implementation
supporting slf4j API.

Change-Id: I89e85cd3c76e954c3434622510975ce65dc227d4
2021-12-31 01:09:52 +01:00
Matthias Sohn e06eeb0167 Update orbit to R20211213173813
and update
- com.google.gson to 2.8.8.v20211029-0838
- javaewah to 1.1.13.v20211029-0839
- net.i2p.crypto.eddsa to 0.3.0.v20210923-1401
- org.apache.ant to 1.10.12.v20211102-1452
- org.apache.commons.compress to 1.21.0.v20211103-2100
- org.bouncycastle.bcprov to 1.69.0.v20210923-1401
- org.junit to 4.13.2.v20211018-1956

Change-Id: I3ac39fc8a5df571d2e290241a03668f1e60880b4
2021-12-31 01:09:51 +01:00
Matthias Sohn 4d34cdf345 Merge branch 'stable-6.0'
* stable-6.0:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: Icc38ce20c1c791c8cf0ca3dcab46b1151f122805
2021-12-31 00:33:30 +01:00
Matthias Sohn fa82b80c9b Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Use FileSnapshot without using configs for FileBasedConfig

Change-Id: Iaff40b144aea9ad5e6d0fa6b448ad7d527992832
2021-12-31 00:30:51 +01:00