Commit Graph

8513 Commits

Author SHA1 Message Date
Matthias Sohn 7ca091a526 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  FS: debug logging only if system config file cannot be found

Change-Id: I4f6fd422e15361118134c4d1534507e0bd81a172
2021-11-28 12:04:55 +01:00
Matthias Sohn b868f91013 Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  FS: debug logging only if system config file cannot be found

Change-Id: Ib32079d58900328f9fc7c91648a1eee9c2bccba5
2021-11-28 12:04:15 +01:00
Matthias Sohn 91233eea56 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  FS: debug logging only if system config file cannot be found

Change-Id: Icd52afc3a74e0d2618181f62515d7e2a8d6c9739
2021-11-28 12:02:45 +01:00
Matthias Sohn 684ec2c316 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  FS: debug logging only if system config file cannot be found

Change-Id: Ib9eb4f4cc44ecc1958c9b6455ab7865913f8b247
2021-11-28 12:02:03 +01:00
Thomas Wolf 122237439d FS: debug logging only if system config file cannot be found
The command 'git config --system --show-origin --list -z' fails if
the system config doesn't exist. Use debug logging instead of a
warning for failures of that command. Typically the user cannot do
anything about it anyway, and JGit will just work without system
config.

Bug: 577492
Change-Id: If628ab376182183aea57a385c169e144d371bbb2
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-11-28 12:01:04 +01:00
Thomas Wolf 9e4fe739c7 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Better git system config finding

Change-Id: Iebbae08b6bb6ef510ca07329df77223bc2128ec1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-11-23 19:52:01 +01:00
Thomas Wolf 6fe66b7382 Set JSch global config values only if not set already
Bug: 576604
Change-Id: I04415f07bf2023bc8435c097d1d3c65221d563f1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit f8b0c00e6a)
2021-11-23 19:52:01 +01:00
Thomas Wolf 6e0ac5840c Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Better git system config finding

Change-Id: I47946b91658cb2e38709179b7a8513b1211431e4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-11-23 19:46:39 +01:00
Thomas Wolf ecbe49efbd Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Better git system config finding

Change-Id: I460d855ea7878b279dbaffa6eb7ce5ca93f4c12c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-11-23 19:45:50 +01:00
Thomas Wolf e29325f064 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Better git system config finding

Change-Id: I1022d003e0d3b9d452abfed9ac49663d0a93c6e6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-11-23 19:44:30 +01:00
Thomas Wolf 6aa29d1166 Better git system config finding
We've used "GIT_EDITOR=edit git config --system --edit" to determine
the location of the git system config for a long time. But git 2.34.0
always expects this command to have a tty, but there isn't one when
called from Java. If there isn't one, the Java process may get a
SIGTTOU from the child process and hangs.

Arguably it's a bug in C git 2.34.0 to unconditionally assume there
was a tty. But JGit needs a fix *now*, otherwise any application using
JGit will lock up if git 2.34.0 is installed on the machine.

Therefore, use a different approach if the C git found is 2.8.0 or
newer: parse the output of

  git config --system --show-origin --list -z

"--show-origin" exists since git 2.8.0; it prefixes the values with
the file name of the config file they come from, which is the system
config file for this command. (This works even if the first item in
the system config is an include.)

Bug: 577358
Change-Id: I3ef170ed3f488f63c3501468303119319b03575d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 9446e62733)
2021-11-23 19:42:34 +01:00
Matthias Sohn 8e7dfb5e4b Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Fix target platforms

- jetty 9.4.43 moved to archive.eclipse.org
- use the final release p2 repo instead of staging for 2021-09

Change-Id: I35c1264b8e9c63874663141288dca03b47f263ed
2021-11-03 23:18:21 +01:00
Matthias Sohn dc766feffc Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Fix target platforms

- jetty 9.4.41 moved to archive.eclipse.org
- use the final release p2 repo instead of staging for 2021-06

Change-Id: I7a02b1fe271910dd82acf45242d8425aba2f52f9
2021-11-03 23:13:39 +01:00
Matthias Sohn 8ff0859703 Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Fix target platforms

- jetty 9.4.36 moved to archive.eclipse.org
- use the final release p2 repo instead of staging for 2021-03

Change-Id: I6ce2294bb150fedf8ded7fe6c1e23019d9c178d6
2021-11-03 23:09:15 +01:00
Matthias Sohn 587a273d85 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Fix target platforms

Change-Id: Ib1876923ba98252682f41a03877792fd22b418d6
2021-11-03 23:01:44 +01:00
Matthias Sohn 4f8d434623 Fix target platforms
- jetty 9.4.30 moved to archive.eclipse.org
- use the final release p2 repo instead of staging for 2020-09

Change-Id: Ia096200e5983f0022c6c0da4dae035433e852807
2021-11-03 22:41:33 +01:00
Matthias Sohn 78043b07e2 Merge branch 'stable-5.12' into stable-5.13
* stable-5.12:
  Fix missing peel-part in lsRefsV2 for loose annotated tags
  Fix RevWalk.getMergedInto() ignores annotated tags
  Optimize RevWalk.getMergedInto()
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix running benchmarks from bazel
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: Ie5da8d2d07f39ab5aaeca560040637635d4eb04c
2021-10-15 22:58:21 +02:00
Matthias Sohn 91e44e08cf Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  Fix missing peel-part in lsRefsV2 for loose annotated tags
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix running benchmarks from bazel
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I2122b3966bddb10d43d439eb8c5d6cc4583c6ba6
2021-10-15 22:48:01 +02:00
Matthias Sohn 4060936bdd Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  Fix missing peel-part in lsRefsV2 for loose annotated tags
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix running benchmarks from bazel
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I23ccb89306263ce042ce782d881812d4b5187672
2021-10-15 22:45:18 +02:00
Matthias Sohn 50411fa147 Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Fix missing peel-part in lsRefsV2 for loose annotated tags
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2
  Fix running benchmarks from bazel
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I4e7c7727ebfa9a7748477f2300bd66d775548ad6
2021-10-15 22:32:00 +02:00
Saša Živkov d160e8a935 Fix missing peel-part in lsRefsV2 for loose annotated tags
We observed the following issue:

  $ git tag -a v1.0 -m v1.0
  $ git push origin tag v1.0
  $ git ls-remote origin v1.0^{}
  ... empty result ...

  On the server (Gerrit) side run git-gc to pack the refs:
  $ git gc

  Repeat the ls-remote from the client and the result is correct:
  $ git ls-remote origin v1.0^{}
  7ad85c810de3ae922903d4bdd17c53cd627260ba        refs/tags/v1.0^{}

Unfortunately, the existing UploadPackTest didn't reveal this issue
although it provided the test case needed to do so: testV2LsRefsPeel.
This is because The UploadPackTest uses InMemoryRepository which
internally uses Dfs* implementations. The issue is only reproducible
when using the FileRepository.

It is a non-trivial task to refactor the UploadPackTest to work against
both InMemoryRepository and FileRepository and this change is not trying
to do that. This change creates a new test:
UploadPackLsRefsFileRepositoryTest and copies the necesssary code from
the UploadPackTest.

Change-Id: Icfc7d0ca63f1524bafe24c9626ce12ea72aa3718
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
2021-10-15 22:29:28 +02:00
kylezhao 60b81c5a92 Fix RevWalk.getMergedInto() ignores annotated tags
If an annotated tag refers to a commit, we should not ignore it.

Change-Id: I77504f93636e9e984540e7d8535ef301adce6a80
Signed-off-by: kylezhao <kylezhao@tencent.com>
2021-10-08 12:05:48 +02:00
kylezhao c5b3054735 Optimize RevWalk.getMergedInto()
Transitive Relation Definition:
On the DAG of commit history, if A can reach B, C can reach A, then C
can reach B.

Example:
As is shown in the graph below:

  1 - 2 - 3 - 4 (side)
            \
             5 -  6^ (master) - 7 (topic)

Find out which branches is 2 merged into:
After we calculated that master contains 2, we can mark 6 as TEMP_MARK
to avoid unwanted walks.
When we want to figure out if 2 is merge into the topic, the traversal
path would be [7, 6] instead of [7, 6, 5, 3, 2].

Test:
This change can significantly improve performance for tags.
On a copy of the Linux repository, the command 'git tag --contains
<commit>' had the following performance improvement:

commit      | Before   | After   | Rel %
47a44d27ca  | 29251ms  | 6687ms  | -77%
90327e7dff  | 21388ms  | 6256ms  | -70%
f85fac0efa  | 11150ms  | 7338ms  | -34%

The current version ignores tags, even though the tag is a type of
the ref.
Follow-up commits I'll fix it.

Change-Id: Ie6295ca4d16070499912af462239e679a97cce47
Signed-off-by: kylezhao <kylezhao@tencent.com>
Reviewed-by: Christian Halstrick <christian.halstrick@sap.com>
Reviewed-by: Martin Fick <mfick@codeaurora.org>
2021-10-08 12:05:47 +02:00
Matthias Sohn 5a7a5d5ae9 Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I9e66ef90dc9a65bac47b35705d679bf992bd72b9
2021-10-08 00:36:14 +02:00
Matthias Sohn 721a971d5d Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I88c47ff57f4829baec5b19aad3d8d6bd21f31a86
2021-10-08 00:33:33 +02:00
Matthias Sohn 4fd8c1406f Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  reftable: drop code for truncated reads
  reftable: pass on invalid object ID in conversion
  Update eclipse-jarsigner-plugin to 1.3.2

Change-Id: I1c18f5f435f4a4a86e0548a310dbfc74191e1ed5
2021-10-08 00:18:42 +02:00
Han-Wen Nienhuys 5f8c484136 reftable: drop code for truncated reads
The reftable format is a block based format, but allows for variably
sized blocks. This obviously happens for reflog blocks (which are zlib
compressed), but is also accepted for index blocks: In the spec, this
is motivated as

     To achieve constant O(1) disk seeks for lookups the index must be
     a single level, which is permitted to exceed the file's
     configured block size, but not the format's max block size of
     15.99 MiB.

Hence, when parsing a block, one cannot be sure of its exact size:
after reading a default-size block (eg. 4kb), the block header may
state that the block is in fact larger.

Before, the code would mark the block as `truncated`, noting

     // Its OK during sequential scan for an index block to have been
     // partially read and be truncated in-memory. This happens when
     // the index block is larger than the file's blockSize. Caller
     // will break out of its scan loop once it sees the blockType.

This looks like either

* a remnant of never-implemented functionality. There is no reason to
  ever sequentially scan an index block.

* alluding to sequential scan of the data blocks before the index
  blocks (eg. scanning refs, which ends when we find the first ref index
  block, and we can then ignore the index block).

This comment is followed by code that populates the
restartTbl/restartCnt fields relative to the (possibly truncated)
buffer. If the buffer is truncated, this essentially reads garbage,
leading to OOB array access when using the index block.

Fix this by dropping the truncated logic and issuing a second read if
the first read was short.

Add a test.

We have never observed this failure scenario at Google. We use 64kb
blocksize, which requires us to need fewer index entries. The reftable
spec mentions an Android repo of size 36M. With 64kb blocks, that's
just 562 index entries. Even with historical growth, we are long from
requiring an index whose size exceeds a single block.

When adding the analogous test for seeking refs, there was no failure.
This points to another possibility which is that the code tries to
avoid writing large index blocks for refs.

I did not investigate further which one it is.

Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=576250

Bug: 576250
Change-Id: I41ec21fac9e526ef57b3d6fb57b988bd353ee338
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2021-09-28 20:35:02 +02:00
Han-Wen Nienhuys b4782d74fd reftable: pass on invalid object ID in conversion
Before, while trying to determine if an object ID was a tag or not,
the reftable conversion would yield an exception.

Change-Id: I3688a0ffa9e774ba27f320e3840ff8cada21ecf0
2021-09-27 11:30:13 -04:00
Matthias Sohn 9c3190ce7a Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Id5d05d96c392913de7b4451421c2ffb7b63ab83f
(cherry picked from commit c70c0acb47)
2021-09-27 11:28:33 -04:00
Matthias Sohn 283c23012f Fix running benchmarks from bazel
add missing dependency to
- javaewah
- slf4j-api

Change-Id: I28dc982791b32f10d20b2fd0671aa8d2514a0fb3
2021-09-27 16:50:15 +02:00
Matthias Sohn c70c0acb47 Update eclipse-jarsigner-plugin to 1.3.2
Change-Id: Id5d05d96c392913de7b4451421c2ffb7b63ab83f
2021-09-24 00:19:00 +02:00
Matthias Sohn 26d36848fe Add org.bouncycastle.bcutil to p2 repository
Bug: 575621
Change-Id: I57de7af9f9b236cac570d9c10c3d1c3886720868
2021-09-20 17:09:34 +02:00
Matthias Sohn 81cd70d5d0 Prepare 5.13.1-SNAPSHOT builds
Change-Id: Ib2f689d8d13eab022da5b5e83d6d6bebc1bb81d3
2021-09-08 20:24:36 +02:00
Matthias Sohn b3d51a399e JGit v5.13.0.202109080827-r
Change-Id: If3b2d4256712cc7e577c23e75c0d4ad940870e72
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-09-08 14:27:06 +02:00
Matthias Sohn e296af1697 Merge branch 'master' into stable-5.13
* master:
  Revert "DFS block cache: Refactor to enable parallel index loading"

Change-Id: Ifd7cf7a366bf682da6e92ac9dd416bd88e9f1654
2021-09-04 11:45:33 +02:00
Minh Thai a3a8de3108 Revert "DFS block cache: Refactor to enable parallel index loading"
This reverts commit 3cd7eb1b23.

Change-Id: I71ce68ce19503f0f9ad83069dc53eba6ab2c489b
Signed-off-by: Minh Thai <mthai@google.com>
2021-09-03 14:15:42 -07:00
Matthias Sohn efdc03e164 Prepare 5.13.0-SNAPSHOT builds
Change-Id: I2a1d7ab24d5ca718348f4ce3cda351553e48cd1f
2021-09-01 20:00:56 +02:00
Matthias Sohn 4949a295df JGit v5.13.0.202109011149-rc1
Change-Id: Id8d0970102f18e61a2fc7cf941267c9089d71c1a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-09-01 16:49:12 +02:00
Matthias Sohn 6fc54671e0 Merge branch 'master' into stable-5.13
* master:
  GitServlet: allow to override default error handlers
  Silence API error for new interface method ProtocolV2Hook#onObjectInfo
  transport: add object-info capability
  Ignore IllegalStateException if JVM is already shutting down
  Update orbit to R20210825222808 for 2021-09
  RevWalk: getMergedInto's result is wrong on the second call

Change-Id: I007882d0c663e406d8845cc807f3792ae7673230
2021-09-01 16:31:13 +02:00
Matthias Sohn 35aaca1efb Merge branch 'stable-5.12'
* stable-5.12:
  GitServlet: allow to override default error handlers

Change-Id: I4710cec30c58d7a6525a42dc5f1fdffc912ed073
2021-09-01 16:23:47 +02:00
Matthias Sohn c0380e9cdd Merge branch 'stable-5.11' into stable-5.12
* stable-5.11:
  GitServlet: allow to override default error handlers

Change-Id: I65c01857ab42ec3023da16e6816d58e1a36e40e4
2021-09-01 16:22:21 +02:00
Matthias Sohn 5a3569f04b Merge branch 'stable-5.10' into stable-5.11
* stable-5.10:
  GitServlet: allow to override default error handlers

Change-Id: If10a99bcdc91105059b15a1d137dd38af228c58d
2021-09-01 16:08:03 +02:00
Matthias Sohn 738ef7782d Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  GitServlet: allow to override default error handlers

Change-Id: I55273087f8a67762130728eaaf635519c25255b3
2021-09-01 15:55:58 +02:00
Antonio Barone 8470771510 GitServlet: allow to override default error handlers
GitServlet delegates repository access over HTTP to the GitFilter
servlet.

GitServlet, in turn, can be extended by jgit consumers to provide custom
logic when handling such operations.

This is the case, for example, with Gerrit Code Review, which provides
custom behavior with a GitOverHttpServlet [1].

Among possible customizations, the ability of specifying a custom error
handler for UploadPack and ReceivePack was already introduced in
GitFilter by Idd3b87d6b and I9c708aa5a2, respectively.

However the `setUploadPackErrorHandler` and `setReceivePackErrorHandler`
methods were never added to the GitServlet.

Expose the `setUploadPackErrorHandler` and `setReceivePackErrorHandler`
methods to the GitServlet, so that consumers of the jgit library might
specify custom error handlers.

[1] https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-3.2/java/com/google/gerrit/httpd/GitOverHttpServlet.java#95

Change-Id: I712d485ff68b662b48c71ef75650c5a155950d23
2021-09-01 15:25:30 +02:00
Matthias Sohn 525127ebb4 Silence API error for new interface method ProtocolV2Hook#onObjectInfo
Change-Id: Ie4d09db2b89f26bc6ceb4e84f10c7e66b0ff684f
2021-09-01 14:50:48 +02:00
Bruno Albuquerque 5b8e387c67 transport: add object-info capability
Sometimes it is useful to obtain metadata associated with an object
without the need to first download it locally. This is specially useful
when using partial clones.

This change implements the object-info capability that allows clients to
query the remote server for object metadata (currently only size). This
is a backport of the same capability that was recently added to the Git
project a2ba162cda (object-info: support for retrieving object info,
2021-04-20).

Signed-off-by: Bruno Albuquerque <bga@google.com>
Change-Id: I4dc9828e1c247f08b0976b8810be92d124366165
2021-08-31 14:36:06 -07:00
Matthias Sohn c1961ad809 Merge branch 'stable-5.12'
* stable-5.12:
  RevWalk: getMergedInto's result is wrong on the second call

Change-Id: Iee51703b2b009df15c0d8ee19fd480058b665bce
2021-08-30 22:25:02 +02:00
Matthias Sohn 6817b7e3b4 Ignore IllegalStateException if JVM is already shutting down
Trying to register/unregister a shutdown hook when the JVM is already in
shutdown throws an IllegalStateException. Ignore this exception since we
can't do anything about it.

Bug: 575367
Change-Id: Ic967c16c7f566c84778795315ab369e76668b364
2021-08-29 05:02:59 -04:00
Matthias Sohn c697f02e12 Update orbit to R20210825222808 for 2021-09
Change-Id: I5a98f2134d19c3495d7d39cccfd7cf6f07640bd5
2021-08-26 20:42:26 +02:00
Matthias Sohn 574e899051 Prepare 5.13.0-SNAPSHOT builds
Change-Id: I6db044fdd57b60adbc2dd078c4af8cb54bb331c3
2021-08-25 22:33:09 +02:00