Commit Graph

9150 Commits

Author SHA1 Message Date
kylezhao 8a7348df69 CommitGraph: add commit-graph for FileObjectDatabase
This change makes JGit can read .git/objects/info/commit-graph file
and then get CommitGraph.

Loading a new commit-graph into memory requires additional time. After
testing, loading a copy of the Linux's commit-graph(1039139 commits)
is under 50ms.

Bug: 574368
Change-Id: Iadfdd6ed437945d3cdfdbe988cf541198140a8bf
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-23 13:06:06 +08:00
Anna Papitto 6722f25d56 PackWriter#prepareBitmapIndex: add clarifying comments
New readers of #prepareBitmapIndex may be confused about the manual
memory management (hidden mutation and nulling out pointers).

Add two clarifying comments to help future readers.

Change-Id: I93cab1919066efda37e96c47667f6991f67e377e
2022-12-20 13:56:11 -05:00
Ivan Frade fb3fd69584 Merge "IO#readFully: provide overload that fills the full array" 2022-12-19 13:50:08 -05:00
Anna Papitto 9b7c3ac11f IO#readFully: provide overload that fills the full array
IO#readFully is often called with the intent to fill the destination
array from beginning to end. The redundant arguments for where to start
and stop filling are opportunities for bugs if specified incorrectly or
if not changed to match a changed array length.

Provide a overloaded method for filling the full destination array.

Change-Id: I964f18f4a061189cce1ca00ff0258669277ff499
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-19 10:26:41 -08:00
Thomas Wolf f38a0614f5 Fix API warnings for the new CommitGraph
Mark the internal package as internal, visible only to the test bundle.
Add an API filter for CoreConfig.DEFAULT_COMMIT_GRAPH_ENABLE.

Change-Id: Ib62a93b873c93daf638b6c57e62fd267e16801bb
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-12-17 10:36:50 +01:00
Anna Papitto 0d10ebe560 PackReverseIndex#findPosition: fix typo in method name
The package-private findPostion method has a type in it. The typo will
become more widespread when a file-based implementation class is
introduced.

Correct the spelling to findPosition before the file-based
implementation is introduced.

Change-Id: Ib285f5a3f9a333ace1782dae9b5d425505eb962a
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-16 16:48:43 -05:00
kylezhao b082c58e0f GC: Write commit-graph files when gc
If 'core.commitGraph' and 'gc.writeCommitGraph' are both true, then gc
will rewrite the commit-graph file when 'git gc' is run. Defaults to
false while the commit-graph feature matures.

Bug: 574368
Change-Id: Ic94cd69034c524285c938414610f2e152198e06e
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 11:11:45 -05:00
kylezhao 7016e2ddae CommitGraph: add core.commitGraph config
Change-Id: I3b5e735ebafba09ca18fd83da479c7950fa3ea8d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 10:21:09 -05:00
Ivan Frade 6ea36794d1 Merge "Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering" 2022-12-16 08:20:24 -05:00
kylezhao 7b0f633b67 CommitGraph: implement commit-graph read
Git introduced a new file storing the topology and some metadata of
the commits in the repo (commitGraph). With this data, git can browse
commit history without parsing the pack, speeding up e.g.
reachability checks.

This change teaches JGit to read commit-graph-format file, following
the upstream format([1]).

JGit can read a commit-graph file from a buffered stream, which means
that we can provide this feature for both FileRepository and
DfsRepository.

[1] https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: Ib5c0d6678cb242870a0f5841bd413ad3885e95f6
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 06:57:06 -05:00
Anna Papitto 5c6c374ff6 Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering
Deleting orphan files depends on .pack and .keep being reverse-sorted
to before the corresponding index files that could be orphans. The new
reverse index file extension (.rev) will break that frail dependency.

Rewrite Gc#deleteOrphans to avoid that dependence by tracking which pack
names have a .pack or .keep file and then deleting any index files that
without a corresponding one. This approach takes linear time instead of
the O(n logn) time needed for sorting.

Change-Id: If83c378ea070b8871d4b01ae008e7bf8270de763
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-15 11:54:11 -08:00
Jeremy T. Braun 514ebfdc7e WalkPushConnection: Sanitize paths given to transports
These paths are given to the underlying URI-based transports (s3, sftp,
http), all of which expect forward-slash as the path separator
character.

Change-Id: I3cbb5928c9531a4da4691411bd8ac248fdf47ef2
2022-12-15 12:53:19 -05:00
Matthias Sohn 61f4a036c6 Merge changes Iad832fe1,Icef9658c
* changes:
  CommitGraphWriter: fix UnusedException errorprone error
  Update jetty to 10.0.13
2022-12-14 04:50:54 -05:00
Ivan Frade 91587aebd0 GraphCommits: Remove unused getter by position
CommitGraphWriter uses the GraphCommits in for-each loops and doesn't
need the access by position anymore. This was a left-over from
https://git.eclipse.org/r/c/jgit/jgit/+/182832

Remove the unused method.

Change-Id: I39df9bfab2601d581705ddf4cea3c04ed4765ff9
2022-12-13 14:37:37 -08:00
Matthias Sohn 04245494bb CommitGraphWriter: fix UnusedException errorprone error
Errorprone run in the bazel build raised this exception:

org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/commitgraph/CommitGraphWriter.java:105:
error: [UnusedException] This catch block catches an exception and
re-throws another, but swallows the caught exception rather than setting
it as a cause. This can make debugging harder.
		} catch (InterruptedIOException e) {
		  ^
    (see https://errorprone.info/bugpattern/UnusedException)
  Did you mean 'throw new
IOException(JGitText.get().commitGraphWritingCancelled, e);'?

Change-Id: Iad832fe17955fc1e60e6a4902bc50fd9dca76b9d
2022-12-13 15:04:45 +01:00
Matthias Sohn 6ca3d2194f Update jetty to 10.0.13
Since Oomph's p2 repo for jetty 10.0.13 doesn't have source bundles, we
remove them. Eclipse platform doesn't create p2 repos for jetty anymore
and we aren't yet ready to use maven dependencies like the platform
does.

Change-Id: Icef9658ce441be43931e32d931adf717e2fa222c
2022-12-13 15:01:24 +01:00
Xing Huang 3a136d2000 PackExt: Add a commit graph extension.
There is no commit graph PackExt because the non-DFS stack is not writing using PackExt mechanism. The extension is needed in DFS to determine the stream to write the commit-graph.

Add a commit graph extension that matches the one in cgit
(https://git-scm.com/docs/commit-graph#_file_layout)
in preparation for adding DFS support for reading and writing commit graphs.

Change-Id: Id14eda9f116a319124981e0bcbc533928b1b5e8c
Signed-off-by: Xing Huang <xingkhuang@google.com>
2022-12-12 17:26:12 -05:00
Matthias Sohn e74f3855ad Merge "commitgraph package: fix exports/imports, add @since tag for new API" 2022-12-08 14:10:23 -05:00
Sergey ec7c61eac3 BatchRefUpdate: Consistent switch branches in ref update
The expression RefUpdate ru = newUpdate(cmd) is eagerly evaluated before the switch statement.
But it is not used in some switch cases and thus is calculated uselessly.

Move expression evaluation to the switch case where it is actually used.
After such a move, several cases became identical and thus were squashed.

Change-Id: Ifd1976f1c28378e092fb24d7ca9c415cba49f07f
2022-12-08 05:27:35 -05:00
Sergey 2b21d9bbb6 RefWriter#writePackedRefs: Remove a redundant "if" check
After checking the variable, the same variable was checked again inside
the "if" block, and after the first check, this variable does not
change. Remove the second unnecessary check.

Change-Id: I6a38e67073f7f93105575b8f415ad32d350af602
2022-12-08 10:59:58 +01:00
Matthias Sohn ebc1f7d65c commitgraph package: fix exports/imports, add @since tag for new API
Change-Id: I9175b1d796f91f5ba4e21d3418550ae451c054b0
2022-12-08 02:00:58 +01:00
Ivan Frade 64d37a20bd Merge "CommitGraph: implement commit-graph writer" 2022-12-06 12:53:52 -05:00
kylezhao cf70e7cbe4 CommitGraph: implement commit-graph writer
Teach JGit to write a commit-graph formatted file by walking commit
graph from specified commit objects.

See: https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: I34f9f28f8729080c275f86215ebf30b2d05af41d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-06 20:34:46 +08:00
Thomas Wolf 8e1053ac4c [releng] bump japicmp base version
Bump the japicmp base version to 6.4.0.202211300538-r. (Up from 6.2.0.
We forgot to bump in the 6.4.0 release cycle.)

Change-Id: Ied39865563fb57f5ee9059241449e5e101e75d86
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-12-01 00:03:50 +01:00
Matthias Sohn e7b801c9a4 Merge branch 'stable-6.4'
* stable-6.4:
  Prepare 6.4.1-SNAPSHOT builds
  JGit v6.4.0.202211300538-r
  Fix crashes on rare combination of file names

Change-Id: I9b5c5f408d1d45868c138c15675346a6693ab998
2022-11-30 18:15:52 +01:00
Matthias Sohn 339b38340f Prepare 6.4.1-SNAPSHOT builds
Change-Id: I860bfde113c05015c41304c4a77c44c224bd0923
2022-11-30 15:41:41 +01:00
Matthias Sohn acd079b372 JGit v6.4.0.202211300538-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: If4001b255a209849b4acabd2083164d0794f00c4
2022-11-30 11:38:12 +01:00
Dmitrii Filippov cb9f058f9b Fix crashes on rare combination of file names
The NameConflictTreeWalk class is used in merge for iterating over
entries in commits. The class uses a separate iterator for each
commit's tree. In rare cases it can incorrectly report the same entry
twice. As a result, duplicated entries are added to the merge result
and later jgit throws an exception when it tries to process merge
result.

The problem appears only when there is a directory-file conflict for
the last item in trees. Example from the bug:
Commit 1:
* subtree - file
* subtree-0 - file
Commit 2:
* subtree - directory
* subtree-0 - file
Here the names are ordered like this:
"subtree" file <"subtree-0" file < "subtree" directory.

The NameConflictTreeWalk handles similar cases correctly if there are
other files after subtree... in commits - this is processed in the
AbstractTreeIterator.min function. Existing code has a special
optimization for the case, when all trees are pointed to the same
entry name - it skips additional checks. However, this optimization
incorrectly skips checks if one of trees reached the end.

The fix processes a situation when some trees reached the end, while
others are still point to an entry.

bug: 535919
Change-Id: I62fde3dd89779fac282479c093400448b4ac5c86
2022-11-29 10:49:27 +01:00
Han-Wen NIenhuys 1d5a6c77a6 Merge "Fix crashes on rare combination of file names" 2022-11-28 09:34:46 -05:00
Matthias Sohn 2e28f27c26 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I4238b6181e96e22e540cf34802a332f868cb6dfb
2022-11-23 19:09:33 +01:00
Matthias Sohn 5f91f5f996 Merge branch 'stable-6.4'
* stable-6.4:
  Prepare 6.4.0-SNAPSHOT build
  JGit v6.4.0.202211231055-rc1

Change-Id: I5bf49cb31d8c401e8f8f12a69d96ac8b42ff3605
2022-11-23 19:08:58 +01:00
Matthias Sohn ce0c739f21 Prepare 6.4.0-SNAPSHOT build
Change-Id: I41c4f73472bb47d8f9d2d117d17e11bba4802928
2022-11-23 17:10:33 +01:00
Matthias Sohn a3a726be6a JGit v6.4.0.202211231055-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Ia34696d07568b298544ee2cdc6f4b6746774bb82
2022-11-23 16:56:09 +01:00
Matthias Sohn ae39a004fe Merge branch 'master' into stable-6.4
* master:
  [pgm] Add options --name-only, --name-status to diff, log, show
  Update Orbit to R20221123021534 for 2022-12
  RBE: Update toolchain with bazel-toolchains 5.1.2 release
  SshTestGitServer: : ensure UploadPack is closed to fix resource leak
  UploadPackTest: ensure UploadPack is closed to fix resource leak
  [pgm] Ensure UploadPack is closed to fix resource leak
  UploadPackServlet#doPost use try-with-resource to ensure up is closed
  Fix warnings in PatchApplierTest
  Fix boxing warnings in TransportTest
  Silence warnings about unclosed BasePackPushConnection
  Fix warning about non-externalized String
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings
  Add 4.26 target platform
  Use "releases" repository for 4.25 target platform
  Update Apache Mina SSHD to 2.9.2
  Update Orbit to S20221118032057
  DfsBlockCache: Report IndexEventConsumer metrics for reverse indexes.
  DfsStreamKey: Replace ForReverseIndex to separate metrics.
  RawText.isBinary(): handle complete buffer correctly
  PackExt: Add a reverse index extension.

Change-Id: Ia41a2b0662bdb174e05a1adba6502191ef5ce4c5
2022-11-23 16:54:34 +01:00
alerthak c14e6f5c6c [pgm] Add options --name-only, --name-status to diff, log, show
Change-Id: Ib218bd2ccbd7990feca4c35d8c8dc34d4a8291e6
2022-11-23 16:12:54 +01:00
Matthias Sohn 3d522c6900 Update Orbit to R20221123021534 for 2022-12
Change-Id: I9de11cc198ce3926d5233c4e30dc74f36674d3b5
2022-11-23 10:32:51 +01:00
David Ostrovsky 19de967857 RBE: Update toolchain with bazel-toolchains 5.1.2 release
Due to this platform style migration: [1] the RBE toolchain needs to be
updated to use the latest rbe_config_gen from bazel-toolchains (at least
version 5.1.2 so that it contains: [2]).

This change makes RBE build forwards compatible so that Bazel could be
updated to the upcoming major 6.0 release.

[1] https://github.com/bazelbuild/bazel/issues/8622
[2] 8c717f8258

Change-Id: Ia3a108fa05b1d52cd2018ba901ffd4c9e55ae8ab
2022-11-23 09:31:39 +01:00
Matthias Sohn 396508e283 SshTestGitServer: : ensure UploadPack is closed to fix resource leak
Change-Id: Ib766f1d56b3ca7940c593ebfe0ac68fc5028a11c
2022-11-21 00:08:42 +01:00
Matthias Sohn bde09c185f UploadPackTest: ensure UploadPack is closed to fix resource leak
Change-Id: I4c8cf6041b4011934d338138d4531d190fdd6abb
2022-11-21 00:08:42 +01:00
Matthias Sohn 4ddc64a2dd [pgm] Ensure UploadPack is closed to fix resource leak
Change-Id: I40da4b331012dd1c6f4b497e46c973fd776b737f
2022-11-21 00:08:42 +01:00
Matthias Sohn 3e72816629 UploadPackServlet#doPost use try-with-resource to ensure up is closed
Change-Id: Iadbf81f183bb94f3b00b9940f065586b13e85c95
2022-11-21 00:08:38 +01:00
Matthias Sohn cddff2b7fd Fix warnings in PatchApplierTest
- don't use final for method parameters
- fix hiding member warnigns

Change-Id: I73c386f669918d3291ee3380024c018483aa3c97
2022-11-20 20:41:20 +01:00
Matthias Sohn 143188e831 Fix boxing warnings in TransportTest
Change-Id: I7e6dc845b89899cff262fab77c3977dbef5eea02
2022-11-20 20:34:57 +01:00
Matthias Sohn f849b8d0d9 Silence warnings about unclosed BasePackPushConnection
Change-Id: If52e8462e6222dd58d1004dd5ac174a27d96d098
2022-11-20 20:32:37 +01:00
Matthias Sohn 9fc9ae6de9 Fix warning about non-externalized String
Change-Id: I00b2132eac6d6ad2701d3f8151eb8c80ad6d679d
2022-11-20 20:30:45 +01:00
Matthias Sohn fe3908076e Merge branch 'stable-6.3'
* stable-6.3:
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: I6778c8266bc7e918c943dcabf23aa230f4e998d5
2022-11-20 20:24:14 +01:00
Matthias Sohn 246981c850 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: I71aa9f890c5eb05849ad16a00b9974da5e51171e
2022-11-20 20:22:24 +01:00
Matthias Sohn af87046e56 Remove unused imports
Change-Id: I57a3ae0a8608a517d9cc5cc27cee136f9faeb810
2022-11-20 20:18:01 +01:00
Matthias Sohn 3301032c6a Suppress non-externalized String warnings
Change-Id: I95c61bd906bd3f7cdea9581e0ffa92ec607c6140
2022-11-20 20:16:28 +01:00
Matthias Sohn f67cca6cb3 Remove unused API problem filters
Change-Id: I890716bc052dcce2ce136040035b442247c08332
2022-11-20 20:14:20 +01:00