Commit Graph

6223 Commits

Author SHA1 Message Date
kylezhao 414bfe05ff CommitGraph: teach ObjectReader to get commit-graph
FileRepository's ObjectReader#getCommitGraph will return commit-graph
when it exists and core.commitGraph is true.

DfsRepository is not supported currently.

Change-Id: I992d43d104cf542797e6949470e95e56de025107
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-01-04 14:50:38 +08:00
Ivan Frade 93ac99b52a Merge "CommitGraph: add commit-graph for FileObjectDatabase" 2023-01-03 14:56:53 -05:00
Thomas Wolf 9a6d602488 PatchApplier: fix handling of last newline in text patch
If the last line came from the patch, use the patch to determine whether
or not there should be a trailing newline. Otherwise use the old text.

Add test cases for
- no newline at end, last line not in patch hunk
- no newline at end, last line in patch hunk
- patch removing the last newline
- patch adding a newline at the end of file not having one

all for core.autocrlf false, true, and input.

Add a test case where the "no newline" indicator line is not the last
line of the last hunk. This can happen if the patch ends with removals
at the file end.

Bug: 581234
Change-Id: I09d079b51479b89400ad300d0662c1dcb50deab6
Also-by: Yuriy Mitrofanov <a2terminator@mail.ru>
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-12-26 11:51:25 +01:00
kylezhao 8a7348df69 CommitGraph: add commit-graph for FileObjectDatabase
This change makes JGit can read .git/objects/info/commit-graph file
and then get CommitGraph.

Loading a new commit-graph into memory requires additional time. After
testing, loading a copy of the Linux's commit-graph(1039139 commits)
is under 50ms.

Bug: 574368
Change-Id: Iadfdd6ed437945d3cdfdbe988cf541198140a8bf
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-23 13:06:06 +08:00
Thomas Wolf aeb74f63d4 Reformat PatchApplier and PatchApplierTest
Some lines were too long, unnecessary fully qualified class names,
and an assertEquals(actual, expected) when it should have been
assertEquals(expected, actual).

Change-Id: I3b3c46c963afe2fb82a79c1e93970e73778877e5
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-12-22 23:30:02 +01:00
Anna Papitto 6722f25d56 PackWriter#prepareBitmapIndex: add clarifying comments
New readers of #prepareBitmapIndex may be confused about the manual
memory management (hidden mutation and nulling out pointers).

Add two clarifying comments to help future readers.

Change-Id: I93cab1919066efda37e96c47667f6991f67e377e
2022-12-20 13:56:11 -05:00
Ivan Frade fb3fd69584 Merge "IO#readFully: provide overload that fills the full array" 2022-12-19 13:50:08 -05:00
Anna Papitto 9b7c3ac11f IO#readFully: provide overload that fills the full array
IO#readFully is often called with the intent to fill the destination
array from beginning to end. The redundant arguments for where to start
and stop filling are opportunities for bugs if specified incorrectly or
if not changed to match a changed array length.

Provide a overloaded method for filling the full destination array.

Change-Id: I964f18f4a061189cce1ca00ff0258669277ff499
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-19 10:26:41 -08:00
Thomas Wolf f38a0614f5 Fix API warnings for the new CommitGraph
Mark the internal package as internal, visible only to the test bundle.
Add an API filter for CoreConfig.DEFAULT_COMMIT_GRAPH_ENABLE.

Change-Id: Ib62a93b873c93daf638b6c57e62fd267e16801bb
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-12-17 10:36:50 +01:00
Anna Papitto 0d10ebe560 PackReverseIndex#findPosition: fix typo in method name
The package-private findPostion method has a type in it. The typo will
become more widespread when a file-based implementation class is
introduced.

Correct the spelling to findPosition before the file-based
implementation is introduced.

Change-Id: Ib285f5a3f9a333ace1782dae9b5d425505eb962a
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-16 16:48:43 -05:00
kylezhao b082c58e0f GC: Write commit-graph files when gc
If 'core.commitGraph' and 'gc.writeCommitGraph' are both true, then gc
will rewrite the commit-graph file when 'git gc' is run. Defaults to
false while the commit-graph feature matures.

Bug: 574368
Change-Id: Ic94cd69034c524285c938414610f2e152198e06e
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 11:11:45 -05:00
kylezhao 7016e2ddae CommitGraph: add core.commitGraph config
Change-Id: I3b5e735ebafba09ca18fd83da479c7950fa3ea8d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 10:21:09 -05:00
Ivan Frade 6ea36794d1 Merge "Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering" 2022-12-16 08:20:24 -05:00
kylezhao 7b0f633b67 CommitGraph: implement commit-graph read
Git introduced a new file storing the topology and some metadata of
the commits in the repo (commitGraph). With this data, git can browse
commit history without parsing the pack, speeding up e.g.
reachability checks.

This change teaches JGit to read commit-graph-format file, following
the upstream format([1]).

JGit can read a commit-graph file from a buffered stream, which means
that we can provide this feature for both FileRepository and
DfsRepository.

[1] https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: Ib5c0d6678cb242870a0f5841bd413ad3885e95f6
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-16 06:57:06 -05:00
Anna Papitto 5c6c374ff6 Gc#deleteOrphans: avoid dependence on PackExt alphabetical ordering
Deleting orphan files depends on .pack and .keep being reverse-sorted
to before the corresponding index files that could be orphans. The new
reverse index file extension (.rev) will break that frail dependency.

Rewrite Gc#deleteOrphans to avoid that dependence by tracking which pack
names have a .pack or .keep file and then deleting any index files that
without a corresponding one. This approach takes linear time instead of
the O(n logn) time needed for sorting.

Change-Id: If83c378ea070b8871d4b01ae008e7bf8270de763
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-12-15 11:54:11 -08:00
Jeremy T. Braun 514ebfdc7e WalkPushConnection: Sanitize paths given to transports
These paths are given to the underlying URI-based transports (s3, sftp,
http), all of which expect forward-slash as the path separator
character.

Change-Id: I3cbb5928c9531a4da4691411bd8ac248fdf47ef2
2022-12-15 12:53:19 -05:00
Matthias Sohn 61f4a036c6 Merge changes Iad832fe1,Icef9658c
* changes:
  CommitGraphWriter: fix UnusedException errorprone error
  Update jetty to 10.0.13
2022-12-14 04:50:54 -05:00
Ivan Frade 91587aebd0 GraphCommits: Remove unused getter by position
CommitGraphWriter uses the GraphCommits in for-each loops and doesn't
need the access by position anymore. This was a left-over from
https://git.eclipse.org/r/c/jgit/jgit/+/182832

Remove the unused method.

Change-Id: I39df9bfab2601d581705ddf4cea3c04ed4765ff9
2022-12-13 14:37:37 -08:00
Matthias Sohn 04245494bb CommitGraphWriter: fix UnusedException errorprone error
Errorprone run in the bazel build raised this exception:

org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/commitgraph/CommitGraphWriter.java:105:
error: [UnusedException] This catch block catches an exception and
re-throws another, but swallows the caught exception rather than setting
it as a cause. This can make debugging harder.
		} catch (InterruptedIOException e) {
		  ^
    (see https://errorprone.info/bugpattern/UnusedException)
  Did you mean 'throw new
IOException(JGitText.get().commitGraphWritingCancelled, e);'?

Change-Id: Iad832fe17955fc1e60e6a4902bc50fd9dca76b9d
2022-12-13 15:04:45 +01:00
Xing Huang 3a136d2000 PackExt: Add a commit graph extension.
There is no commit graph PackExt because the non-DFS stack is not writing using PackExt mechanism. The extension is needed in DFS to determine the stream to write the commit-graph.

Add a commit graph extension that matches the one in cgit
(https://git-scm.com/docs/commit-graph#_file_layout)
in preparation for adding DFS support for reading and writing commit graphs.

Change-Id: Id14eda9f116a319124981e0bcbc533928b1b5e8c
Signed-off-by: Xing Huang <xingkhuang@google.com>
2022-12-12 17:26:12 -05:00
Matthias Sohn e74f3855ad Merge "commitgraph package: fix exports/imports, add @since tag for new API" 2022-12-08 14:10:23 -05:00
Sergey ec7c61eac3 BatchRefUpdate: Consistent switch branches in ref update
The expression RefUpdate ru = newUpdate(cmd) is eagerly evaluated before the switch statement.
But it is not used in some switch cases and thus is calculated uselessly.

Move expression evaluation to the switch case where it is actually used.
After such a move, several cases became identical and thus were squashed.

Change-Id: Ifd1976f1c28378e092fb24d7ca9c415cba49f07f
2022-12-08 05:27:35 -05:00
Sergey 2b21d9bbb6 RefWriter#writePackedRefs: Remove a redundant "if" check
After checking the variable, the same variable was checked again inside
the "if" block, and after the first check, this variable does not
change. Remove the second unnecessary check.

Change-Id: I6a38e67073f7f93105575b8f415ad32d350af602
2022-12-08 10:59:58 +01:00
Matthias Sohn ebc1f7d65c commitgraph package: fix exports/imports, add @since tag for new API
Change-Id: I9175b1d796f91f5ba4e21d3418550ae451c054b0
2022-12-08 02:00:58 +01:00
kylezhao cf70e7cbe4 CommitGraph: implement commit-graph writer
Teach JGit to write a commit-graph formatted file by walking commit
graph from specified commit objects.

See: https://git-scm.com/docs/commit-graph-format/2.21.0

Bug: 574368
Change-Id: I34f9f28f8729080c275f86215ebf30b2d05af41d
Signed-off-by: kylezhao <kylezhao@tencent.com>
2022-12-06 20:34:46 +08:00
Han-Wen NIenhuys 1d5a6c77a6 Merge "Fix crashes on rare combination of file names" 2022-11-28 09:34:46 -05:00
Matthias Sohn 2e28f27c26 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I4238b6181e96e22e540cf34802a332f868cb6dfb
2022-11-23 19:09:33 +01:00
alerthak c14e6f5c6c [pgm] Add options --name-only, --name-status to diff, log, show
Change-Id: Ib218bd2ccbd7990feca4c35d8c8dc34d4a8291e6
2022-11-23 16:12:54 +01:00
Matthias Sohn 9fc9ae6de9 Fix warning about non-externalized String
Change-Id: I00b2132eac6d6ad2701d3f8151eb8c80ad6d679d
2022-11-20 20:30:45 +01:00
Matthias Sohn fe3908076e Merge branch 'stable-6.3'
* stable-6.3:
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: I6778c8266bc7e918c943dcabf23aa230f4e998d5
2022-11-20 20:24:14 +01:00
Matthias Sohn 246981c850 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Remove unused imports
  Suppress non-externalized String warnings
  Remove unused API problem filters
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: I71aa9f890c5eb05849ad16a00b9974da5e51171e
2022-11-20 20:22:24 +01:00
Matthias Sohn 3301032c6a Suppress non-externalized String warnings
Change-Id: I95c61bd906bd3f7cdea9581e0ffa92ec607c6140
2022-11-20 20:16:28 +01:00
Matthias Sohn f67cca6cb3 Remove unused API problem filters
Change-Id: I890716bc052dcce2ce136040035b442247c08332
2022-11-20 20:14:20 +01:00
Matthias Sohn 71f50e52af Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Silence API errors
  Silence API errors
  Silence API warnings

Change-Id: Id3efa1b32710661cde63e93c5b8832785a805415
2022-11-20 20:10:37 +01:00
Matthias Sohn fec271c11b Silence API errors
Change-Id: I07c42fe9417edb0570dd475a7e935112a878a93b
2022-11-20 20:09:56 +01:00
Matthias Sohn 97ad9bdae6 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Silence API errors
  Silence API warnings

Change-Id: I2b8336652e60dec97666582cf9331c8505729473
2022-11-20 20:08:42 +01:00
Matthias Sohn 41b33a16b8 Silence API errors
Change-Id: Ie112b2099ea2125bc85863524e56f09ba4907373
2022-11-20 19:55:22 +01:00
Matthias Sohn 12f48276bd Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Silence API warnings

Change-Id: If5ab988a0e177c37b125e0b10625e506eeb2a74f
2022-11-20 19:54:44 +01:00
Matthias Sohn aa9f736c33 Silence API warnings
introduced by
- addition of configurable SHA1 implementation in 5.13.2
- 3-digit @since 5.9.1 annotations on GitServlet methods

Change-Id: If19853fcc5e3677e5b18e8e3fbbcd2773378dffc
2022-11-20 19:45:54 +01:00
Anna Papitto bd515f1c2a DfsBlockCache: Report IndexEventConsumer metrics for reverse indexes.
IndexEventConsumer metrics are reported per index PackExt and reverse
indexes did not have one, so they were not included.

Now that there is a REVERSE_INDEX PackExt, enable reporting
IndexEventConsumer metrics for reverse indexes.

Change-Id: Ia6a752f6eb8932a5b4ba45cc15cbc7e0786fd247
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-11-17 12:38:48 -05:00
Anna Papitto accacc27a1 DfsStreamKey: Replace ForReverseIndex to separate metrics.
Keys used for identifying reverse indexes in the DfsBlockCache use a
custom subclass ForReverseIndex because there was no PackExt for them.
This conflates BlockCacheMetrics for reverse indexes with those for
packs, since the key falls back onto 0 when there is no extension.

Replace the custom ForReverseIndex with a DfsStreamKey usage to bring
keys for the new REVERSE_INDEX extension in line with INDEX and BITMAP
and separate reverse index and pack BlockCacheMetrics.

Change-Id: I305e2c16d2a8cb2a824855ea92e0c9a9b188fce5
Signed-off-by: Anna Papitto <annapapitto@google.com>
2022-11-17 12:38:36 -05:00
Ivan Frade 5c033a98f6 Merge "PackExt: Add a reverse index extension." 2022-11-17 12:38:10 -05:00
Thomas Wolf 1c886d92f6 RawText.isBinary(): handle complete buffer correctly
Make sure we always get consistent results, whether or not we have the
full data in the buffer.

Change-Id: Ieb379a0c375ad3dd352e63ac2f23bda6ef16c215
Signed-off-by: Thomas Wolf <twolf@apache.org>
2022-11-16 15:17:19 -05:00
Matthias Sohn 0fb9d26eff Merge branch 'stable-6.3'
* stable-6.3:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Fix API breakage caused by extracting WorkTreeUpdater
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Ignore IllegalStateException if JVM is already shutting down
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Ib58879be292c54a2a7f4936ac0986997985c822b
2022-11-16 10:15:30 +01:00
Matthias Sohn 1cd9a1f804 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: I562be0802efa231023c5f10e6461339b2d7fbacf
2022-11-16 10:14:13 +01:00
Matthias Sohn a24b22632f Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Extract Exception -> HTTP status code mapping for reuse
  Don't handle internal git errors as an HTTP error
  Allow to perform PackedBatchRefUpdate without locking loose refs

Change-Id: Icb321779184d20f3871e236fda1a3acba605a6da
2022-11-16 10:13:20 +01:00
Matthias Sohn 77e2f4bd27 Merge "Merge branch 'stable-6.0' into stable-6.1" into stable-6.1 2022-11-16 04:10:43 -05:00
Matthias Sohn f3e0e9d5a3 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I9c1576011c11b4ff8f453d18d9e786cee59860fa
2022-11-16 09:56:08 +01:00
Matthias Sohn d588c2c9ad Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: Ie433c46a01a0f33848d54ecf99b30a44ca01e286
2022-11-16 09:55:22 +01:00
Matthias Sohn 7f36943d0c Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  [benchmarks] Remove profiler configuration
  Add SHA1 benchmark
  [benchmarks] Set version of maven-compiler-plugin to 3.8.1
  Fix running JMH benchmarks
  Add option to allow using JDK's SHA1 implementation
  Ignore IllegalStateException if JVM is already shutting down

Change-Id: I176419026c3f4fdd8ebd34c61468c1ec3482ff45
2022-11-16 09:54:28 +01:00