Commit Graph

3439 Commits

Author SHA1 Message Date
Nitzan Gur-Furman c353645a09 Move footer-line parsing methods from RevCommit to FooterLine
This allows extracting footers from a messages not associated with a
commit.

The public API of RevCommit is kept intact.

Change-Id: I5809c23df7b7d49641a4be3a26d6f987d3d57c9b
Bug: Google b/287891316
2023-08-01 10:37:24 +02:00
Haamed Gheibi 462c57ec8d Merge: Add diff3 style merge conflict formatter.
Add base section to the merge conflict hunks.

Bug: 442284
Change-Id: I977b43e7dd8119d6b72d11f09c4e8ec241750383
2023-07-31 11:57:28 -07:00
Jonathan Tan ec11129b1d Merge changes I8c60d970,I09bdd4b8,I87ff3933
* changes:
  Pack: open reverse index from file if present
  PackReverseIndex: open file if present otherwise compute
  PackReverseIndex: verify checksums
2023-07-26 16:39:13 -04:00
Jonathan Tan c77fb93478 Merge "Identify a commit that generates a diffEntry on a rename Event." 2023-07-25 12:09:40 -04:00
Ronald Bhuleskar ec3d919aa5 Identify a commit that generates a diffEntry on a rename Event.
When using FollowFilter's rename callback, a callback is generated with the diff. The caller that is interested in the renames knows what the diff's are but have no idea what commit generated that diff.

This will allow FollowFilter's rename callback to track diffEntry for a given commit.

Change-Id: If1e63ccd19fdcb9c58c59137110fe24e0ce023d2
2023-07-24 19:42:51 -04:00
Jonathan Tan 0f4af2bc36 Merge changes I60a92463,Ic3b68220
* changes:
  PackReverseIndexV1: reverse index parsed from version 1 file
  ComputedPackReverseIndex: Clarify custom bucket sort algorithm
2023-07-21 14:05:38 -04:00
Anna Papitto 000e7caf5e PackReverseIndexV1: reverse index parsed from version 1 file
The reverse index for a pack is used to quickly find an object's
position in the pack's forward index based on that object's pack offset.
It is currently computed from the forward index by sorting the index
entries by the corresponding pack offset. This computation uses
insertion sort, which has an average runtime of O(n^2).

Cgit persists a pack reverse index file
to avoid recomputing the reverse index ordering. Instead they write a
file with format
https://git-scm.com/docs/pack-format#_pack_rev_files_have_the_format
which can later be read and parsed into the in-memory reverse index
each time it is needed.

PackReverseIndexV1 parses a reverse index file with the official
version 1 format into an in-memory representation of the reverse index
which implements methods to find an object's forward index position
from its offset in logorithmic time.

Change-Id: I60a92463fbd6a8cc9c1c7451df1c14d0a21a0f64
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-07-18 15:19:26 -07:00
Anna Papitto 2eba4e5b41 PackReverseIndex: open file if present otherwise compute
The existing #read and #computeFromIndex static builder methods require
the caller to choose whether to supply an input stream of a reverse
index file or a forward index to compute the reverse index from, which
is slower.

Allow a caller to provide a file path where the pack's reverse index
might be and the pack's forward index index and simply get some reverse
index instance back. Prefer opening and parsing the file if it is
present, to save computation time. Otherwise, fall back onto computing
the reverse index from the pack's forward index.

Change-Id: I09bdd4b813ad62c86add586417b2ab86e9331aec
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-07-18 15:19:26 -07:00
Anna Papitto 8123dcd699 PackReverseIndex: verify checksums
The new version 1 file-based reverse index has a footer with the
checksum of the corresponding pack file and a checksum of its own
contents. The initial implementation doesn't enforce that the pack
checksum matches the checksum found in the forward index nor that the
self checksum matches the contents of the file just read in.

Offer a method for reverse index users to verify the checksums in a way
appropriate to the version being used. For the pre-existing computed
version, always succeed since it is not based on a file so there is no
possibility of corruption.

Check for corruption of the file itself during parsing the checksum
footer, by comparing the self checksum with the digest of the file
contents read.

Change-Id: I87ff3933cf1afa76663350400b616695e4966cb6
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-07-18 15:19:26 -07:00
Ronald Bhuleskar 3b77e33ad8 CommitGraphWriter: add option for writing/using bloom filters
Currently, bloom filters are written and used without any way to turn
them off. Add a per-repo config variable to control whether bloom
filters are written. As for reading, add a JGit option to control this.
(A JGit option is used instead of a per-repo config variable as there is
usually no reason not to use the bloom filters if they are present, but
a global control to disable them is useful if there turns out to be an
issue with the implementation of bloom filters.)

The config that controls reading is the same as C Git, but the config
for writing is not: C Git has no config to control writing, but whether
bloom filters are written depends on whether bloom filters are already
present and what arguments are passed to "git commit-graph write". See
the manpage of "git commit-graph" for more information.

Change-Id: I1b7b25340387673506252b9260b22bfe147bde58
2023-07-18 14:21:48 -07:00
Jonathan Tan 77aec62141 CommitGraphWriter: reuse changed path filters
Teach CommitGraphWriter to reuse changed path filters that have been
read from the commit graph file whenever possible.

Change-Id: I1acbfa1613ca7198386a49209028886af360ddb6
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2023-07-18 14:21:48 -07:00
Jonathan Tan d3b40e72ac RevWalk: use changed path filters
Teach RevWalk, TreeRevFilter, PathFilter, and FollowFilter to use
changed path filters, whenever available, to speed revision walks by
skipping commits that fail the changed path filter.

This work is based on earlier work by Kyle Zhao
(I441be984b609669cff77617ecfc838b080ce0816).

Change-Id: I7396f70241e571c63aabe337f6de1b8b9800f7ed
Signed-off-by: kylezhao <kylezhao@tencent.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2023-07-18 14:21:48 -07:00
Jonathan Tan ff0f7c174f CommitGraphLoader: read changed-path filters
As described in the parent commit, add support for reading the BIDX and
BDAT chunks of the commit graph file, as described in man gitformat-
commit-graph(5).

This work is based on earlier work by Kyle Zhao
(I160f6b022afaa842c331fb9a086974e49dced7b2).

Change-Id: I82e02e6a3a3b758e6bf9d7bbd2198f0ffe3a331b
Signed-off-by: kylezhao <kylezhao@tencent.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2023-07-18 14:21:48 -07:00
Jonathan Tan 49beb5ae51 CommitGraphWriter: write changed-path filters
Add support for writing the BIDX and BDAT chunks of the commit graph
file, as described in man gitformat-commit-graph(5). The ability to read
such chunks will be added in a subsequent commit.

This work is based on earlier work by Kyle Zhao
(Ib863782af209f26381e3ca0a2c119b99e84b679c).

Change-Id: Ic18e6f0eeec7da1e1ff31751aabda5e6952dbe6e
Signed-off-by: kylezhao <kylezhao@tencent.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2023-07-18 14:21:48 -07:00
Matthias Sohn f41c5e8c01 Remove unnecessary @SuppressWarnings("serial")
Change-Id: Ib07fd89541dbd4b5095d49b2696429cbacc5bb4c
2023-07-16 15:14:19 +02:00
Matthias Sohn b2f7dc189a Remove redundant specification of type arguments
Change-Id: I8289e0a6ca9154d6411993d250176a35df7cb905
2023-07-16 15:11:17 +02:00
Ivan Frade 760bdd09b1 DfsPackParser: Create object indices if config says so
The DfsInserter writes the pack and its indices in the flush() method,
but when the writing happens via DfsPackParser, it is the parser which
writes the pack and indices. When combined with a parser, flushing the
inserter is a noop.

Add the writing of the object size index to the packparser#parse
method, mirroring how the primary index is written.

Change-Id: I52c5db153fea7e4a8ecd8b3d5de7ad21f7f81a60
2023-07-14 10:51:18 -07:00
Ivan Frade afb013b983 PackParserTest: Extract pack-writing helper code to its own class
PackParserTest has code to create a small pack in memory. We can use
it to test also the DfsPackParser.

Extract the helper functions to a new class (InMemoryPack).

Change-Id: I6dea4d6ea2cac2da8201702df92fd09a687e90a6
2023-07-14 10:34:56 -07:00
Ivan Frade cb99ff5bbb DfsInserter: generate object size index if config says so
DfsInserter receives objects and on flush() writes a pack and its
primary index.

Teach the DfsInserter to write also the object size index if the
config says so.

Change-Id: I89308312f8fd898d4c714a9b68ff948d3663800b
2023-07-14 10:34:46 -07:00
Ivan Frade 4d2a003b91 DfsInserter: populate full size on object insertion
We need the full size of the object to populate the object size index
later.

Save the size the PackedObjectInfo while adding objects to the
pack. Then we don't need to re-read it from the pack at indexing time.

Change-Id: I5bd7ad402df60b4637038def8ef7be2ab45faf87
2023-07-14 10:25:20 -07:00
Ivan Frade 12a4a4ccaa DFSGarbargeCollector: Write object size indices
PackWriter knows how to add an object size index to the pack, but the
garbage collector is not using it yet.

Teach DfsGarbageCollector to write the object size index on
writePack(). Disable by default in the unreachable-garbage pack.

Callers control the content/presence of the index through the
PackConfig option (minBytesForObjSizeIndex) for all other packs, so
there is no need of a specific flag in DfsGarbageCollector.

Change-Id: I86f5f17310e6913381125bec4caab32dc45b7c9d
2023-07-14 10:25:06 -07:00
Ivan Frade 9dace2e6d6 DfsReader/PackFile: Implement isNotLargerThan using the obj size idx
isNotLargerThan() can avoid reading the size of a blob from disk using
the object size idx if available.

Load the object size index in the DfsPackfile following the same
pattern than the other indices. Override isNotLargerThan in DfsReader
to use the index when available.

Following CL introduces the writing of the object size index and the
tests cover this code.

Change-Id: I15c95b84c1424707c487a7d29c5c46b1a9d0ceba
2023-07-13 11:24:17 -07:00
Luca Milanesio 88ca88a32b Add verification in GcKeepFilesTest that bitmaps are generated
The packfiles with the .keep extensions are meant to prevent
a packfile from being processed or removed during GC.
From the point of view of the GC process then, the associated
packfile should be completely transparent:
- it should not included in the repacked file
- it should not pruned
- its objects should be left untouched, even if unreachable
- the GC process, including the bitmap generation should continue
  as usual, as the the packfiles with .keep file did not exist

Add one explicit test for making sure that the management
of .keep file is also transparent to the generation of bitmaps,
which are still generated if a .keep file exists.

Bug: 582039
Change-Id: I14f6adc3f961c606fbc617e51ea6ed6e2ef8604f
2023-07-05 15:30:11 +02:00
Matthias Sohn dceebe350e Update mockito to 5.4.0 and bytebuddy to 1.14.5
Change-Id: Ia9de3f9fb6f51ac55a7c551cab4ce199318c1114
2023-07-02 19:16:14 +02:00
Matthias Sohn dd92c1aa98 Prepare 5.13.3-SNAPSHOT builds
Change-Id: I02b9388c8bc1c266bb29b4502504d137dd42142f
2023-06-22 02:15:21 +02:00
Matthias Sohn 5aa8a7e276 JGit v5.13.2.202306221912-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: Id0ee779fba85a6d5557f6319969adb2c74feebcf
2023-06-22 01:12:05 +02:00
Anna Papitto 8e61971620 PackReverseIndex: separate out the computed implementation
PackReverseIndex is a concrete class whose implementation is computed
from a pack's forward index. Callers which have a reverse index file may
want to use an implementation that is file-based instead.

Generalize PackReverseIndex into an interface without
implementation-specific logic and separate out the logic for the
computed implementation into a new concrete class.

Change-Id: I98d9835363c5e1c8c3c11a81b0761af3cdeaa41a
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-06-21 14:04:12 -07:00
Thomas Wolf faefa90f99 Default for global (user) git ignore file
C git has a default for git config core.excludesfile: "Its default
value is $XDG_CONFIG_HOME/git/ignore. If $XDG_CONFIG_HOME is either
not set or empty, $HOME/.config/git/ignore is used instead." [1]

Implement this in the WorkingTreeIterator$RootIgnoreNode.

To make this testable, mock the "user.home" directory for all JGit
tests, otherwise tests might pick up a real user's git ignore file.
Also ensure that JGit code always reads "user.home" via the
SystemReader.

Add tests for both locations.

[1] https://git-scm.com/docs/gitignore#_description

Bug: 436127
Change-Id: Ie510259320286c3c13a6464a37da1bd9ca1e373a
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-06-19 08:19:29 +02:00
Antoine Musso 7b955048eb Fix all Javadoc warnings and fail on them
This fixes all the javadoc warnings, stops ignoring doclint 'missing'
category and fails the build on javadoc warnings for public and
protected classes and class members.

Since javadoc doesn't allow access specifiers when specifying doclint
configuration we cannot set `-Xdoclint:all,-missing/private`
hence there is no simple way to skip private elements from doclint.
Therefore we check javadoc using the Eclipse Java compiler
(which is used by default) and javadoc configuration in
`.settings/org.eclipse.jdt.core.prefs` files.
This allows more fine grained configuration.

We can reconsider this when javadoc starts supporting access specifiers
in the doclint configuration.

Below are detailled explanations for most modifications.

@inheritDoc
===========
doclint complains about explicits `{@inheritDoc}` when the parent does
not have any documentation. As far as I can tell, javadoc defaults to
inherit comments and should only be used when one wants to append extra
documentation from the parent. Given the parent has no documentation,
remove those usages which doclint complains about.

In some case I have moved up the documentation from the concrete class
up to the abstract class.

Remove `{@inheritDoc}` on overriden methods which don't add additional
documentation since javadoc defaults to inherit javadoc of overridden
methods.

@value to @link
===============
In PackConfig, DEFAULT_SEARCH_FOR_REUSE_TIMEOUT and similar are forged
from Integer.MAX_VALUE and are thus not considered constants (I guess
cause the value would depends on the platform). Replace it with a link
to `Integer.MAX_VALUE`.

In `StringUtils.toBoolean`, @value was used to refer to the
`stringValue` parameter. I have replaced it with `{@code stringValue}`.

{@link <url>} to <a>
====================
@link does not support being given an external URL. Replaces them with
HTML `<a>`.

@since: being invalid
=====================

org.eclipse.jgit/src/org/eclipse/jgit/util/Equality.java has an invalid
tag `@since: ` due to the extra `:`. Javadoc does not complain about it
with version 11.0.18+10 but does with 11.0.19.7. It is invalid
regardless.

invalid HTML syntax
===================

- javadoc doesn't allow <br/>, <p/> and </p> anymore, use <br> and <p>
instead
- replace <tt>code</tt> by {@code code}
- <table> tags don't allow summary attribute, specify caption as
<caption>caption</caption> to fix this

doclint visibility issue
========================

In the private abstract classes `BaseDirCacheEditor` and
`BasePackConnection` links to other methods in the abstract class are
inherited in the public subclasses but doclint gets confused and
considers them unreachable. The HTML documentation for the sub classes
shows the relative links in the sub classes, so it is all correct. It
must be a bug somewhere in javadoc.
Mute those warnings with: @SuppressWarnings("doclint:missing")

Misc
====
Replace `<` and `>` with HTML encoded entities (`&lt; and `&gt;`).
In `SshConstants` I went enclosing a serie of -> arrows in @literal.

Additional tags
===============
Configure maven-javad0c-plugin to allow the following additional tags
defined in https://openjdk.org/jeps/8068562:
- apiNote
- implSpec
- implNote

Missing javadoc
===============
Add missing @params and descriptions

Change-Id: I840056389aa59135cfb360da0d5e40463ce35bd0
Also-By: Matthias Sohn <matthias.sohn@sap.com>
2023-06-16 01:08:13 +02:00
Anna Papitto 74547f4a68 PackReverseIndex: use static builder instead of constructor
PackReverseIndex instances are created using the constructor directly,
which limits control over the construction logic and refactoring
opportunities for the class itself. These will be needed for a
file-based implementation of the reverse index.

Use a static builder method to create a PackReverseIndex instance using
a pack's forward index.

Change-Id: I4421d907cd61d9ac932df5377e5e28a81679b63f
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-05-31 10:09:50 +02:00
Anna Papitto 181b629f7d Gc#writePack: write the reverse index file to disk
The reverse index is currently created in-memory when needed. A writer
for reverse index files was already implemented.

Make garbage collection write the reverse index file when the PackConfig
enables it. Write it during #writePack, which mirrors how the primary
index is written.

Change-Id: I50131af6622c41a7b24534aaaf2a423ab4178981
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-05-31 10:09:50 +02:00
David Ostrovsky f4eb38f9fe Bump bazel vesion to 6.2.0
Also demote severity level for bug pattern: PreferredInterfaceType, aka
MutableConstantField: [1] from error to warning.

Also fix DoubleBraceInitialization error prone bug pattern [2].

[1] https://errorprone.info/bugpattern/PreferredInterfaceType
[2] https://errorprone.info/bugpattern/DoubleBraceInitialization

Change-Id: I04b7d0792e67cbc51e3939bd47c2ea13a685cf0d
(cherry picked from commit 0f071a7bce)
2023-05-31 00:30:02 -04:00
Matthias Sohn ceeb9ad80d Prepare 6.6.1-SNAPSHOT builds
Change-Id: I0036999e2be076d4ad8231410faeff51bf9cbf52
2023-05-30 17:24:58 +02:00
Matthias Sohn d3962e0e72 JGit v6.6.0.202305301015-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I33b45b0cf36835b289ecbb5a1a9fc4ad7fc200cd
2023-05-30 16:11:53 +02:00
David Ostrovsky 0f071a7bce Bump bazel vesion to 6.2.0
Also demote severity level for bug pattern: PreferredInterfaceType, aka
MutableConstantField: [1] from error to warning.

Also fix DoubleBraceInitialization error prone bug pattern [2].

[1] https://errorprone.info/bugpattern/PreferredInterfaceType
[2] https://errorprone.info/bugpattern/DoubleBraceInitialization

Change-Id: I04b7d0792e67cbc51e3939bd47c2ea13a685cf0d
2023-05-29 18:39:59 +02:00
Matthias Sohn 264b91c3ee Prepare 6.6.0-SNAPSHOT builds
Change-Id: If0e4e8ce5f3e2f5170f313fb9b26b4ec0e34dab9
2023-05-24 18:23:45 +02:00
Matthias Sohn 9afff3e808 Prepare 6.7.0-SNAPSHOT builds
Change-Id: I50ff7ee31046cfc29a087c8963be3deae24b1c9c
2023-05-24 17:31:26 +02:00
Matthias Sohn 246f37d193 JGit v6.6.0.202305241045-m3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I204708812b9cb6f98f9c29e28548b91da0d88d91
2023-05-24 16:46:23 +02:00
Jonathan Tan 6b3b2b33a5 GraphObjectIndex: fix search in findGraphPosition
In findGraphPosition, when there is no object whose OID starts with
the first byte of the sought OID, low equals high. This violates an
invariant of the loop, and when the sought OID is lexicographically
greater than every other OID in the repository, causes an
ArrayIndexOutOfBoundsException (because we're trying to read outside the
list of OIDs).

Therefore, check the "low < high" condition at the start of the loop,
not only after the first iteration.

Change-Id: Ic8ac198c151bd161c4996b9e7cb6e6660f151733
Helped-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2023-05-23 13:57:32 -07:00
Simon Sohrt e6f216119f RewriteGeneratorTest: Introduce test cases for the RewriteGenerator
Bug: 577948
Signed-off-by: Simon Sohrt <sohrt@his.de>
Change-Id: I5af1a43d49379e2417611eaacdd5f06be8147bc4
2023-05-09 10:51:29 +02:00
Ivan Frade 73f9f55e3b Merge "PackWriter: write the PackReverseIndex file" 2023-05-08 15:00:46 -04:00
Anna Papitto ce88e62edc PackWriter: write the PackReverseIndex file
PackWriter offers the ability to write out the pack file and its various
index files, except for the newly introduced file-based reverse index.

Now that PackReverseIndexWriter can write reverse index files,
PackWriter#writeReverseIndex will write one for a pack if the
corresponding config flag PackConfig#writeReverseIndex is on.

Change-Id: Ib75dd2bbfb9ee9366d5aacb46700d8cf8af4823a
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-05-08 11:23:30 -07:00
Matthias Sohn 84461e8086 Update slf4j to 1.7.36 and consume it directly from Maven Central
BundleSymbolicNames changed to the names used upstream:
- slf4j.api
- slf4j.simple

Change-Id: Ibcaafae282d3a6fa013bc39d879def1b66073bca
2023-05-02 17:05:18 +02:00
Thomas Wolf 8c0c96e0a7 Support rebasing independent branches
With completely independent branches, there is no merge base. In this
case, the list of commits must include the root commit of the branch to
be rebased.

Bug: 581832
Change-Id: I0f5bdf179d5b07ff09f1a274d61c7a0b1c0011c6
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-04-29 13:24:58 +02:00
Thomas Wolf 8bc13fb79d Support cherry-picking a root commit
Handle the case of the commit to be picked not having any parents.

Since JGit implements cherry-pick as a 3-way-merge between the commit
to be picked and the target commit, using the parent of the picked
commit as merge base, this is super simple: just don't set a base tree.
The merger will not find any merge base and will supply an empty tree
iterator for the base.

Bug: 581832
Change-Id: I88985f1b1723db5b35ce58bf228bc48d23d6fca3
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-04-29 13:24:32 +02:00
Thomas Wolf 3ed4cdda6b AddCommand: ability to switch off renormalization
JGit's AddCommand always renormalizes tracked files. C git does so only
on git add --renormalize. Especially for git add . and the JGit
equivalent git.add().addFilepattern(".").call() this can make a big
difference if there are many files, or large files.

Add a "renormalize" option to AddCommand. To maintain compatibility with
existing uses, this option is "true" by default, and the behavior of
AddCommand is as it has always been in JGit.

If set to "false", use an IndexDiffFilter (in addition to a path filter,
if any). This skips any unchanged files (that are not racily clean) from
content checks. Note that changes in CRLF settings or in filters will be
ignored for such files if renormalize == false.

Add the "--renormalize" option to the Add command in the JGit command
line program. For the command-line program, the default is as in C git:
renormalize is off by default and enabled only if the option is given.
Note that --renormalize implies --update in the command line program, as
in C git. In AddCommand, the two settings are independent.

Additionally, avoid opening input streams unnecessarily in
WorkingTreeIterator.getEntryContentLength() and fix some bogus
indentation.

Add a simple test that adds 1000 files of 10kB in 10 directories twice
and that fails if the second invocation (without any changes) with
renormalize=false is not significantly faster.

Locally, I observe for that second invocation

* git.add().addFilepattern(".").call()                        ~660ms
* git.add().addFilepattern(".").setRenormalize(false).call()   ~16ms

Bug: 494323
Change-Id: I30f9d518563fa55d7058a48c27c425f3b60aeb4c
Signed-off-by: Thomas Wolf <twolf@apache.org>
2023-04-28 17:04:47 -04:00
Matthias Sohn 6b831000c5 [errorprone] Fix MissingOverride error
see https://errorprone.info/bugpattern/MissingOverride
Change-Id: Iec8833eb52d91a4ef117160407b9151f25617cb5
2023-04-28 21:46:19 +02:00
Matthias Sohn 2277f13041 Merge "Merge branch 'stable-6.5'" 2023-04-28 15:22:52 -04:00
Matthias Sohn 4d9db14a5e Merge branch 'stable-6.5'
* stable-6.5:
  [bazel] Move ToolTestCase to src folder (6.2)
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest
  Fix after_open config and Snapshotting RefDir tests to work with bazel
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  Parse pull.rebase=preserve as alias for pull.rebase=merges
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I16e8553d187a8ef541f578291f47fc39c3da4ac0
2023-04-28 19:51:01 +02:00
Anna Papitto 64615b44e6 PackReverseIndexWriter: write out version 1 reverse index file
The reverse index for a pack is used to quickly find an object's
position in the pack's forward index based on that object's pack offset.
It is currently computed from the forward index by sorting the index
entries by the corresponding pack offset. This computation uses
bucket sort with insertion sort, which has an average runtime of
O(n log n) and worst case runtime of O(n^2); and memory usage of
3*size(int)*n because it maintains 3 int arrays, even after sorting is
completed. The computation must be performed every time that the reverse
index object is created in memory.

In contrast, Cgit persists a pack reverse index file to avoid
recomputing the reverse index ordering every time that it is needed.
Instead they write a file with format
https://git-scm.com/docs/pack-format#_pack_rev_files_have_the_format
which can later be read and parsed into an in-memory reverse index each
time it is needed.

Introduce these reverse index files to JGit. PackReverseIndexWriter
writes out a reverse index file to be read later when needed. Subclass
PackReverseIndexWriterV1 writes a file with the official version 1
format.

To avoid temporarily allocating an Integer collection while sorting and
writing out the contents, using memory 4*size(Integer)*n, use an
IntList and its #sort method, which uses quicksort.

Change-Id: I6437745777a16f723e2f1cfcce4e0d94e599dcee
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-04-28 10:19:18 -07:00
Anna Papitto 7d3f893d31 IntList: add #sort using quick sort for O(n log n) runtime.
IntList is a class for working with lists of primitive ints without
boxing them into Integers. For writing the reverse index file format,
sorting ints will be needed but IntList doesn't provide a sorting
method yet.

Add the #sort method to sort an IntList by an IntComparator, using
quicksort, which has a average runtime of O(n log n) and sorts in-place
by using O(log n) stack frames for recursive calls.

Change-Id: Id69a687c8a16d46b13b28783b194a880f3f4c437
Signed-off-by: Anna Papitto <annapapitto@google.com>
2023-04-28 10:19:18 -07:00
Matthias Sohn 34a81889b8 Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  [bazel] Move ToolTestCase to src folder (6.2)
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest
  Fix after_open config and Snapshotting RefDir tests to work with bazel
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I6d20fea3a417e4361b61e81756253343668eb5de
2023-04-27 02:30:20 +02:00
Matthias Sohn f87c456e8a Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  [bazel] Move ToolTestCase to src folder (6.2)
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest
  Fix after_open config and Snapshotting RefDir tests to work with bazel
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I463f8528e623316add204848d551c44d44d04858
2023-04-27 02:20:10 +02:00
Matthias Sohn cdf35e8ead Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  [bazel] Move ToolTestCase to src folder (6.2)
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest
  Fix after_open config and Snapshotting RefDir tests to work with bazel
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I736c7d0ed9c6e9718fa98976c3dc5a25ab8cda85
2023-04-27 02:08:05 +02:00
Matthias Sohn 206f0f44f6 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest
  Fix after_open config and Snapshotting RefDir tests to work with bazel
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I9863cbce95d845efc891724898954b0b2f8dbf7b
2023-04-27 01:48:07 +02:00
Matthias Sohn 4451f466e0 Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest

Change-Id: I9c622de39200c6ee248d600736dc5db477f6bf9f
2023-04-27 01:37:29 +02:00
Matthias Sohn 74a13c7712 Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  GcConcurrentTest: @Ignore flaky testInterruptGc
  Fix CommitTemplateConfigTest

Adapt GcConcurrentTest to changed FileRepositoryBuilder API.

Change-Id: I8906662a6391ceed20ab29cfbe238daf13e1be63
2023-04-27 01:35:59 +02:00
Jonathan Tan 2fd050c567 GcConcurrentTest: @Ignore flaky testInterruptGc
During my development of Id7721cc5b7ea650e77c2db47042715487983cae6, I
have found this test to be flaky when run by CI. As a speculative fix,
mark this test as @Ignore so it won't be run.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Idfe04d7f1fb72a772d4c8d249ca86a9c2eec0b1a
2023-04-27 01:00:58 +02:00
Matthias Sohn 2aaa561136 Fix CommitTemplateConfigTest
The cherry-picked 61d4e313 doesn't match 5.13 APIs which changed in
newer versions.

Change-Id: I61ed0242472ed822028d86d3038f956f6bd5735c
2023-04-27 01:00:58 +02:00
Prudhvi Akhil Alahari 70bc5aad30 Fix after_open config and Snapshotting RefDir tests to work with bazel
The changes I1db6fcf414b and I634b92877f added tests which were failing
with errors [1] and [2] with "bazel test //...". This was not caught
because we don't have CI running with bazel. Fix bazel build file so
that these errors are no longer thrown when run with bazel.

[1] error: cannot find symbol FileRepositoryBuilderTest
[2] error: cannot find symbol RefDirectoryTest

Bug: 581816
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
Change-Id: I1e57111662825f5f14f373bc4f8d24cce1fec0b8
2023-04-26 22:01:05 +02:00
Matthias Sohn 6082ae25dd Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
  Demote severity of some error prone bug patterns to warnings
  UploadPack: Fix NPE when traversing a tag chain

Change-Id: I5e13d5b5414aef97e518898166bfa166c692e60f
2023-04-26 21:55:16 +02:00
Matthias Sohn 61d4e31349 [bazel] Skip ConfigTest#testCommitTemplatePathInHomeDirecory
Move this test to another class and skip it when running tests with
bazel since the bazel test runner does not allow to create files in the
home directory.

FS#userHome retrieves the home directory on the first call and caches it
for subsequent calls to avoid overhead in case path translation is
required (currently on cygwin). This prevents that the test can mock the
home directory using MockSystemReader like SshTestHarness does.

Change-Id: I6a22f37f4a19eb4b4935509eae508a23e56db7aa
2023-04-26 21:26:05 +02:00
Matthias Sohn 032eef5b12 Parse pull.rebase=preserve as alias for pull.rebase=merges
This ensures backwards compatibility to the old config value which was
removed in git 2.34 which JGit followed in Ic07ff954e2.

Change-Id: I2b4e27fd71898b6e0e227e406c40682bd9786cd4
2023-04-22 23:43:40 +02:00
Kaushik Lingarkar 064691e90c UploadPack: Fix NPE when traversing a tag chain
Always parse RevTags including their body before getting their object
to ensure that non-cached objects are handled correctly when traversing
a tag chain. An NPE in UploadPack#addTagChain will occur on a depth=1
fetch of a branch containing a tag chain and the ref to one of the
middle tags in the chain is deleted.

Change-Id: Ifd8fe868869070b365df926fec5dcd8e64d4f521
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-04-21 02:04:35 +02:00
Matthias Sohn 2e3f12a0fc Merge branch 'stable-6.5'
* stable-6.5:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I5bfff523b3174c7b741ab0eaf53937c3ab501252
2023-04-15 22:50:11 +02:00
Matthias Sohn 831da296d9 Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I7f822e8a751516a32afccd180cbf6afb389f3a28
2023-04-15 21:39:03 +02:00
Matthias Sohn 4ec1252f90 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I18cf0da3a5dcc74865c44d82e7c328329814acae
2023-04-15 21:38:27 +02:00
Matthias Sohn 34dc17ac3a Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: If0bb5f1a317ab981e6bbf5671851f124b18ab8ca
2023-04-15 21:35:16 +02:00
Matthias Sohn de5cb9a031 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Ief50a2ca8e5a8630627506f4d2142d62c0554615
2023-04-15 21:34:41 +02:00
Matthias Sohn 20b7e9435b Merge branch 'stable-6.0' into stable-6.1
* stable-6.0:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: Idf757bcab0d7a65ea63504674a681170c6db2f94
2023-04-15 00:49:59 +02:00
Matthias Sohn 273df319fe Merge branch 'stable-5.13' into stable-6.0
* stable-5.13:
  Remove blank in maven.config
  DirCache: support option index.skipHash

Change-Id: I0cc3033b1876c8c691c2a6876206cd71fa07d2e0
2023-04-15 00:49:08 +02:00
Matthias Sohn 060dcf1cca ListTagCommand: implement git tag --contains
Change-Id: I07e57ba098eace9656393837fad4cb3590f31b22
2023-04-12 13:56:52 +02:00
kylezhao 5cc9ecde8f RevWalk: use generation number to optimize getMergedInto()
A commit A can reach a commit B only if the generation number of A is
strictly larger than the generation number of B. This condition allows
significantly short-circuiting commit-graph walks.

On a copy of the Linux repository where HEAD is contained in v6.3-rc4
but no earlier tag, the command 'git tag --contains HEAD' of
ListTagCommand#call() had the following peformance improvement:
(excluded the startup time of the repo)

Before: 2649ms    (core.commitgraph=true)
        11909ms   (core.commitgraph=false)
After:  91ms     (core.commitgraph=true)
        11934ms   (core.commitgraph=false)

Bug: 574368
Change-Id: Ia2efaa4e9ae598266f72e70eb7e3b27655cbf85b
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-04-12 11:29:09 +08:00
Matthias Sohn 75db060673 Merge branch 'stable-6.5'
* stable-6.5:
  Ensure parsed RevCommitCG has derived data from commit-graph
  Downgrade maven-site-plugin to 3.12.1
  Use wagon-ssh-external to deploy Maven site

Change-Id: Ide721fb088fa04f6276ac495968a45e732f6e139
2023-04-06 22:16:41 +02:00
kylezhao d3ba40c803 Ensure parsed RevCommitCG has derived data from commit-graph
If a RevCommitCG was newly created and called #parseCanonical(RevWalk,
byte[]) method immediately, its flag was marked as PARSED, but no
derived data was obtained from the commit-graph. This is different from
what we expected.

Change-Id: I5d417efa3c42d211f19e6acf255f761e84d84450
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-04-06 20:05:13 +02:00
Jonathan Tan 3589c5cd1c PatchApplierTest: specify charset to avoid warning
Running

  bazel test //org.eclipse.jgit.test:org_eclipse_jgit_patch_PatchApplierTest

results in a DefaultCharset error on my machine, which can be avoided
by explicitly specifying a charset when calling getBytes on a string. In
these tests, the charset used doesn't really matter, so go with UTF_8.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Id7721cc5b7ea650e77c2db47042715487983cae6
2023-04-05 13:46:56 -07:00
Jonathan Tan 5a6e356082 GcConcurrentTest: @Ignore flaky testInterruptGc
During my development of Id7721cc5b7ea650e77c2db47042715487983cae6, I
have found this test to be flaky when run by CI. As a speculative fix,
mark this test as @Ignore so it won't be run.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Change-Id: Idfe04d7f1fb72a772d4c8d249ca86a9c2eec0b1a
2023-04-05 13:44:59 -07:00
Han-Wen NIenhuys d174684273 Merge "PatchApplier: Check for existence of src/dest files before any operation" 2023-03-31 06:24:32 -04:00
Nitzan Gur-Furman 903645835b PatchApplier: Check for existence of src/dest files before any operation
Change-Id: Ia3ec0ce1af65114b48669157a934f70f1e22fd37
Bug: Google b/271474227
2023-03-31 06:22:29 -04:00
Kaushik Lingarkar 33c00f3347 Implement a snapshotting RefDirectory for use in request scope
Introduce a SnapshottingRefDirectory class which allows users to get
a snapshot of the ref database and use it in a request scope (for
example a Gerrit query) instead of having to re-read packed-refs
several times in a request.

This can potentially be further improved to avoid scanning/reading a
loose ref several times in a request. This would especially help
repeated lookups of a packed ref, where we check for the existence of
a loose ref each time.

Change-Id: I634b92877f819f8bf36a3b9586bbc1815108189a
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:25:53 +02:00
Kaushik Lingarkar d7400517bf Fix RefDirectoryTest.testGetRefs_LooseSorting_Bug_348834
Since the first attempt to read a ref is not expected to trigger
a RefsChangedEvent, update the test to ensure 'lastNotifiedModCnt'
is not 0 before we start the actual work. The test has been passing
luckily because createBareRepository in setUp() happens to bump
'lastNotifiedModCnt'.

Change-Id: Ibd981f677920e8c3b965aa742fe669c42b8c1c93
Signed-off-by: Kaushik Lingarkar <quic_kaushikl@quicinc.com>
2023-03-30 22:25:53 +02:00
Ivan Frade a7f8f66523 Merge "BasePackFetchConnection: support negotiationTip feature" 2023-03-28 19:29:17 -04:00
Ronald Bhuleskar 738dacb7fb BasePackFetchConnection: support negotiationTip feature
By default, Git will report, to the server, commits reachable from all local refs to find common commits in an attempt to reduce the size of the to-be-received packfile. If specified with negotiation tip, Git will only report commits reachable from the given tips. This is useful to speed up fetches when the user knows which local ref is likely to have commits in common with the upstream ref being fetched.

When negotation-tip is on, use the wanted refs instead of all refs as source of the "have" list to send.

This is controlled by the `fetch.usenegotationtip` flag, false by default. This works only for programmatic fetches and there is no support for it yet in the CLI.

Change-Id: I19f8fe48889bfe0ece7cdf78019b678ede5c6a32
2023-03-28 14:29:54 -07:00
Matthias Sohn 23b9693a75 DirCache: support option index.skipHash
Support the new option index.skipHash which was introduced in git 2.40
[1]. If it is set to true skip computing the git index checksum. This
accelerates Git commands that manipulate the index, such as git add, git
commit, or git status. Instead of storing the checksum, write a trailing
set of bytes with value zero, indicating that the computation was
skipped.

Accept a skipped checksum consisting of 20 null bytes when reading the
index since the option could have been set to true at the time when the
index was written.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-indexskipHash

Bug: 581723
Change-Id: I28ebe44c5ca1cbcb882438665d686452a0c111b2
2023-03-28 23:16:08 +02:00
Ivan Frade ff74708466 PatchApplierTest: Remove test data with Apache license
Some test data introduced in [1] includes code with Apache
license. This triggers warnings in license analyzers.

Remove the licensed files and the test case as a quick relief. We
should restore the test with appropiate data in a follow-up change.

[1] https://git.eclipse.org/r/c/jgit/jgit/+/197724

Change-Id: I42670dc7d994f77d2c7f2c2156bcf1e112374022
2023-03-28 05:58:28 -04:00
Han-Wen NIenhuys 5166ded098 Merge "Fix PatchApplier error handling." 2023-03-28 05:51:18 -04:00
Nitzan Gur-Furman 3a913a8c34 Fix PatchApplier error handling.
1. For general errors, throw IOException instead of wrapping them with
PatchApplyException. The wrapping was moved (back) to ApplyCommand.
2. For file specific errors, log the errors as part of
PatchApplier::Result.
3. Change applyPatch() to receive the parsed Patch object, so the caller
can decide how to handle parsing errors.

Background: this utility class was extracted from ApplyCommand on V6.4.0.
During the extraction, we left the exception wrapping by
PatchApplyException intact. This attitude made it harder for the callers to
distinguish between the actual error causes.

Change-Id: Ib0f2b5e97a13df2339d8b65f2fea1c819c161ac3
2023-03-28 11:18:08 +02:00
Matthias Sohn a30c1da323 Merge branch 'stable-6.5'
* stable-6.5:
  Ensure FileCommitGraph scans commit-graph file if it already exists

Change-Id: I5218ff5214222c7d6d96e452cf427eea1f20c316
2023-03-27 11:02:52 +02:00
kylezhao 827849017d Ensure FileCommitGraph scans commit-graph file if it already exists
When commit-graph file already exists in the repository, a newly
created FileCommitGraph didn't scan CommitGraph until the file was
modified, resulting in wrong result.

Change-Id: Ic85676f2d3b6a88f3ae28d4065729926b6fb2f23
Signed-off-by: kylezhao <kylezhao@tencent.com>
2023-03-27 10:51:07 +02:00
Matthias Sohn 8dcb02140d Prepare 6.5.1-SNAPSHOT builds
Change-Id: Idd9977ac08a339906e33beb73f57f8f6885ad86f
2023-03-07 16:39:19 +01:00
Matthias Sohn c72dd241f4 JGit v6.5.0.202303070854-r
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I8da37ead0bd527bc4990ed5f8d5d4fb4f4d5cf01
2023-03-07 14:54:32 +01:00
Matthias Sohn 0687c73a12 Merge branch 'stable-6.5'
* stable-6.5:
  [errorprone] Suppress [Finally] warnings
  Update Orbit to R20230302014618 for 2023-03
  Improve test coverage when core.trustPackedRefsStat set to after_open
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302281825-rc1
  Prepare 6.5.0-SNAPSHOT builds
  JGit v6.5.0.202302221508-m3

Change-Id: Ice109c060d14c455262f61aed088111b238d735b
2023-03-03 16:04:00 +01:00
Matthias Sohn ec1dcfe500 Merge branch 'stable-6.4' into stable-6.5
* stable-6.4:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: Id0a3b571aa6bbf159b225794ddc884f79431c670
2023-03-03 16:03:26 +01:00
Matthias Sohn cd72ecbc07 Merge branch 'stable-6.3' into stable-6.4
* stable-6.3:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I236c5cbb638493cc4b23475be50d0f724d62a688
2023-03-03 16:02:49 +01:00
Matthias Sohn 9dd912a1c2 Merge branch 'stable-6.2' into stable-6.3
* stable-6.2:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I02d7444b28b662eb46f3d0ba1a62d342820bed1f
2023-03-03 16:02:16 +01:00
Matthias Sohn ebad3881b7 Merge branch 'stable-6.1' into stable-6.2
* stable-6.1:
  Improve test coverage when core.trustPackedRefsStat set to after_open

Change-Id: I63ba388ee2e174019733e63f8177792e1808a5c3
2023-03-03 16:01:26 +01:00
Prudhvi Akhil Alahari a4ca500d26 Improve test coverage when core.trustPackedRefsStat set to after_open
As of today, we don't have test coverage for RefDirectory when
core.trustPackedRefsStat config is set to after_open. Thus create new
test classes which set core.trustPackedRefsStat config to after_open in
setup and extend RefDirectoryTest and FileRepositoryBuilderTest
respectively.

Change-Id: I1db6fcf414bc488106ad4c85fb934480f299c995
Signed-off-by: Prudhvi Akhil Alahari <quic_prudhvi@quicinc.com>
2023-03-02 21:20:02 +05:30
Matthias Sohn f34ae6fe31 Prepare 6.6.0-SNAPSHOT builds
Change-Id: I17893f9db12bcb208866f40a06cd4f1ccbb4fe30
2023-03-01 15:40:45 +01:00
Matthias Sohn 69671a7026 Prepare 6.5.0-SNAPSHOT builds
Change-Id: I313e3deed8fa00df0406b3d7b73e5b643dc25a05
2023-03-01 15:30:29 +01:00
Matthias Sohn f43560a760 JGit v6.5.0.202302281825-rc1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Change-Id: I1eb2e87b70c2da1dc81468cdc7ecf7dbd21d4190
2023-03-01 00:23:58 +01:00