* stable-5.3:
SystemReader: Respect passed-in parent when no system config is present
Change-Id: I758ed0c2a3527ffcf6255158a49090aba090e942
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The default implementation of openSystemConfig has special handling for
when the FS returns null from getGitSystemConfig: it should return a
"real" FileBasedConfig instance that never actually tries to load a
file. However, this codepath was not respecting the passed-in parent
config.
Change-Id: Id0bcdc93bb42f9ebe3f5ee4c6b4be8863e0133f9
Every caller would need to check if bitmaps are available in the repo to
instantiate a reachability checker.
Offer a method to create the reachability checker in the walk: the
caller has already a walk over the repo, and the walk has all the
information required.
This allows us to make the implementation classes package-private.
Change-Id: I355e47486fcd9d55baa7cb5700ec08dcc230eea5
Signed-off-by: Ivan Frade <ifrade@google.com>
The new ReachabilityChecker interface and its implementations are marked
as @since 5.5, but they will make it to the 5.4 release.
Change-Id: I88c31b3300ccf35d18c35faddb2517f0a57bdcfd
Signed-off-by: Ivan Frade <ifrade@google.com>
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Some filesystems expose unique file identifiers, e.g. inodes in Posix
filesystems which are named filekeys in Java's BasicFileAttributes. Use
them as another means to detect file modifications based on stat
information.
Running git gc on a repository yields a new packfile with the same id as
a packfile which existed before the gc if these packfiles contain the
same set of objects. The content of the old and the new packfile might
differ if a different PackConfig was used when writing the packfile.
Considering filekeys in FileSnapshot may help to detect such packfile
modifications.
Bug: 546891
Change-Id: I711a80328c55e1a31171d540880b8e80ec1fe095
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
FileSnapshot.notRacyClean() assumed a worst case filesystem timestamp
resolution of 2.5 sec (FAT has a resolution of 2 sec). Instead measure
timestamp resolution to avoid unnecessary IO caused by false positives
in detecting the racy git problem caused by finite filesystem timestamp
resolution [1].
Cache the measured resolution per FileStore since timestamp resolution
depends on the respective filesystem type. If timestamp resolution
cannot be measured or fails due to an exception fallback to the worst
case FAT timestamp resolution and avoid caching this value.
Add a 10% safety margin in FileSnapshot.notRacyClean(), though running
FsTest.testFsTimestampResolution() 1000 times which is not using a
safety margin didn't fail on Mac using APFS and Java 8, 11, 12.
Measured Java file timestamp resolution: [2]
[1] https://github.com/git/git/blob/master/Documentation/technical/racy-git.txt
[2] https://docs.google.com/spreadsheets/d/1imy0y6WmRqBf0kjCxzxj2X7M50eIVfa7oaUIzEOHmjo
Bug: 546891
Change-Id: I493f3b57b6b306285ffa7d392339d253e5966ab8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* fix equals() and hashCode() methods to consider size
* fix toString() to show size
Change-Id: I5485db55eda5110121efd65d86c7166b3b2e93d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Check whether the value of the git config user.signingKey is a suffix
of the full fingerprint of the key. This was already used for finding
keys in secring.gpg, but not in pubring.kbx. This mechanism allows a
user to use any unique suffix to identify keys; to avoid needless
collisions it's recommended to use at least the last 16 characters of
the hex representation of the fingerprint, which is the key id.[1]
[1] https://tools.ietf.org/html/rfc4880#section-12.2
Bug: 545673
Change-Id: If6fb4879502b6ee4b8c26c21b2714aeac4e4670c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In 7b96bd812e ("UploadPack: Use reachability checker to validate
non-advertised wants", 2019-05-16), a "walk.setRetainBody(false);"
statement was inadvertently deleted. (An earlier version of this commit
had this line in another part of the code and a review comment suggested
to move it back here; the line was then deleted from the other part of
the code but not readded.) Restore this line.
Change-Id: I96ff6106ba9e4eef429388c83e898b3363295f69
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* changes:
UploadPack: Use reachability checker to validate non-advertised wants
BitmappedReachabilityChecker: Reachability check using bitmaps
BitmapCalculator: Get the reachability bitmap of a commit
ReachabilityChecker: Default implementation with a RevWalk
In "Reachable commit" request validators, we need to check that a "want"
in the request, that hasn't been advertised, is reachable from the refs
visible to the user.
Current code has intermixed the translation of ObjectIds to RevCommits
(and its error handling) with the actual walk, with the delegation to
bitmaps in restricted circunstances.
Refactor the code to make it "flatter" and more readable. Move ObjectIds
to RevCommits translation to its own functions. Use the reachability
checker instead of a newly defined walk.
Before the non-advertised wants were validated with bitmaps only if any
"want" refered to an non-commit. Now they will be validated with bitmaps
also if the "wants" refer all to commits.
Change-Id: Ib925a48cde89672b07a88bba4e24d0457546baff
Signed-off-by: Ivan Frade <ifrade@google.com>
The "basic" reachability check walks the graph starting from the tips
marking things as "uninteresting". If the target commit is marked as
"uninteresting" it was reached; it is reachable from those tips.
This requires a lot of walking and can be solved directly with bitmaps.
Most of the time the bitmaps are already calculated or a short walk
away.
This should improve the performance of reachability checks, for example
in Gitiles.
Change-Id: I83d33271f58d95d2dc9ed151967b3eda513c99f7
Signed-off-by: Ivan Frade <ifrade@google.com>
To make reachability checks with bitmaps, we need to get the
reachability bitmap of a commit, which is not always precalculated.
There is already a class returning such bitmap (BitmapWalker) but it
does too much unnecessary work: it calculates ALL reachable objects from
a commit (i.e. including trees and blobs), when for reachability the
commits are just enough.
Introduce BitmapCalculator to get the bitmap of a commit: either because
it is precalculated or generating it with a walk only over commits.
Change-Id: Ibb6c78affe9eeaf1fa362a06daf4fd2d91c1caea
Signed-off-by: Ivan Frade <ifrade@google.com>
It is common to check if a certain commit is reachable from some
starting points. For example gitiles does it to check if a commit
is visible to a user based on its permissions.
Offer this functionality in JGit.
Split the interface as the next commit will introduce an implementation
using bitmap indices.
Change-Id: I0933b305c8d734f7a64502910ff4d9ef4fc92ae1
Signed-off-by: Ivan Frade <ifrade@google.com>
- ObjectWalk.getTreeDepth() returns int hence there is no need to use
long depths in the lowestDepthVisited map.
- Also fix boxing warnings introduced in 0a15cb3a.
Change-Id: I6d73b6f41d5d20975d02f376c8588e411eaff0ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Replace:
in the "core section"
in the "diff section"
in the "merge section"
with:
in the "core" section
in the "diff" section
in the "merge" section
Change-Id: Ided7bf73e9c8aae4fc4e43d5d5b9f6d7e3066f0a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.3:
Prepare 5.3.2-SNAPSHOT builds
JGit v5.3.1.201904271842-r
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I8a8671f7767444a77b809bd66a27d776c8332736
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: Ie7e572ac7e346f21fe0c387d7448be168a9c127a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Making gitlinks and folders match in a tree walk was the wrong
approach to fix bug 467631. The problem is that in such a conflict
the tree walk may then not descend into the folder.
Revert the changes to Paths.java and PathsTest.java from commit
4678f4b. Instead test for the problem case from bug 467631 explicitly
in IndexDiff. Add Daniel's test case from bug 545162, and add yet
another test case for DiffEntry.scan() that covers the problem
originally reported in bug 545162.
Bug: 545162
Change-Id: Ie2214c5d5ee32ac6596b621f0f1c7b86d38fa9b7
Also-by: Daniel Veihelmann <daniel.veihelmann@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-4.8:
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: Idaa789e699f1ef568ea957184d0641355d9e3181
* stable-4.7:
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I7984f68833f2d615399296e53cb9a64e5b4ca6ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.6:
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I6ea51dde6608a4163d681aa1ebf710f06da44b21
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.5:
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I0b4eaa521ebdea83ab18c05915d691c07a575a7d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a tree is visited during pack and filtered out with tree:<depth>, we
may need to include it if it is visited again at a lower depth.
Until now we revisit it no matter what the depth is. Now, avoid
visiting it if it has been visited at a lower or equal depth.
Change-Id: I68cc1d08f1999a8336684a05fe16e7ae51898866
Signed-off-by: Matthew DeVore <matvore@gmail.com>
If we are traversing a tree which is too deep, then there is no need to
traverse the children. Skipping children is much faster than traversing
the possibly thousands of objects which are directly or indirectly
referenced by the tree.
Change-Id: I6d68cc1d35da48e3288b9cc80356a281ab36863d
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This is used when fetching, and in particular to populate a partial
clone or a virtual file system cache as the user navigates. With this,
a client can pre-fetch a few directories deeper than only the current
directory.
depth:0 will omit all trees, and is useful if you only want to fetch
the commits of a repository, or fetch just a single tree or blob object.
depth:1 will fetch only the root tree of all commits fetched. depth:2
will fetch the root tree and all blobs and tree objects directly
referenced from it. depth:3 gets one more level, and so on. depth:#
will not filter a blob or tree that is explicitly marked wanted.
Bitmaps are disabled when this filter is used.
This implementation is quite slow because it iterates over all omitted
objects rather than skipping them. This will be addressed in follow-up
commits.
Change-Id: Ic312fee22d60e32cfcad59da56980e90ae2cae6a
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This reverts commit 39b0b51b12. Before
that change, SubmoduleWalk.forPath transferred ownership to the caller
on success. Afterward, it returns a closed SubmoduleWalk to the caller,
which does not appear to be intentional.
Change-Id: I9381daac5153706e24fd9117700089848b58c54e
Replace simple uses of Iterator with a corresponding for-loop.
Also add missing braces on loops as necessary.
Change-Id: I708d82acdf194787e3353699c07244c5ac3de189
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
DepthGenerator marks commits reinteresting for the ones that are
reachable from unshallow commits as it walks over the revisions. Those
unshallow commits won't necessarily be processed first. Because of this,
even if a commit is reachable from unshallow commits, if it's processed
before the uninteresting commits, it will not be processed as
reinteresting and processed as uninteresting. This causes unshallow
git-fetch to be failed.
This changes DepthGenerator to process unshallow commits first
independent to their depth. This makes uninteresting flag carry work
properly.
Change-Id: I94378271cf85fbe6302cefc19a167d8cf68e1a69
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Keep track of the original cause for a packfile invalidation.
It is needed for the sysadmin to understand if there is a real
underlying filesystem problem and repository corruption or if it is
simply a consequence of a concurrency of Git operations (e.g. repack
or GC).
Change-Id: I06ddda9ec847844ec31616ab6d17f153a5a34e33
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since Java 7 the diamond operator can be used instead of explicit
type parameters.
Change-Id: I2dee5fce7afebb1d9088eeaec4484ee58b4fa492
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Use of String.equals("") can be replaced with with String.length() == 0
(for JDK5 and lower) or String.isEmpty() (for JDK6 and higher)
Change-Id: Id1462d22c5d249485d87993263a9239809e73c55
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Do not reload packfiles when their associated filesnapshot is not
modified on disk compared to the one currently stored in memory.
Fix the regression introduced by fef78212 which, in conjunction with
core.trustfolderstats = false, caused any lookup of objects inside
the packlist to loop forever when the object was not found in the pack
list.
Bug: 546190
Change-Id: I38d752ebe47cefc3299740aeba319a2641f19391
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add resolveTipSha1, an inverse of exactRef(String ...), to RefDatabase
and provide a default implementation that runs in O(n) time where n is
the number of refs. For RefTable, provide an implementation that runs
in O(log(n)) time.
[ifrade@google.com: with tests in InMemoryRepositoryTest to exercise
the reftable code path, too]
Change-Id: I2811ccd0339cdc1c74b42cce2ea003f07a2ce9e1
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Ivan Frade <ifrade@google.com>
Use Files.delete() instead of File.delete(), and if there is
an exception thrown propagate it unless errors are to be ignored so
that the actual deletion failure cause is available to the caller
(and will be logged).
Change-Id: I5fdb5a4052942437ab365289ad4bb1b563c29456
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add missing source bundles for
- org.eclipse.jgit.ant
- org.eclipse.jgit.archive
- org.eclipse.jgit.http.apache
- org.eclipse.jgit.http.server
- org.eclipse.jgit.junit
- org.eclipse.jgit.junit.http
- org.eclipse.jgit.junit.ssh
- org.eclipse.jgit.lfs
- org.eclipse.jgit.lfs.server
- org.eclipse.jgit.ui
Combine all source bundles into a single source feature
org.eclipse.jgit.source and delete the other source features.
Ensure all bundles are added to the jgit p2 repository.
Change-Id: I56785f49c940b79f41f763c26e63a4a820ed7cce
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The new references created in the peeling do not receive the update
index. In other words, the update index of a reference (if set) is lost
in the peeling.
Pass-through the update index to the newly created references.
Tested via InMemoryRepository, which uses DfsReftableDatabase.
Change-Id: I7ff7c737a9c3366fdec296a4d9b2e51d10227957
Signed-off-by: Ivan Frade <ifrade@google.com>
Code that creates references and wants to support ref dbs with and
without update indexes needs to set this value. This leds to a
proliferation of "-1" in the code base.
Make the "undefined" value a constant in the ref interface.
Change-Id: I2622a37536a84b4a4036dd55792e185486fa0628
Signed-off-by: Ivan Frade <ifrade@google.com>
- UploadPack.getFilterBlobLimit() didn't have a wrong @since tag but was
marked final shortly after 5.3 was released. This was probably caused by
using an outdated API baseline. Currently we should use 5.3.0 as the
baseline.
- remove unused filter on FS.fileAttributes()
Change-Id: I9adc1703e99a9ddb3ea2a1c12a83dccbc1f69a99
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use .api_filters to suppress API baseline errors. The justification is
included in comments in that file.
Change-Id: I7b5c69d8d13733719b28ef7317fc9780ed2828d8
Signed-off-by: Matthew DeVore <matvore@gmail.com>
Inline newTreeVisit into enterTree and call the new method pushTree. Use
pushTree both for pushing children of the existing currVisit.
Change-Id: I75ea37f48b2befb738a3e88bed40ac08f1df9a03
Signed-off-by: Matthew DeVore <matvore@gmail.com>
Use the FilterSpec object so that less code has to know about the make-up of
FilterSpecs. When fields are added to FilterSpec, these pieces of code won't
need updating again.
Change-Id: I2b9e59a9926ff112faf62a3fa2d33c961a1779e5
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This increases type-safety and is ground work for support of the
"tree:<depth>" filter.
Change-Id: Id19eacdcdaddb9132064c642f6d554b1060efe9f
Signed-off-by: Matthew DeVore <matvore@gmail.com>
When using @NonNull annotations in new code, if I write
public void setFrobber(@NonNull frobber) {
this.frobber = frobber;
}
then consumers of the JGit library that do not have nullness checking
enabled can easily pass in null by mistake. On the other hand, if I
write
public void setFrobber(@NonNull frobber) {
if (frobber == null) {
throw new NullPointerException();
}
this.frobber = frobber;
}
then Eclipse JDT complains:
Null comparison always yields false: The variable frobber is specified as @NonNull
Add a checkNotNull helper that offers the best of both worlds:
public void setFrobber(@NonNull frobber) {
this.frobber = checkNotNull(frobber);
}
Briefer code, null check is intact, and no warning.
Inspired by Guava's com.google.common.base.Preconditions.checkNotNull.
Change-Id: If59588d13a1119e899657ed2296931ea18ed0e2a
This code is not specific to V2 of the protocol, so it doesn't belong
there. Move it to its own class.
Change-Id: I8084764147d1d0e2bceaaead59b332a8ec1a0825
Signed-off-by: Matthew DeVore <matvore@gmail.com>
HashMap<String, Ref> has a memory overhead for refs. Use RefMap.
Change-Id: I3fb4616135dacf687cc3bc2b473effc66ccef5e6
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
The prune method did not delete empty fanout directories when loose
objects moved to a new pack file but only when loose unreferenced
objects were pruned.
Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ensure the sentences to document fields consistenly end with a dot.
Change-Id: I084899cafe4dea6f2424132dce9f3adca2e20985
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
One occurrence in core JGit, several in jgit.pgm. One unused
occurrence in jgit.pgm; remove it.
Change-Id: I04c3dd9d9f542f1e1ac6df4cbf03bcefb6bfdf78
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
RefAdvertiser.send(Map<String, Ref>) is deprecated in favour of
RefAdvertiser.send(Collection<Ref>). Subclasses that need to override
the "send" method need to override also the deprecated version, because
it is still invoked by BaseReceivePack and UploadPack.
Remove the last usages of the deprecated method.
Change-Id: I7eba426970251f78801ddf96b87a65d1baaebdcf
Signed-off-by: Ivan Frade <ifrade@google.com>
* stable-5.3:
Prepare 5.3.1-SNAPSHOT builds
JGit v5.3.0.201903130848-r
Prepare 5.1.7-SNAPSHOT builds
JGit v5.1.6.201903130242-r
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ief9f6a9e89c4a1ef122daedbfd933676393fab06
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.1.7-SNAPSHOT builds
JGit v5.1.6.201903130242-r
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ia866587a910e620cb297bcb2a81fe0786f4e3db2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
David Pursehouse