* stable-5.1:
Prepare 5.1.9-SNAPSHOT builds
JGit v5.1.8.201906050907-r
Test detecting modified packfiles
Enhance fsTick() to use filesystem timer resolution
Add debug trace to measure time needed to open pack index
Extend FileSnapshot for packfiles to also use checksum to detect changes
Wait opening new packfile until it can't be racy anymore
Avoid null PackConfig in GC
Add FileSnapshot test testing recognition of file size changes
Capture reason for result of FileSnapshot#isModified
Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows
Tune max heap size for tests
Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc
ObjectDirectory: fix closing of obsolete packs
Include filekey file attribute when comparing FileSnapshots
Measure file timestamp resolution used in FileSnapshot
Fix FileSnapshot's consideration of file size
Fix API problem filters
Change-Id: I3ac77bfa03f7436de12ab86e1bba29afee5ccd01
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Test that JGit detects that packfiles have changed even if they are
repacked multiple times in one tick of the filesystem timer.
Test that this detection works also when repacking doesn't change the
length or the filekey of the packfile. In this case where a modified
file can't be detected by looking at file metadata JGit should still
detect too fast modification by racy git checks and trigger rescanning
the pack list and consequently rereading of packfile content.
Change-Id: I67682cfb807c58afc6de9375224ff7489d6618fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If the attributes of FileSnapshot don't detect modification of a
packfile read the packfile's checksum and compare it against the
checksum cached in the loaded packfile.
Since reading the checksum needs less IO than reloading the complete
packfile this may help to reduce the overhead to detect modficiation
when a gc completes while ObjectDirectory scans for packfiles in another
thread.
Bug: 546891
Change-Id: I9811b497eb11b8a85ae689081dc5d949ca8c4be5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
PacketLineIn.END is only referenced in tests. Replace most of those
with a new package visible end() method.
Remaining usages of PacketLineIn.END are in the form:
while ((line = pckIn.readString()) != PacketLineIn.END) {
and are not trivial replacements, hence are not touched in this change.
Change-Id: Id77c5321ddcad127130b246bde8f08736e60e1ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Deprecate DELIM with the intention of making it private in a future
release.
Callers that want to test if a packet line string is the delimiter
should use the isDelimiter(String) method.
The only other references to DELIM in the JGit code are in tests. For
those, introduce a package visible delimiter() method.
Change-Id: I21e8bbac0ffb9ef710c9753e23435416b09a4891
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace reference comparisons of PacketLineIn's DELIM and END strings
with usage of the helper methods isDelimiter() and isEnd().
Change-Id: I52dcfc4ee9097f1bd6970601c716701847d9eebd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Also now use JcaKeyBoxBuilder constructor in
BouncyCastleGpgKeyLocator.readKeyBoxFile(Path).
CQ: 19868
CQ: 19869
CQ: 19870
Change-Id: I45bd80e158aecd90448b0c7e59615db27aaef892
Signed-off-by: Brandon Weeks <bweeks@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
These methods will allow clients to check for END and DELIM without
doing a reference comparison on the String objects, which raises
warnings from Error Prone.
Change-Id: I9e7e59843553ed4488ee8e864033198bbb60d67c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This allows to verify the expected behavior in
FileSnapshotTest#testSimulatePackfileReplacement and enables extending
FileSnapshot for packfiles to read the packfile's checksum as another
criterion to detect modifications without reading the full content.
Also add another field capturing the result of the last check if
lastModified was racily clean.
Remove unnecessary determination of raciness in the constructor. It was
determined twice in all relevant cases.
Change-Id: I100a2f49d7949693d7b72daa89437e166f1dc107
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
NTFS does not support FileKey hence ignore this test on Windows.
Change-Id: I7b53a591daa5e03eb5e401b5b26d612ab68ce10d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This is an attempt to fix crashes observed on the new Jenkins
infrastructure running on Kubernetes [1].
Increase it to 512m for
- org.eclipse.jgit.ant.test
- org.eclipse.jgit.http.test
- org.eclipse.jgit.lfs.server.test
- org.eclipse.jgit.lfs.test
- org.eclipse.jgit.pgm.test
Decrease it to 768m for
- org.eclipse.jgit.test
[1] e.g. https://ci-staging.eclipse.org/jgit/job/stable/job/jgit.gerrit/16074/console
Change-Id: Id074ed0f7bcb8a13da649a547342af2a08439d9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit e19e859977)
testNewFileNoWait() was identical to testNewFileWithWait() but claims it
doesn't wait at all. Hence remove the waits.
Change-Id: I49b8ca5cb49a43c55fe61870c18c42f32fb4b74d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This is an attempt to fix crashes observed on the new Jenkins
infrastructure running on Kubernetes [1].
Increase it to 512m for
- org.eclipse.jgit.ant.test
- org.eclipse.jgit.http.test
- org.eclipse.jgit.lfs.server.test
- org.eclipse.jgit.lfs.test
- org.eclipse.jgit.pgm.test
Decrease it to 768m for
- org.eclipse.jgit.test
[1] e.g. https://ci-staging.eclipse.org/jgit/job/stable/job/jgit.gerrit/16074/console
Change-Id: Id074ed0f7bcb8a13da649a547342af2a08439d9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Some filesystems expose unique file identifiers, e.g. inodes in Posix
filesystems which are named filekeys in Java's BasicFileAttributes. Use
them as another means to detect file modifications based on stat
information.
Running git gc on a repository yields a new packfile with the same id as
a packfile which existed before the gc if these packfiles contain the
same set of objects. The content of the old and the new packfile might
differ if a different PackConfig was used when writing the packfile.
Considering filekeys in FileSnapshot may help to detect such packfile
modifications.
Bug: 546891
Change-Id: I711a80328c55e1a31171d540880b8e80ec1fe095
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
FileSnapshot.notRacyClean() assumed a worst case filesystem timestamp
resolution of 2.5 sec (FAT has a resolution of 2 sec). Instead measure
timestamp resolution to avoid unnecessary IO caused by false positives
in detecting the racy git problem caused by finite filesystem timestamp
resolution [1].
Cache the measured resolution per FileStore since timestamp resolution
depends on the respective filesystem type. If timestamp resolution
cannot be measured or fails due to an exception fallback to the worst
case FAT timestamp resolution and avoid caching this value.
Add a 10% safety margin in FileSnapshot.notRacyClean(), though running
FsTest.testFsTimestampResolution() 1000 times which is not using a
safety margin didn't fail on Mac using APFS and Java 8, 11, 12.
Measured Java file timestamp resolution: [2]
[1] https://github.com/git/git/blob/master/Documentation/technical/racy-git.txt
[2] https://docs.google.com/spreadsheets/d/1imy0y6WmRqBf0kjCxzxj2X7M50eIVfa7oaUIzEOHmjo
Bug: 546891
Change-Id: I493f3b57b6b306285ffa7d392339d253e5966ab8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
4e196faa1b ("ReachabilityChecker: Default implementation with a
RevWalk", 2019-05-15) added ReachabilityCheckerTestCase but did not add
it as a test helper in the corresponding BUILD file, making tests no
longer runnable with Bazel. Resolve this issue.
Change-Id: Iccc00b0d169dbaa137e130ce2ddd1b7669960b52
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* changes:
UploadPack: Use reachability checker to validate non-advertised wants
BitmappedReachabilityChecker: Reachability check using bitmaps
BitmapCalculator: Get the reachability bitmap of a commit
ReachabilityChecker: Default implementation with a RevWalk
The "basic" reachability check walks the graph starting from the tips
marking things as "uninteresting". If the target commit is marked as
"uninteresting" it was reached; it is reachable from those tips.
This requires a lot of walking and can be solved directly with bitmaps.
Most of the time the bitmaps are already calculated or a short walk
away.
This should improve the performance of reachability checks, for example
in Gitiles.
Change-Id: I83d33271f58d95d2dc9ed151967b3eda513c99f7
Signed-off-by: Ivan Frade <ifrade@google.com>
To make reachability checks with bitmaps, we need to get the
reachability bitmap of a commit, which is not always precalculated.
There is already a class returning such bitmap (BitmapWalker) but it
does too much unnecessary work: it calculates ALL reachable objects from
a commit (i.e. including trees and blobs), when for reachability the
commits are just enough.
Introduce BitmapCalculator to get the bitmap of a commit: either because
it is precalculated or generating it with a walk only over commits.
Change-Id: Ibb6c78affe9eeaf1fa362a06daf4fd2d91c1caea
Signed-off-by: Ivan Frade <ifrade@google.com>
It is common to check if a certain commit is reachable from some
starting points. For example gitiles does it to check if a commit
is visible to a user based on its permissions.
Offer this functionality in JGit.
Split the interface as the next commit will introduce an implementation
using bitmap indices.
Change-Id: I0933b305c8d734f7a64502910ff4d9ef4fc92ae1
Signed-off-by: Ivan Frade <ifrade@google.com>
Add encrypted ed25519 keys in the tests; sshd 2.2.0 can finally
decrypt encrypted new-style OpenSSH key files. (Needs the "unlimited
strength" JCE, which is the default since Java 8u161. On older JREs,
users should install the policy files available from Oracle.)
The "expensive" key added has been generated with OpenSSH's
ssh-keygen -t ed25519 -a 256, i.e., with 256 bcrypt KDF rounds
instead of the default 16. On my machine it takes about 2sec to
decrypt.
Bug: 541703
Change-Id: Id3872ca2fd75d8f009cbc932eeb6357d3d1f267c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Update target platforms, maven and bazel builds to use sshd 2.2.0.
Adapt internal classes to changed sshd interfaces and remove previous
work-arounds for asking repeatedly for key passwords and for loading
keys lazily; both are now done by sshd.
CQ: 19034
CQ: 19035
Bug: 541425
Change-Id: I85e1df6ebb8a94953a912d9b2b8a7b5bdfbd608a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Prepare 5.3.2-SNAPSHOT builds
JGit v5.3.1.201904271842-r
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I8a8671f7767444a77b809bd66a27d776c8332736
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: Ie7e572ac7e346f21fe0c387d7448be168a9c127a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Making gitlinks and folders match in a tree walk was the wrong
approach to fix bug 467631. The problem is that in such a conflict
the tree walk may then not descend into the folder.
Revert the changes to Paths.java and PathsTest.java from commit
4678f4b. Instead test for the problem case from bug 467631 explicitly
in IndexDiff. Add Daniel's test case from bug 545162, and add yet
another test case for DiffEntry.scan() that covers the problem
originally reported in bug 545162.
Bug: 545162
Change-Id: Ie2214c5d5ee32ac6596b621f0f1c7b86d38fa9b7
Also-by: Daniel Veihelmann <daniel.veihelmann@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If a tree is visited during pack and filtered out with tree:<depth>, we
may need to include it if it is visited again at a lower depth.
Until now we revisit it no matter what the depth is. Now, avoid
visiting it if it has been visited at a lower or equal depth.
Change-Id: I68cc1d08f1999a8336684a05fe16e7ae51898866
Signed-off-by: Matthew DeVore <matvore@gmail.com>
If we are traversing a tree which is too deep, then there is no need to
traverse the children. Skipping children is much faster than traversing
the possibly thousands of objects which are directly or indirectly
referenced by the tree.
Change-Id: I6d68cc1d35da48e3288b9cc80356a281ab36863d
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This is used when fetching, and in particular to populate a partial
clone or a virtual file system cache as the user navigates. With this,
a client can pre-fetch a few directories deeper than only the current
directory.
depth:0 will omit all trees, and is useful if you only want to fetch
the commits of a repository, or fetch just a single tree or blob object.
depth:1 will fetch only the root tree of all commits fetched. depth:2
will fetch the root tree and all blobs and tree objects directly
referenced from it. depth:3 gets one more level, and so on. depth:#
will not filter a blob or tree that is explicitly marked wanted.
Bitmaps are disabled when this filter is used.
This implementation is quite slow because it iterates over all omitted
objects rather than skipping them. This will be addressed in follow-up
commits.
Change-Id: Ic312fee22d60e32cfcad59da56980e90ae2cae6a
Signed-off-by: Matthew DeVore <matvore@gmail.com>
Replace simple uses of Iterator with a corresponding for-loop.
Also add missing braces on loops as necessary.
Change-Id: I708d82acdf194787e3353699c07244c5ac3de189
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Test that an exception is raised for an invalid group header:
[group "foo" ]
foo = bar
i.e. where there is a space between the group subsection name
and the closing ']'.
Change-Id: I8933ae100b77634b0afb66bb8aa43d24c955799e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Add resolveTipSha1, an inverse of exactRef(String ...), to RefDatabase
and provide a default implementation that runs in O(n) time where n is
the number of refs. For RefTable, provide an implementation that runs
in O(log(n)) time.
[ifrade@google.com: with tests in InMemoryRepositoryTest to exercise
the reftable code path, too]
Change-Id: I2811ccd0339cdc1c74b42cce2ea003f07a2ce9e1
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Ivan Frade <ifrade@google.com>
Replace usage with the recommended setFilterSpec(FilterSpec).
Change-Id: Icc528d175f25234eeb2daa6b4c29a67a7a6d1e0a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The new references created in the peeling do not receive the update
index. In other words, the update index of a reference (if set) is lost
in the peeling.
Pass-through the update index to the newly created references.
Tested via InMemoryRepository, which uses DfsReftableDatabase.
Change-Id: I7ff7c737a9c3366fdec296a4d9b2e51d10227957
Signed-off-by: Ivan Frade <ifrade@google.com>
This increases type-safety and is ground work for support of the
"tree:<depth>" filter.
Change-Id: Id19eacdcdaddb9132064c642f6d554b1060efe9f
Signed-off-by: Matthew DeVore <matvore@gmail.com>
HashMap<String, Ref> has a memory overhead for refs. Use RefMap.
Change-Id: I3fb4616135dacf687cc3bc2b473effc66ccef5e6
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
The prune method did not delete empty fanout directories when loose
objects moved to a new pack file but only when loose unreferenced
objects were pruned.
Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Reduce contention on PackFile.idx() function.
Use SystemReader in JSchConfigSessionFactoryTest
Avoid NPE in ObjectId.isId()
Change-Id: I1d13f6fb705258ae6d6e5fa5e733bfacd4f3d0e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This isolates the test from the concrete system it's running on.
SshSessionFactory reads the user also through SystemReader.
Change-Id: I1c796aa1c498fe3967456d8589e6be0a82ab8f44
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.11:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.10:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.2:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
Change-Id: Ib972641105cef8089791fc65389c4f43e218620a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.1:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
Change-Id: I42b72a3bba3b4c9389d91a1a35a8004836567e7c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
With text=auto or core.autocrlf=true, git does not normalize upon
check-in if the file in the index contains already CR/LFs. The
documentation says: "When text is set to "auto", the path is
marked for automatic end-of-line conversion. If Git decides that
the content is text, its line endings are converted to LF on
checkin. When the file has been committed with CRLF, no conversion
is done."[1]
Implement the last bit as in canonical git: check the blob in the
index for CR/LFs. For very large files, we check only the first 8000
bytes, like RawText.isBinary() and AutoLFInputStream do.
In Auto(CR)LFInputStream, ensure that the buffer is filled as much as
possible for the isBinary() check.
Regarding these content checks, there are a number of inconsistencies:
* Canonical git considers files containing lone CRs as binary.
* RawText checks the first 8000 bytes.
* Auto(CR)LFInputStream checks the first 8096 (not 8192!) bytes.
None of these are changed with this commit. It appears that canonical
git will check the whole blob, not just the first 8k bytes. Also
note: the check for CR/LF text won't work with LFS (neither in JGit
nor in git) since the blob data is not run through the smudge filter.
C.f. [2].
Two tests in AddCommandTest actually tested that normalization was
done even if the file was already committed with CR/LF.These tests
had to be adapted. I find the git documentation unclear about the
case where core.autocrlf=input, but from [3] it looks as if this
non-normalization also applies in this case.
Add new tests in CommitCommandTest testing this for the case where
the index entry is for a merge conflict. In this case, canonical git
uses the "ours" version.[4] Do the same.
[1] https://git-scm.com/docs/gitattributes
[2] https://github.com/git/git/blob/3434569fc/convert.c#L225
[3] https://github.com/git/git/blob/3434569fc/convert.c#L529
[4] https://github.com/git/git/blob/f2b6aa98b/read-cache.c#L3281
Bug: 470643
Change-Id: Ie7310539fbe6c737d78b1dcc29e34735d4616b88
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Deprecate the method in favor of setEncoding(Charset).
Update the only caller in the code base that was still using
the deprecated variant.
Change-Id: I6357f2d0c727007013c72e9d5b7c72a3f5f3f2b1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Bug caused the pack to be 12 bytes short when cold cache. Also added
test for copyPackAsIs method.
Change-Id: Idf8fb0e50d1215245d4b032e2e00df4b218c115f
Signed-off-by: Minh Thai <mthai@google.com>
This method tried to iterate spurious files which may exist in the
.git/refs folder, e.g. on Mac a .DS_Store may have been created there by
inspecting the folder using the finder application. This led to a
NotDirectoryException when deleteEmptyRefsFolders tried to create an
iterator for such a file entry. Skip files contained in the refs folder
to ensure the method only tries to iterate contained folders but not
files.
Change-Id: I5f31e733072a35db1e93908a9c69a8891ae5c206
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Single-branch-clone should be able to clone a single tag. Enhance
CloneCommand to accept also full refs of tags in setBranchesToClone().
Make sure we also include fetch ref specs for the fetch command for
tags. This mimics the behavior of native git's single-branch clone:
git clone --branch <tag> --single-branch <URI>
Bug: 542611
Change-Id: I285cf043751d9b0ba71258ee8214c0e5d1191428
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Several problems:
* The command didn't specify whether it expected short or full names.
* For the new name, it expected a short name, but then got confused
if tags or both local and remote branches with the same name existed.
* For the old name, it accepted either a short or a full name, but
again got confused if a short name was given and a tag with the
same name existed.
With such an interface, one cannot use Repository.findRef() to
reliably find the branch to rename. Use exactRef() for the new
name as by the time the Ref is needed its full name is known.
For determining the old Ref from the name, do the resolution
explicitly: first try exactRef (assuming the old name is a full
name); if that doesn't find anything, try "refs/heads/<old>" and
"refs/remotes/<old>" explicitly. Throw an exception if the name
is ambiguous, or if exactRef returned something that is not a
branch (refs/tags/... or also refs/notes/...).
Document in the javadoc what kind of names are valid, and add tests.
A user can still shoot himself in the foot if he chooses exceptionally
stupid branch names. For instance, it is still possible to rename a
branch to "refs/heads/foo" (full name "refs/heads/refs/heads/foo"),
but it cannot be renamed further using the new short name if a branch
with the full name "refs/heads/foo" exists. Similar edge cases exist
for other dumb branch names, like a branch with the short name
"refs/tags/foo". Renaming using the full name is always possible.
Bug: 542446
Change-Id: I34ac91c80c0a00c79a384d16ce1e727c550d54e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The new API is intended for UIs to check if signing will be possible or
would fail
Bug: 543579
Change-Id: I6ce1fd4210e46d49dcdf420c99d08c93e022136c
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
BundleFetchConnection.readLine() must abort on EOF, otherwise
it gets stuck in an endless loop.
Bug: 543390
Change-Id: I4cb3428560277888af114b928950d620bb6564f9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Except the cases where the warning was suppressed in bf757cd because
they are explicitly testing functionality that is not possible in t-w-r.
Change-Id: I6c66eb31a2314028e09a9213dc695cd163f907e9
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The tests were written for Java 7 which did not have AutoCloseable
and the try-with-resource concept. When the project was updated to
build with Java 8, the warnings were suppressed.
Remove the suppressions and convert to use try-with-resource.
Change-Id: Ic805bd571c4a2e4376ce5e7c34ca7ac86cbf5104
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Since 8ed59c5 ("Make TestRepository AutoCloseable", Jan 11, 2019) the
TestRepository class is auto-closeable, but instantiations of it were
not converted to use try-with-resource.
Converting to try-with-resource results, in several cases, in the
repository being closed twice because LocalDiskRepositoryTestCase has
logic to close created repositories in the tearDown method. This results
in several tests emitting a warning to the console:
close() called when useCnt is already zero
Change the default behavior of the createRepository method to not use
the auto-close logic in LocalDiskRepositoryTestCase, so that thy will
instead be closed (only once) using the AutoCloseable implementation.
Deprecate the method that has the autoClose parameter.
Change-Id: I63d62c9913f9b61271667861dae144e551d358c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This also includes a change to generating the jgit CLI jar. Shading is
no longer possible because it breaks the signature of BouncyCastle.
Instead, the Spring Boot Loader Maven plug-in is now used to generate an
executable jar.
Bug: 382212
Change-Id: I35ee3d4b06d9d479475ab2e51b29bed49661bbdc
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This change introduces the concept of a GpgSigner which will sign
commits. The GpgSigner will be of a specific implementation (eg.,
Bouncycastle or OpenPgP executable). The actual implementation is not
part of this change.
Bug: 382212
Change-Id: Iea5da1e885c039e06bc8d679d46b124cbe504c8e
Also-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Version 0.20.0 of buildifier has a lint mode that can fix the
formatting issues automatically:
buildifier --lint=fix
Apply this to BUILD files.
Change-Id: I1120a0f15771d0bbea4daa34037be0b860385389
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.
Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.
This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.
Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Its implementation contains
} catch (IOException e) {
// Legacy API, assume error means "no"
return false;
}
Better to use ObjectDatabase#has, which throws IOException to report
errors.
Change-Id: I7de02f7ceb8f57b2a8ebdb16d2aa4376775ff933
Signed-off-by: Jonathan Nieder <jrn@google.com>
Using findRef instead of getRef makes it clearer that the caller wants
to search for the ref in the search path, instead of looking for a ref
that exactly matches the input.
This change introduces the new findRef method and deprecates getRef.
It updates Repository#findRef to use the new method, ensuring some
test coverage. Other callers will be updated in followup changes.
A nice side effect of introducing the new findRef method is that it is
final and based on firstExactRef, so implementers can focus on
implementing the latter efficiently and do not have to carefully write
custom path search code respecting SEARCH_PATH.
Change-Id: Id3bb944344a9743705fd1f20193ab679298fa51c
Signed-off-by: Jonathan Nieder <jrn@google.com>
Psuedorefs like FETCH_HEAD and MERGE_HEAD are supposed to be directly
under the .git directory, not in other locations in the SEARCH_PATH
like refs/ and refs/heads/. Use exactRef to access them.
Change-Id: Iab8ac47008822fa78fc0691e239e518c34d7a98e
Signed-off-by: Jonathan Nieder <jrn@google.com>
This is simpler to implement than getRef. Make it abstract so
implementers remember to override it.
Change-Id: I5f319be1fb1206d7a0142ea939dc4e1039f850ab
Signed-off-by: Jonathan Nieder <jrn@google.com>
Now the reference carries its updateIndex, so the cursor doesn't need
to expose it.
Change-Id: Icbfca46f92a13f3d8215ad10b2a166a6f40b0b0f
Signed-off-by: Ivan Frade <ifrade@google.com>
In DFS implementations the reference table can fall out of sync, but
it is not possible to check this situation in the current API.
Add a property to the Refs indicating the order of its updates. This
version is set only by RefDatabase implementations that support
versioning (e.g reftable based).
Caller is responsible to check if the reference db creates versioned
refs before accessing getUpdateIndex(). E.g:
Ref ref = refdb.exactRef(...);
if (refdb.hasVersioning()) {
ref.getUpdateIndex();
}
Change-Id: I0d5ec8e8df47c730301b2e12851a6bf3dac9d120
Signed-off-by: Ivan Frade <ifrade@google.com>
ProtocolV2Parser explains:
// TODO(ifrade): This validation should be done after the
// protocol parsing. It is not a protocol problem asking for an
// unexisting ref and we wouldn't need the ref database here.
Do so. This way all ref database accesses are in one place, in the
UploadPack class.
No user-visible change intended --- this is just to make the code
easier to manipulate.
Change-Id: I68e87dff7b9a63ccc169bd0836e8e8baaf5d1048
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When executing a test with LANG environment variable set to non UTF-8
encoding, it seems that JRE cannot handle Unicode file paths. This
happens when this test is executed in Bazel as it unsets LANG
(https://docs.bazel.build/versions/master/test-encyclopedia.html#initial-conditions).
Skip the test if the runtime cannot handle Unicode file paths.
Change-Id: I16bd3cd959dbaf2335b9c5202873e2f12ed0ba21
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
By using File#setLastModified, we can create a racy git situation
stably.
Tested with --runs_per_test=100
Bug: 526111
Change-Id: I60b3632d353e19f335668325aa603640be423f58
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Configure Maven build to capture test coverage using jacoco.
Add new org.eclipse.jgit.coverage Maven module to aggregate
jacoco test coverage results and generate test coverage HTML report at
org.eclipse.jgit.coverage/target/site/jacoco-aggregate/index.html
See https://www.eclemma.org/jacoco/trunk/doc/maven.html
Change-Id: Iaeec4033e448ebc16965c05ab54109c4155a307a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Include the net.i2p.crypto.eddsa bundle via a hard dependency.
Add tests for dealing with ed25519 host keys and user key files.
Manual tests: fetching from git.eclipse.org with an ed25519 user key,
and pushing this change itself using the same ed25519 key.
Note that sshd 2.0.0 does not yet support encrypted ed25519 private
keys.
Bug: 541272
Change-Id: I7072f4014d9eca755b4a2412e19c086235e5eae9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Set GIT_DIR and GIT_WORK_TREE when calling hooks.
Bug: 541622
Change-Id: I6153d8a6a934ec37a3a5e7319c2d0e516f539ab7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The new dependency to mockito for JDKHttpConnectionTest wasn't added for
bazel test run.
Change-Id: Ie967b7152340a0b3e05cced7eadfd40af78d9995
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
After cloning a repo with a submodule, non-recursively, JGit would
encounter in its TreeWalk in IndexDiff:
* first, a missing gitlink (in index & HEAD, not in working tree)
* second, the untracked folder (not in index and head, in working tree)
As a result, it would report the submodule as missing. Canonical git
reports a clean workspace.
The root cause of this is that the path of a gitlink "x" did
not compare equal to the path of a tree "x" in JGit.
Correct Paths.compare() to account for that. If two paths are otherwise
equal, then let gitlinks match both trees and files. Matching trees
solves the bug. Matching files is necessary to handle the case where
the gitlink directory was replaced by a file; see the new test case
IndexDiffSubmoduleTest.testSubmoduleReplacedByFile(). Comparisons of
unequal paths are left untouched, so the sort order is unchanged.
After the fix, another bug(?) in WorkingTreeIterator became apparent:
with core.dirNoGitLinks = true, it was no longer possible to overwrite
a gitlink in the index. This is now fixed in WorkingTreeIterator.
Add new test cases for the bug itself and for some related cases
(submodule directory deleted or replaced by a file) in
IndexDiffSubmoduleTest. Add a test for missing files in IndexDiffTest,
and adapt the PathsTest to test matching gitlinks.
Bug: 467631
Change-Id: I0549d10d46b1858e5eec3cc15095aa9f1d5f5280
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.
See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt
Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
According to RFC 2616 [1] header field names are case insensitive.
Header fields defined as a comma separated list can have multiple header
fields with the same field name. Add a method to HttpConnection which
retrieves all values with a given header field name with the field name
compared case insensitive.
[1] https://tools.ietf.org/html/rfc2616#section-4.2"
Change-Id: I7f601b21cda99e84f43f866c7c7cb4cb0e3cf5c3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This partially reverts commit a551b646: revert the changes in
RawParseUtils.lineMap(). Forcing all blobs containing a NUL byte
as a single line causes blame to produce useless results as soon
as it hits any version containing a NUL byte.
Doing binary detection at this level also has the problem that the
user cannot control it. Not by setting the text attribute nor in any
other way.
This came up in bug 541036, where a Java source inadvertently
contained NUL bytes in strings. Even fixing this by using escapes
"\000" will not fix JGit's blame for this file because the past
versions will still contain the NUL byte.
Native git can blame that file from bug 541036 fine.
Added new tests verifying that blaming a text file containing a NUL
byte produces sensible results.
Bug: 541036
Change-Id: I8991bec88e9827cc096868c6026ea1890b6d0d32
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for password and keyboard-interactive authentication.
Implement password authentication; the default provided by sshd
is non-interactive, which is not useful for JGit.
Make sure the CredentialsProvider gets reset on successive password
retrieval attempts. Otherwise it might always return the same non-
accepted password from a secure storage. (That one was discovered
by actually trying this via EGit; the JGit tests don't catch this.)
Change the default order of authentication mechanisms to prefer
password over keyboard-interactive. This is a mitigation for upstream
bug SSHD-866.[1]
Also include a fix for upstream bug SSHD-867.[2]
[1] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-866
[2] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-867
Bug: 520927
Change-Id: I423e548f06d3b51531016cf08938c8bd7acaa2a9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Create the bundle and move the SshTestGitServer there. Verified that
the Eclipse build still works and ran JSchSshTest and ApacheSshTest as
junit tests inside Eclipse.
Update maven build and features to account for that. Verified by
running full maven build including packaging.
Update bazel build files to account for that. Verified by a
clean-slate bazel build :all, followed by running the JSchSshTest
and the ApacheSshTest via bazel.
Change-Id: Ia084942f4425b454529de148e00417e7da786a90
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
sshd only asks exactly once for the password. C.f. upstream issue
SSHD-850.[1] So we have to work around this limitation for now.
Once we move to sshd > 2.1.0, this can be simplified somewhat.
[1] https://issues.apache.org/jira/browse/SSHD-850
Bug: 520927
Change-Id: Id65650228486c5ed30affa9c62eac982e01ae207
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
By default sshd will use its default built-in list, which matches
the one of openssh (as far as the algorithms exist in sshd at all).
But it doesn't handle HostKeyAlgorithms from the ssh config at all.
Implement this as in openssh, including the '+' and '-' modifiers
and reordering the default if there are known host keys for a
server already.
Add tests for the reordering.
Also use a more robust reader for the known hosts file. The default
aborts on the first error.
Bug: 520927
Change-Id: Ib1684440bfe2e96140536aa1a93c4bd4a0d35916
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Tests that it works with unknown authentications in the list, and
fails if there are no common authentications between server and
client. The latter also tests that the ssh config setting is taken
into account at all.
And promptly the JGit sshd client didn't. Add a fix for this. It's
a tiny bit hacky: Apache MINA looks up a custom property set on a
hierarchy of "PropertyResolver"s starting with the session. On the
session itself this property can never be set since it's read
already in the session constructor before anyone had any chance
to set it. The next element in the resolver hierarchy is the sshd
SshClient, and so we set that property there. Since we use one
SshClient and one ClientSession per JGit SshdSession, this is OK.
Bug: 520927
Change-Id: I62446fc1fffde125a8965c030240f0918ae234b7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new ssh client implementation based on Apach MINA sshd 2.0.0.
This implementation uses JGit's own config file parser and host entry
resolver. Code inspection of the Apache MINA implementation revealed
a few bugs or idiosyncrasies that immediately would re-introduce bugs
already fixed in the past in JGit.
Apache MINA sshd is not without quirks either, and I had to configure
and override more than I had expected. But at least it was all doable
in clean ways.
Apache MINA boasts support for Bouncy Castle, so in theory this should
open the way to using more ssh key algorithms, such as ed25519.
The implementation is in a separate bundle and is still not used in
the core org.eclipse.jgit bundle. The tests re-use the ssh tests from
the core test bundle.
Bug: 520927
Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add support for git-receive-pack to the ssh git server and add two
new tests for pushing.
This actually uncovered an undocumented requirement in TransportSftp:
the FTP rename operation assumes POSIX semantics, i.e., that the
target is removed. This works as written only for servers that
support and advertise the "posix-rename@openssh.com" FTP extension.
Our little Apache MINA server does not advertise this extension.
Fix the FtpChannel implementation for Jsch to handle this case in a
meaningful way so that it can pass the new "push over sftp" test.
Add more tests to test the behavior of server host key checking.
Also refactor the tests generally to separate better the test
framework from the actual tests.
Bug: 520927
Change-Id: Ia4bb85e17ddacde7b36ee8c2d5d454bbfa66dfc3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Correct behaviour as git 1.7.1.1 is to resolve tie-breakers to choose
the most recent tag.
https://github.com/git/git/blob/master/Documentation/RelNotes/1.7.1.1.txt:
* "git describe" did not tie-break tags that point at the same commit
correctly; newer ones are preferred by paying attention to the
tagger date now.
Bug: 538610
Change-Id: Ib0b2a301997bb7f75935baf7005473f4de952a64
Signed-off-by: Håvard Wall <haavardw@gmail.com>
Suppose that a repository has the following commit graph:
B C
\ /
A
and it was cloned with --shallow-exclude=A. DepthGenerator does not mark
C as shallow, causing an invalid repository to be produced on the
client, because A is not sent. (A similar issue occurs when
--shallow-since is used to exclude A but neither B nor C.)
This happens whenever an excluded commit has more than one child that is
to be sent to the client. Fix DepthGenerator to handle this case
correctly.
While we're editing DepthWalk.Commit, fix the documentation of
DepthWalk.Commit#isBoundary.
Change-Id: I7068abf0fe0c864d1b0e56e1616dad1aa8719411
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Move the bulk of the basic parsing and host entry handling into a
new class OpenSshConfigFile that has no dependencies on any concrete
ssh implementation. Make the existing OpenSshConfig use the new
parser.
Introduce a new class SshConstants collecting all the various ssh-
related string literals. Also use TreeMaps with a case-insensitive
key comparator instead of converting keys to uppercase. Add a test
to verify that keys are matched case-insensitively.
Most of the parsing code was simply moved, except that the new
parser supports looking up entries given host name, port, and user
name, and can thus handle more %-substitutions correctly. This
feature is not yet used and cannot be used with JSch since JSch
only has a ConfigRepository.getConfig(String) interface.
The split is still worth the trouble as it opens the way to using
another ssh client altogether. Apache MINA sshd, for instance,
resolves host entries giving host name, port, and user name.
(Apache MINA has a built-in ssh config handling, but that has
problems, too: its pattern matching is case-insensitive, and its
merging of host entries if several match is not the same as in
OpenSsh. But with this refactoring, it will be possible to plug in
OpenSshConfigFile into an Apache MINA sshd client without dragging
along JSch.)
One test case that doesn't make sense anymore has been removed. It
tested that repeatedly querying for a host entry returned the same
object. That is no longer true since the caching has been moved to
a deeper level.
Bug: 520927
Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a simple ssh git server based on Apache MINA sshd, and use it
in new tests that verify ssh operations and in particular a number
of bugs that had cropped up over time in JSch.
The git server supports fetching only, and sftp access.
The tests are all in an abstract base class; the concrete JschSshTest
class only provides ssh-specific test setup. So the same tests could
be run easily also with some other ssh client.
Bug: 520927
Change-Id: Ide6687b717fb497a29fc83f22b07390a26dfce1d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When a source folder is marked as a test folder, JDT requires that it
has an output folder different from the one used for regular sources.
Therefore give the test folders in org.eclipse.jgit.test a separate
output folder "bin-tst".
Moreover JDT reports errors if non-test classes have dependencies on
test classes. Therefore remove the "test" annotation from
org.eclipse.jgit.junit.
Change-Id: Ib527439ff5b7d7b570b8a60819ecaa70f59c63a3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The copyfile entry in the manifest file copies the contents of the file
but doesn't keep the executable flag. This is inconsistent with repo
tool behaviour, plus is natural to expect that the copy of a executable
file is executable.
Transfer the executable bit when copying the file, aligning the
RepoCommand with repo tool and user expectations.
Change-Id: I01b24f482d5939e01d496f032388b3a5c02a912a
Signed-off-by: Ivan Frade <ifrade@google.com>
The RepoCommand.RemoteReader interface doesn't offer access to the mode
of a file. Caller can only default to mark the copied objects as regular
files, losing e.g. the executable bit (if set).
Add a new method readFileWithMode that returns the contents and mode of
the remote file. It supersedes the readFile method, that is marked as
deprecated.
Now callers can set correctly the file mode of the copied file.
Change-Id: I8fce01e4bc5707434c0cbc4aebbae1b6b64756f0
Signed-off-by: Ivan Frade <ifrade@google.com>
Many tests verify the contents of files in a try-with-resources
incantation that clutters the code.
Extract that verification to an "assertContents" method, that is easier
to read.
Change-Id: If430eac6f5b9ae352e42b2d43867ceb6cd618fbb
Signed-off-by: Ivan Frade <ifrade@google.com>
Patterns should treat \r in file names as normal characters
Change-Id: Ica3e0fa4a58acf5326db46bb28571fe5f20f6cd2
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
ProtocolV2Parser has unit tests but protocol v0/v1 is not covered.
Change-Id: I96022e8f8eb60d4da748d1042474fd1efd67e882
Signed-off-by: Ivan Frade <ifrade@google.com>
Parsed requests represent object ids (SHA1) in ObjectId instances but tests
use strings for those ids because they are easier to define.
Create a custom matcher that hides the conversion from string to
ObjectId. Note that this reverses the existing code conversion (it was
transforming ObjectIds into string).
This produces more readable code, consistent with the other hamcrest
assertions.
Change-Id: I47ba1d25557d791fe74fb93c740ff7de9923cc00
Signed-off-by: Ivan Frade <ifrade@google.com>
This allows clients to use the --shallow-exclude parameter (producing a
"deepen-not <ref>" line when communicating with the server) in their fetch
commands when fetching against a JGit server using protocol v2.
Note that the implementation in this commit is somewhat inefficient, as
described in the TODO comment in DepthGenerator.
Change-Id: I9fad3ed9276b624d8f668356ffd99a067dc67ef7
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
In C Git, when a client fetches with "git fetch --shallow-since=<date>
origin <ref>", and all commits reachable from <ref> are older than
<date>, the server dies with a message "no commits selected for shallow
requests". That is, (1) the --shallow-since filter applies to the commit
pointed to by the ref itself, and (2) there is a check that at least one
commit is not filtered out. (The pack-protocol.txt documentation does
not describe this, but the C implementation does this.)
The implementation in commit 1bb430dc21 ("UploadPack: support
deepen-since in protocol v2", 2018-09-27) does neither (1) nor (2), so
do both of these.
Change-Id: I9946327a71627626ecce34ca2d017d2add8867fc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
The request receives a list of capabilities and takes out the "agent" to
offer it on its own setter (getAgent).
Do this at parse time: when reading the line if the capability is
"agent" set it directly in the builder.
This makes the treatment of "agent" consistent in v0/v1 and v2.
Change-Id: Ie4f9f2cad8639adeeaef4921df49a30a8ce5b42f
Signed-off-by: Ivan Frade <ifrade@google.com>
All of the input lines passed to pre-push hook scripts must be properly
terminated by '\n', so that normal shell scripts like the git-supplied
pre-push.sample work properly, even when pushing just a single branch.
With the old code, hook scripts that use the following pattern didn't
process the last line, because 'read' has a non-zero exit status when
EOF is encountered:
while read local_ref local_sha remote_ref remote_sha; do ... done
Change-Id: Id899662ed3fedef6c314fc4b2ddf91a6dcb98cbb
Signed-off-by: Markus Keller <markus.kell.r@gmail.com>
Before this commit, a force checkout would fail if there
were any conflicting files. After this commit, a force
checkout will overwrite the conflicting files, as expected.
Making this work required fixing a bug in DirCacheCheckout.
Before this commit, when DirCacheCheckout had
failOnConflict=false, it would delete all conflicting files
from the working copy and just leave them missing. After
this commit, DirCacheCheckout overwrites conflicting files
with the merge tree.
This change in DirCacheCheckout causes "reset --hard" and
"revert --abort" to behave as expected (previously they
would simply delete conflicting files, now they will be
overwritten from the merge tree).
Change-Id: If7e328ee792ef6511ab7d9c26d8d77c39210ec9f
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
UploadPack.getPeerUserAgent() doesn't produce the expected results for
protocol v2 requests. In v2, the agent reported in the request (in an
"agent=" line) is not in the clientCapabilities but in a field on its
own. This makes getPeerUserAgent default to the transport user agent.
Making "agent" a shared property between protocol v0/v1 and v2 fixes the
problem, simplifies the function and harmonizes the implementation
between protocol versions.
In a follow up commit the "agent" will be identified on parsing time,
instead of taking it from the client capabilities.
Change-Id: Idf9825ec4e0b81a1458c8e3701f3e28aafd8a32a
Signed-off-by: Ivan Frade <ifrade@google.com>
In protocol v2, a command request can be followed by server options
(lines like "agent=<>" and "server-option=<>"), but current code
doesn't accept those lines.
Advertise the "server-option" capability, parse the lines and add
them to the request objects.
Other code in JGit can see this options and act accordingly via the
protocol v2 hooks.
This should not require any change in the client side.
Change-Id: If3946390f9cc02d29644b6ca52534b6f757bda9f
Signed-off-by: Ivan Frade <ifrade@google.com>
Eclipse Photon supports separating sources and test sources.
There are no functional changes in the IDE, except for test source
folders having a different icon color.
Users of different IDEs than Eclipse are not affected, since the
attribute on the classpath entries will be ignored by their IDE.
Bug: 539933
Change-Id: Iac6dcdf0c0730ca775bae90df6a685303dc95380
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The code base has several @SuppressWarnings annotations to suppress
warnings raised by Error Prone, but those are not recognized by
Eclipse and there is currently no way to tell it about them [1].
Suppress them for now.
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=392045
Change-Id: I3de7cfa8ad4370ca5be71e1303879c73ab6829c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports that the unsynchronized method skip overrides the
synchronized method in ByteArrayInputStream [1].
This is a test class, so we can just suppress the warning as recommended
in [1].
Note that the suppression causes a warning in Eclipse, because it doesn't
recognize the "UnsynchronizedOverridesSynchronized" as a valid value for
the @SuppressWarnings annotation [2].
[1] https://errorprone.info/bugpattern/UnsynchronizedOverridesSynchronized
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=392045
Change-Id: I3e798b448211f1363729091f72fb0ef6a873e599
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
In FetchV0Request, the fields "wantsIds" and "options" are called
"wantIds" and "clientCapabilities". Those names describe them better.
Rename FetchV2Request fields to follow fetch v0. This will make easier
to extract a superclass later.
Take also the chance to polish the javadoc.
Change-Id: Ia17dbbab8084f39cc529fef9ca5c65e189073767
Signed-off-by: Ivan Frade <ifrade@google.com>
First-want line parsing accepts lines with an optional whitespace, when
the spec is strict requiring a white space.
Validate the line enforcing that there is a white space between oid and
capabilities list.
Change-Id: I45ada67030e0720f9b402c298be18c7518c799b1
Signed-off-by: Ivan Frade <ifrade@google.com>
In protocol v0/v1 pack negotiation, the first want line contains the
options the client wants in effect. This parsing is done in UploadPack
but it doesn't have any interaction with that class.
Move the code to its own class and package, mark the current one
as deprecated (it is public API) and add unit tests.
Take the chance to move the parsing code from the constructor to a
factory method, making the class a simple container of results.
Change-Id: I1757f535dda78a4111a1c12c3a3b455a4b6f0c51
Signed-off-by: Ivan Frade <ifrade@google.com>
Previous commits block the addition to the repo of dangerous .gitmodules
files, but some could have been committed before those safeguards where
in place.
Add a check in DfsFsck to validate the .gitmodules files in the repo.
Use the same validation than the ReceivePack, translating the
results to FsckErrors.
Note that *all* .gitmodules files in the storage will be checked, not
only the latest version.
Change-Id: I040cf1f31a779419aad0292ba5e6e76eb7f32b66
Signed-off-by: Ivan Frade <ifrade@google.com>
The fsck test needs more detail about the error than an IOException
with an explanatory message.
Add an error identifier to the SubmoduleValidatorException and make
it the only throwable exception when parsing a file.
Change-Id: Ic3f0955b497e1681b25e681e1282e876cdf3d2c5
Signed-off-by: Ivan Frade <ifrade@google.com>
The main concern are submodule urls starting with '-' that could pass as
options to an unguarded tool.
Pass through the parser the ids of blobs identified as .gitmodules
files in the ObjectChecker. Load the blobs and parse/validate them
in SubmoduleValidator.
Change-Id: Ia0cc32ce020d288f995bf7bc68041fda36be1963
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In order to validate .gitmodules files, we first need to find them
in the incoming pack.
Do it in the ObjectChecker stage. Check in the tree objects if they
point to a .gitmodules file and report the tree id and the .gitmodules
blob id.
This can be used later to check if the file is in the root of the
project and if the contents are good.
While we're here, make isMacHFSGit more accurate by detecting variants
of filenames that vary in case.
[jn: tweaked NTFS and HFS+ checking; added more tests]
Change-Id: I70802e7d2c1374116149de4f89836b9498f39582
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In C git versions before 2.19.1, the submodule is fetched by running
"git clone <uri> <path>". A URI starting with "-" would be interpreted
as an option, causing security problems. See CVE-2018-17456.
Refuse to add submodules with URIs, names or paths starting with "-",
that could be confused with command line arguments.
[jn: backported to JGit 4.7.y, bringing portions of Masaya Suzuki's
dotdot check code in v5.1.0.201808281540-m3~57 (Add API to specify
the submodule name, 2018-07-12) along for the ride]
Change-Id: I2607c3acc480b75ab2b13386fe2cac435839f017
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
EolStreamTypeUtil didn't handle these correctly on Windows.
Add three new tests to verify that the crlf attribute is handled as
described at [1], and that core.eol=native produces the expected
line endings on check-out.
[1] https://git-scm.com/docs/gitattributes
Bug: 497290
Change-Id: Idd9b435e3256c1e3251cc7b966f2f0460e787f07
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This probably doesn't provide any benefit for the tests, but gets
rid of a warning from Error Prone.
See https://errorprone.info/bugpattern/InputStreamSlowMultibyteRead
Change-Id: I584d2e0d18475fad38747b688af6301c423f54a7
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Deprecate constructors and methods taking a character set name as
a String, in favor of new variants taking a Charset.
Change-Id: I616c601daf232fa17610dba1087fd902030d46ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I58ad45a87dcf7d646dce056d20776d62faafbfef
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I518b524de7cd3802f03b80450cad02ab3f79d57b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: Ic89f9daef3c31bfbfd1f3c003cb90a373cd74847
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I463510342bb6e6b99b31a0fe264d953340784393
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The tests were set up to expect an IllegalArgumentException when
the Config.getInt method was called with a section.key that has
not been set, or explicitly set to an empty string.
However, the IllegalArgumentException never gets thrown because
the getInt method returns the provided default ("1"), and because
there was no call to "fail" after getInt, the incorrect behavior
of the test was not noticed.
Remove the try/catch around getInt, and instead assert that the
expected default value is returned.
Found by Error Prone, which reported:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: Ie8e692aba9fb8523241fb8f298d57493923d9f78
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Support the deepen-since parameter when requested by a client using
protocol v2. This is done by:
- adding a DepthWalk.RevWalk#setDeepenSince method
- updating DepthGenerator to recognize when deepen-since is set
- recording in DepthWalk.Commit whether a commit is a boundary commit
Existing users of DepthWalk such as UploadPack previously recognized
boundary commits by comparing their depths against the threshold, not
tracking whether any parents were truly excluded. This behavior is
preserved - UploadPack considers a commit as boundary if its depth is
equal to the threshold *or* a parent was excluded (whether by depth or
by deepen-since).
Change-Id: I852bba6b1279f9cc8aee38282e9339d62b8dcddc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
The BufferedWriter is opened in a try-with-resource and thus will be
automatically closed.
Presumably the close was added to make sure it is closed before the
subsequent test statements are executed. Instead of explicitly closing
it, let the try-with-resource automatically close it, and move the
subsequent statements out of the try-block.
Change-Id: If5fada2f580ef9cbaad3a0b9216b5200b917781a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace explicit construction of FileWriter with calls to the
utility method Files.newBufferedWriter, which allows to specify
the charset.
Also convert to try-with-resource.
Change-Id: I9fa3f612b9b2fc5ac12cd79d6e61ca181120dbf5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace explicit construction of BufferedReader with calls to the
utility method Files.newBufferedReader, which allows to specify
the charset.
Change-Id: I61b9451dbc8d9cf83fc8a5981292b8fdc713ce37
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
There should only be one statement after the expect(...) method.
Any additional statements after the statement that is expected to
throw will never be executed in a passing test. This can lead to
inappropriately passing tests where later incorrect assertions are
skipped by the thrown exception.
See https://errorprone.info/bugpattern/ExpectedExceptionChecker
Change-Id: I20fecf8fb7a243e9da097e6d03fbf8cd69151bf0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
There should only be one statement after the expect(...) method.
Any additional statements after the statement that is expected to
throw will never be executed in a passing test. This can lead to
inappropriately passing tests where later incorrect assertions are
skipped by the thrown exception.
See https://errorprone.info/bugpattern/ExpectedExceptionChecker
Change-Id: I0d6350fafb281b6bdb04289f4cd5eb4bb159628b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn