When executing a test with LANG environment variable set to non UTF-8
encoding, it seems that JRE cannot handle Unicode file paths. This
happens when this test is executed in Bazel as it unsets LANG
(https://docs.bazel.build/versions/master/test-encyclopedia.html#initial-conditions).
Skip the test if the runtime cannot handle Unicode file paths.
Change-Id: I16bd3cd959dbaf2335b9c5202873e2f12ed0ba21
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
By using File#setLastModified, we can create a racy git situation
stably.
Tested with --runs_per_test=100
Bug: 526111
Change-Id: I60b3632d353e19f335668325aa603640be423f58
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Configure Maven build to capture test coverage using jacoco.
Add new org.eclipse.jgit.coverage Maven module to aggregate
jacoco test coverage results and generate test coverage HTML report at
org.eclipse.jgit.coverage/target/site/jacoco-aggregate/index.html
See https://www.eclemma.org/jacoco/trunk/doc/maven.html
Change-Id: Iaeec4033e448ebc16965c05ab54109c4155a307a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Include the net.i2p.crypto.eddsa bundle via a hard dependency.
Add tests for dealing with ed25519 host keys and user key files.
Manual tests: fetching from git.eclipse.org with an ed25519 user key,
and pushing this change itself using the same ed25519 key.
Note that sshd 2.0.0 does not yet support encrypted ed25519 private
keys.
Bug: 541272
Change-Id: I7072f4014d9eca755b4a2412e19c086235e5eae9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Set GIT_DIR and GIT_WORK_TREE when calling hooks.
Bug: 541622
Change-Id: I6153d8a6a934ec37a3a5e7319c2d0e516f539ab7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The new dependency to mockito for JDKHttpConnectionTest wasn't added for
bazel test run.
Change-Id: Ie967b7152340a0b3e05cced7eadfd40af78d9995
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
After cloning a repo with a submodule, non-recursively, JGit would
encounter in its TreeWalk in IndexDiff:
* first, a missing gitlink (in index & HEAD, not in working tree)
* second, the untracked folder (not in index and head, in working tree)
As a result, it would report the submodule as missing. Canonical git
reports a clean workspace.
The root cause of this is that the path of a gitlink "x" did
not compare equal to the path of a tree "x" in JGit.
Correct Paths.compare() to account for that. If two paths are otherwise
equal, then let gitlinks match both trees and files. Matching trees
solves the bug. Matching files is necessary to handle the case where
the gitlink directory was replaced by a file; see the new test case
IndexDiffSubmoduleTest.testSubmoduleReplacedByFile(). Comparisons of
unequal paths are left untouched, so the sort order is unchanged.
After the fix, another bug(?) in WorkingTreeIterator became apparent:
with core.dirNoGitLinks = true, it was no longer possible to overwrite
a gitlink in the index. This is now fixed in WorkingTreeIterator.
Add new test cases for the bug itself and for some related cases
(submodule directory deleted or replaced by a file) in
IndexDiffSubmoduleTest. Add a test for missing files in IndexDiffTest,
and adapt the PathsTest to test matching gitlinks.
Bug: 467631
Change-Id: I0549d10d46b1858e5eec3cc15095aa9f1d5f5280
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
when multiple match options are given in git describe the result must
not depend on the order of the match options. JGit wrongly picked the
first match using the match options in the order they were defined. Fix
this by concatenating the streams of matching tags for all match options
and then choosing the first match on the concatenated stream sorted in
tie break order.
See https://git-scm.com/docs/git-describe#git-describe---matchltpatterngt
Change-Id: Id01433d35fa16fb4c30526605bee041ac1d954b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
According to RFC 2616 [1] header field names are case insensitive.
Header fields defined as a comma separated list can have multiple header
fields with the same field name. Add a method to HttpConnection which
retrieves all values with a given header field name with the field name
compared case insensitive.
[1] https://tools.ietf.org/html/rfc2616#section-4.2"
Change-Id: I7f601b21cda99e84f43f866c7c7cb4cb0e3cf5c3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This partially reverts commit a551b646: revert the changes in
RawParseUtils.lineMap(). Forcing all blobs containing a NUL byte
as a single line causes blame to produce useless results as soon
as it hits any version containing a NUL byte.
Doing binary detection at this level also has the problem that the
user cannot control it. Not by setting the text attribute nor in any
other way.
This came up in bug 541036, where a Java source inadvertently
contained NUL bytes in strings. Even fixing this by using escapes
"\000" will not fix JGit's blame for this file because the past
versions will still contain the NUL byte.
Native git can blame that file from bug 541036 fine.
Added new tests verifying that blaming a text file containing a NUL
byte produces sensible results.
Bug: 541036
Change-Id: I8991bec88e9827cc096868c6026ea1890b6d0d32
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add tests for password and keyboard-interactive authentication.
Implement password authentication; the default provided by sshd
is non-interactive, which is not useful for JGit.
Make sure the CredentialsProvider gets reset on successive password
retrieval attempts. Otherwise it might always return the same non-
accepted password from a secure storage. (That one was discovered
by actually trying this via EGit; the JGit tests don't catch this.)
Change the default order of authentication mechanisms to prefer
password over keyboard-interactive. This is a mitigation for upstream
bug SSHD-866.[1]
Also include a fix for upstream bug SSHD-867.[2]
[1] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-866
[2] https://issues.apache.org/jira/projects/SSHD/issues/SSHD-867
Bug: 520927
Change-Id: I423e548f06d3b51531016cf08938c8bd7acaa2a9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Create the bundle and move the SshTestGitServer there. Verified that
the Eclipse build still works and ran JSchSshTest and ApacheSshTest as
junit tests inside Eclipse.
Update maven build and features to account for that. Verified by
running full maven build including packaging.
Update bazel build files to account for that. Verified by a
clean-slate bazel build :all, followed by running the JSchSshTest
and the ApacheSshTest via bazel.
Change-Id: Ia084942f4425b454529de148e00417e7da786a90
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
sshd only asks exactly once for the password. C.f. upstream issue
SSHD-850.[1] So we have to work around this limitation for now.
Once we move to sshd > 2.1.0, this can be simplified somewhat.
[1] https://issues.apache.org/jira/browse/SSHD-850
Bug: 520927
Change-Id: Id65650228486c5ed30affa9c62eac982e01ae207
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
By default sshd will use its default built-in list, which matches
the one of openssh (as far as the algorithms exist in sshd at all).
But it doesn't handle HostKeyAlgorithms from the ssh config at all.
Implement this as in openssh, including the '+' and '-' modifiers
and reordering the default if there are known host keys for a
server already.
Add tests for the reordering.
Also use a more robust reader for the known hosts file. The default
aborts on the first error.
Bug: 520927
Change-Id: Ib1684440bfe2e96140536aa1a93c4bd4a0d35916
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Tests that it works with unknown authentications in the list, and
fails if there are no common authentications between server and
client. The latter also tests that the ssh config setting is taken
into account at all.
And promptly the JGit sshd client didn't. Add a fix for this. It's
a tiny bit hacky: Apache MINA looks up a custom property set on a
hierarchy of "PropertyResolver"s starting with the session. On the
session itself this property can never be set since it's read
already in the session constructor before anyone had any chance
to set it. The next element in the resolver hierarchy is the sshd
SshClient, and so we set that property there. Since we use one
SshClient and one ClientSession per JGit SshdSession, this is OK.
Bug: 520927
Change-Id: I62446fc1fffde125a8965c030240f0918ae234b7
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a new ssh client implementation based on Apach MINA sshd 2.0.0.
This implementation uses JGit's own config file parser and host entry
resolver. Code inspection of the Apache MINA implementation revealed
a few bugs or idiosyncrasies that immediately would re-introduce bugs
already fixed in the past in JGit.
Apache MINA sshd is not without quirks either, and I had to configure
and override more than I had expected. But at least it was all doable
in clean ways.
Apache MINA boasts support for Bouncy Castle, so in theory this should
open the way to using more ssh key algorithms, such as ed25519.
The implementation is in a separate bundle and is still not used in
the core org.eclipse.jgit bundle. The tests re-use the ssh tests from
the core test bundle.
Bug: 520927
Change-Id: Ib35e73c35799140fe050d1ff4fb18d0d3596580e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add support for git-receive-pack to the ssh git server and add two
new tests for pushing.
This actually uncovered an undocumented requirement in TransportSftp:
the FTP rename operation assumes POSIX semantics, i.e., that the
target is removed. This works as written only for servers that
support and advertise the "posix-rename@openssh.com" FTP extension.
Our little Apache MINA server does not advertise this extension.
Fix the FtpChannel implementation for Jsch to handle this case in a
meaningful way so that it can pass the new "push over sftp" test.
Add more tests to test the behavior of server host key checking.
Also refactor the tests generally to separate better the test
framework from the actual tests.
Bug: 520927
Change-Id: Ia4bb85e17ddacde7b36ee8c2d5d454bbfa66dfc3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Correct behaviour as git 1.7.1.1 is to resolve tie-breakers to choose
the most recent tag.
https://github.com/git/git/blob/master/Documentation/RelNotes/1.7.1.1.txt:
* "git describe" did not tie-break tags that point at the same commit
correctly; newer ones are preferred by paying attention to the
tagger date now.
Bug: 538610
Change-Id: Ib0b2a301997bb7f75935baf7005473f4de952a64
Signed-off-by: Håvard Wall <haavardw@gmail.com>
Suppose that a repository has the following commit graph:
B C
\ /
A
and it was cloned with --shallow-exclude=A. DepthGenerator does not mark
C as shallow, causing an invalid repository to be produced on the
client, because A is not sent. (A similar issue occurs when
--shallow-since is used to exclude A but neither B nor C.)
This happens whenever an excluded commit has more than one child that is
to be sent to the client. Fix DepthGenerator to handle this case
correctly.
While we're editing DepthWalk.Commit, fix the documentation of
DepthWalk.Commit#isBoundary.
Change-Id: I7068abf0fe0c864d1b0e56e1616dad1aa8719411
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Move the bulk of the basic parsing and host entry handling into a
new class OpenSshConfigFile that has no dependencies on any concrete
ssh implementation. Make the existing OpenSshConfig use the new
parser.
Introduce a new class SshConstants collecting all the various ssh-
related string literals. Also use TreeMaps with a case-insensitive
key comparator instead of converting keys to uppercase. Add a test
to verify that keys are matched case-insensitively.
Most of the parsing code was simply moved, except that the new
parser supports looking up entries given host name, port, and user
name, and can thus handle more %-substitutions correctly. This
feature is not yet used and cannot be used with JSch since JSch
only has a ConfigRepository.getConfig(String) interface.
The split is still worth the trouble as it opens the way to using
another ssh client altogether. Apache MINA sshd, for instance,
resolves host entries giving host name, port, and user name.
(Apache MINA has a built-in ssh config handling, but that has
problems, too: its pattern matching is case-insensitive, and its
merging of host entries if several match is not the same as in
OpenSsh. But with this refactoring, it will be possible to plug in
OpenSshConfigFile into an Apache MINA sshd client without dragging
along JSch.)
One test case that doesn't make sense anymore has been removed. It
tested that repeatedly querying for a host entry returned the same
object. That is no longer true since the caching has been moved to
a deeper level.
Bug: 520927
Change-Id: I6381d52b29099595e6eaf8b05c786aeeaefbf9cc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add a simple ssh git server based on Apache MINA sshd, and use it
in new tests that verify ssh operations and in particular a number
of bugs that had cropped up over time in JSch.
The git server supports fetching only, and sftp access.
The tests are all in an abstract base class; the concrete JschSshTest
class only provides ssh-specific test setup. So the same tests could
be run easily also with some other ssh client.
Bug: 520927
Change-Id: Ide6687b717fb497a29fc83f22b07390a26dfce1d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When a source folder is marked as a test folder, JDT requires that it
has an output folder different from the one used for regular sources.
Therefore give the test folders in org.eclipse.jgit.test a separate
output folder "bin-tst".
Moreover JDT reports errors if non-test classes have dependencies on
test classes. Therefore remove the "test" annotation from
org.eclipse.jgit.junit.
Change-Id: Ib527439ff5b7d7b570b8a60819ecaa70f59c63a3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The copyfile entry in the manifest file copies the contents of the file
but doesn't keep the executable flag. This is inconsistent with repo
tool behaviour, plus is natural to expect that the copy of a executable
file is executable.
Transfer the executable bit when copying the file, aligning the
RepoCommand with repo tool and user expectations.
Change-Id: I01b24f482d5939e01d496f032388b3a5c02a912a
Signed-off-by: Ivan Frade <ifrade@google.com>
The RepoCommand.RemoteReader interface doesn't offer access to the mode
of a file. Caller can only default to mark the copied objects as regular
files, losing e.g. the executable bit (if set).
Add a new method readFileWithMode that returns the contents and mode of
the remote file. It supersedes the readFile method, that is marked as
deprecated.
Now callers can set correctly the file mode of the copied file.
Change-Id: I8fce01e4bc5707434c0cbc4aebbae1b6b64756f0
Signed-off-by: Ivan Frade <ifrade@google.com>
Many tests verify the contents of files in a try-with-resources
incantation that clutters the code.
Extract that verification to an "assertContents" method, that is easier
to read.
Change-Id: If430eac6f5b9ae352e42b2d43867ceb6cd618fbb
Signed-off-by: Ivan Frade <ifrade@google.com>
Patterns should treat \r in file names as normal characters
Change-Id: Ica3e0fa4a58acf5326db46bb28571fe5f20f6cd2
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
ProtocolV2Parser has unit tests but protocol v0/v1 is not covered.
Change-Id: I96022e8f8eb60d4da748d1042474fd1efd67e882
Signed-off-by: Ivan Frade <ifrade@google.com>
Parsed requests represent object ids (SHA1) in ObjectId instances but tests
use strings for those ids because they are easier to define.
Create a custom matcher that hides the conversion from string to
ObjectId. Note that this reverses the existing code conversion (it was
transforming ObjectIds into string).
This produces more readable code, consistent with the other hamcrest
assertions.
Change-Id: I47ba1d25557d791fe74fb93c740ff7de9923cc00
Signed-off-by: Ivan Frade <ifrade@google.com>
This allows clients to use the --shallow-exclude parameter (producing a
"deepen-not <ref>" line when communicating with the server) in their fetch
commands when fetching against a JGit server using protocol v2.
Note that the implementation in this commit is somewhat inefficient, as
described in the TODO comment in DepthGenerator.
Change-Id: I9fad3ed9276b624d8f668356ffd99a067dc67ef7
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
In C Git, when a client fetches with "git fetch --shallow-since=<date>
origin <ref>", and all commits reachable from <ref> are older than
<date>, the server dies with a message "no commits selected for shallow
requests". That is, (1) the --shallow-since filter applies to the commit
pointed to by the ref itself, and (2) there is a check that at least one
commit is not filtered out. (The pack-protocol.txt documentation does
not describe this, but the C implementation does this.)
The implementation in commit 1bb430dc21 ("UploadPack: support
deepen-since in protocol v2", 2018-09-27) does neither (1) nor (2), so
do both of these.
Change-Id: I9946327a71627626ecce34ca2d017d2add8867fc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
The request receives a list of capabilities and takes out the "agent" to
offer it on its own setter (getAgent).
Do this at parse time: when reading the line if the capability is
"agent" set it directly in the builder.
This makes the treatment of "agent" consistent in v0/v1 and v2.
Change-Id: Ie4f9f2cad8639adeeaef4921df49a30a8ce5b42f
Signed-off-by: Ivan Frade <ifrade@google.com>
All of the input lines passed to pre-push hook scripts must be properly
terminated by '\n', so that normal shell scripts like the git-supplied
pre-push.sample work properly, even when pushing just a single branch.
With the old code, hook scripts that use the following pattern didn't
process the last line, because 'read' has a non-zero exit status when
EOF is encountered:
while read local_ref local_sha remote_ref remote_sha; do ... done
Change-Id: Id899662ed3fedef6c314fc4b2ddf91a6dcb98cbb
Signed-off-by: Markus Keller <markus.kell.r@gmail.com>
Before this commit, a force checkout would fail if there
were any conflicting files. After this commit, a force
checkout will overwrite the conflicting files, as expected.
Making this work required fixing a bug in DirCacheCheckout.
Before this commit, when DirCacheCheckout had
failOnConflict=false, it would delete all conflicting files
from the working copy and just leave them missing. After
this commit, DirCacheCheckout overwrites conflicting files
with the merge tree.
This change in DirCacheCheckout causes "reset --hard" and
"revert --abort" to behave as expected (previously they
would simply delete conflicting files, now they will be
overwritten from the merge tree).
Change-Id: If7e328ee792ef6511ab7d9c26d8d77c39210ec9f
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
UploadPack.getPeerUserAgent() doesn't produce the expected results for
protocol v2 requests. In v2, the agent reported in the request (in an
"agent=" line) is not in the clientCapabilities but in a field on its
own. This makes getPeerUserAgent default to the transport user agent.
Making "agent" a shared property between protocol v0/v1 and v2 fixes the
problem, simplifies the function and harmonizes the implementation
between protocol versions.
In a follow up commit the "agent" will be identified on parsing time,
instead of taking it from the client capabilities.
Change-Id: Idf9825ec4e0b81a1458c8e3701f3e28aafd8a32a
Signed-off-by: Ivan Frade <ifrade@google.com>
In protocol v2, a command request can be followed by server options
(lines like "agent=<>" and "server-option=<>"), but current code
doesn't accept those lines.
Advertise the "server-option" capability, parse the lines and add
them to the request objects.
Other code in JGit can see this options and act accordingly via the
protocol v2 hooks.
This should not require any change in the client side.
Change-Id: If3946390f9cc02d29644b6ca52534b6f757bda9f
Signed-off-by: Ivan Frade <ifrade@google.com>
Eclipse Photon supports separating sources and test sources.
There are no functional changes in the IDE, except for test source
folders having a different icon color.
Users of different IDEs than Eclipse are not affected, since the
attribute on the classpath entries will be ignored by their IDE.
Bug: 539933
Change-Id: Iac6dcdf0c0730ca775bae90df6a685303dc95380
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The code base has several @SuppressWarnings annotations to suppress
warnings raised by Error Prone, but those are not recognized by
Eclipse and there is currently no way to tell it about them [1].
Suppress them for now.
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=392045
Change-Id: I3de7cfa8ad4370ca5be71e1303879c73ab6829c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports that the unsynchronized method skip overrides the
synchronized method in ByteArrayInputStream [1].
This is a test class, so we can just suppress the warning as recommended
in [1].
Note that the suppression causes a warning in Eclipse, because it doesn't
recognize the "UnsynchronizedOverridesSynchronized" as a valid value for
the @SuppressWarnings annotation [2].
[1] https://errorprone.info/bugpattern/UnsynchronizedOverridesSynchronized
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=392045
Change-Id: I3e798b448211f1363729091f72fb0ef6a873e599
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
In FetchV0Request, the fields "wantsIds" and "options" are called
"wantIds" and "clientCapabilities". Those names describe them better.
Rename FetchV2Request fields to follow fetch v0. This will make easier
to extract a superclass later.
Take also the chance to polish the javadoc.
Change-Id: Ia17dbbab8084f39cc529fef9ca5c65e189073767
Signed-off-by: Ivan Frade <ifrade@google.com>
First-want line parsing accepts lines with an optional whitespace, when
the spec is strict requiring a white space.
Validate the line enforcing that there is a white space between oid and
capabilities list.
Change-Id: I45ada67030e0720f9b402c298be18c7518c799b1
Signed-off-by: Ivan Frade <ifrade@google.com>
In protocol v0/v1 pack negotiation, the first want line contains the
options the client wants in effect. This parsing is done in UploadPack
but it doesn't have any interaction with that class.
Move the code to its own class and package, mark the current one
as deprecated (it is public API) and add unit tests.
Take the chance to move the parsing code from the constructor to a
factory method, making the class a simple container of results.
Change-Id: I1757f535dda78a4111a1c12c3a3b455a4b6f0c51
Signed-off-by: Ivan Frade <ifrade@google.com>
Previous commits block the addition to the repo of dangerous .gitmodules
files, but some could have been committed before those safeguards where
in place.
Add a check in DfsFsck to validate the .gitmodules files in the repo.
Use the same validation than the ReceivePack, translating the
results to FsckErrors.
Note that *all* .gitmodules files in the storage will be checked, not
only the latest version.
Change-Id: I040cf1f31a779419aad0292ba5e6e76eb7f32b66
Signed-off-by: Ivan Frade <ifrade@google.com>
The fsck test needs more detail about the error than an IOException
with an explanatory message.
Add an error identifier to the SubmoduleValidatorException and make
it the only throwable exception when parsing a file.
Change-Id: Ic3f0955b497e1681b25e681e1282e876cdf3d2c5
Signed-off-by: Ivan Frade <ifrade@google.com>
The main concern are submodule urls starting with '-' that could pass as
options to an unguarded tool.
Pass through the parser the ids of blobs identified as .gitmodules
files in the ObjectChecker. Load the blobs and parse/validate them
in SubmoduleValidator.
Change-Id: Ia0cc32ce020d288f995bf7bc68041fda36be1963
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In order to validate .gitmodules files, we first need to find them
in the incoming pack.
Do it in the ObjectChecker stage. Check in the tree objects if they
point to a .gitmodules file and report the tree id and the .gitmodules
blob id.
This can be used later to check if the file is in the root of the
project and if the contents are good.
While we're here, make isMacHFSGit more accurate by detecting variants
of filenames that vary in case.
[jn: tweaked NTFS and HFS+ checking; added more tests]
Change-Id: I70802e7d2c1374116149de4f89836b9498f39582
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In C git versions before 2.19.1, the submodule is fetched by running
"git clone <uri> <path>". A URI starting with "-" would be interpreted
as an option, causing security problems. See CVE-2018-17456.
Refuse to add submodules with URIs, names or paths starting with "-",
that could be confused with command line arguments.
[jn: backported to JGit 4.7.y, bringing portions of Masaya Suzuki's
dotdot check code in v5.1.0.201808281540-m3~57 (Add API to specify
the submodule name, 2018-07-12) along for the ride]
Change-Id: I2607c3acc480b75ab2b13386fe2cac435839f017
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
EolStreamTypeUtil didn't handle these correctly on Windows.
Add three new tests to verify that the crlf attribute is handled as
described at [1], and that core.eol=native produces the expected
line endings on check-out.
[1] https://git-scm.com/docs/gitattributes
Bug: 497290
Change-Id: Idd9b435e3256c1e3251cc7b966f2f0460e787f07
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This probably doesn't provide any benefit for the tests, but gets
rid of a warning from Error Prone.
See https://errorprone.info/bugpattern/InputStreamSlowMultibyteRead
Change-Id: I584d2e0d18475fad38747b688af6301c423f54a7
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Deprecate constructors and methods taking a character set name as
a String, in favor of new variants taking a Charset.
Change-Id: I616c601daf232fa17610dba1087fd902030d46ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I58ad45a87dcf7d646dce056d20776d62faafbfef
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I518b524de7cd3802f03b80450cad02ab3f79d57b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: Ic89f9daef3c31bfbfd1f3c003cb90a373cd74847
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: I463510342bb6e6b99b31a0fe264d953340784393
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The tests were set up to expect an IllegalArgumentException when
the Config.getInt method was called with a section.key that has
not been set, or explicitly set to an empty string.
However, the IllegalArgumentException never gets thrown because
the getInt method returns the provided default ("1"), and because
there was no call to "fail" after getInt, the incorrect behavior
of the test was not noticed.
Remove the try/catch around getInt, and instead assert that the
expected default value is returned.
Found by Error Prone, which reported:
Not calling fail() when expecting an exception masks bugs
See https://errorprone.info/bugpattern/MissingFail
Change-Id: Ie8e692aba9fb8523241fb8f298d57493923d9f78
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Support the deepen-since parameter when requested by a client using
protocol v2. This is done by:
- adding a DepthWalk.RevWalk#setDeepenSince method
- updating DepthGenerator to recognize when deepen-since is set
- recording in DepthWalk.Commit whether a commit is a boundary commit
Existing users of DepthWalk such as UploadPack previously recognized
boundary commits by comparing their depths against the threshold, not
tracking whether any parents were truly excluded. This behavior is
preserved - UploadPack considers a commit as boundary if its depth is
equal to the threshold *or* a parent was excluded (whether by depth or
by deepen-since).
Change-Id: I852bba6b1279f9cc8aee38282e9339d62b8dcddc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
The BufferedWriter is opened in a try-with-resource and thus will be
automatically closed.
Presumably the close was added to make sure it is closed before the
subsequent test statements are executed. Instead of explicitly closing
it, let the try-with-resource automatically close it, and move the
subsequent statements out of the try-block.
Change-Id: If5fada2f580ef9cbaad3a0b9216b5200b917781a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace explicit construction of FileWriter with calls to the
utility method Files.newBufferedWriter, which allows to specify
the charset.
Also convert to try-with-resource.
Change-Id: I9fa3f612b9b2fc5ac12cd79d6e61ca181120dbf5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace explicit construction of BufferedReader with calls to the
utility method Files.newBufferedReader, which allows to specify
the charset.
Change-Id: I61b9451dbc8d9cf83fc8a5981292b8fdc713ce37
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
There should only be one statement after the expect(...) method.
Any additional statements after the statement that is expected to
throw will never be executed in a passing test. This can lead to
inappropriately passing tests where later incorrect assertions are
skipped by the thrown exception.
See https://errorprone.info/bugpattern/ExpectedExceptionChecker
Change-Id: I20fecf8fb7a243e9da097e6d03fbf8cd69151bf0
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
There should only be one statement after the expect(...) method.
Any additional statements after the statement that is expected to
throw will never be executed in a passing test. This can lead to
inappropriately passing tests where later incorrect assertions are
skipped by the thrown exception.
See https://errorprone.info/bugpattern/ExpectedExceptionChecker
Change-Id: I0d6350fafb281b6bdb04289f4cd5eb4bb159628b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Support multiple prefixes when querying references to allow
implementor to minimize number of RPC calls.
Change-Id: I5f822fd7eaf9756b44750080d3056de138b64f4a
Signed-off-by: Minh Thai <mthai@google.com>
The commit message template contains a superfluous blank at the end of
the first line, which is deleted by this change. This is only relevant
for jgit contributors using Eclipse, not for jgit users.
Change-Id: I462deb49c26fb64b3dc2d1d75f1e40ef302b0fc9
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Otherwise fetching all tags may pull in commits not on the
specified branches. Canonical git also does this.[1]
[1] https://github.com/git/git/blob/b160b6e69/builtin/clone.c#L1124
Bug: 538768
Change-Id: If0ac75fb9fae0c95d1a48b22954c54d4c3c09a47
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
When not all branches are cloned, the fetch refspec for the
remote should not be "+refs/heads/*:refs/remotes/origin/*":
that would fetch all branches on the very next fetch, thus
making a clone with only a subset of the branches rather
pointless.
Instead, produce refspecs for the cloned branches only.
Canonical git also does this for its --single-branch case;
it doesn't have an option to clone only a subset of the branches
(only one or all).
Bug: 466858
Change-Id: Ie871880f757663437efac1e8b3313094f9e629b3
Also-by: Julian Enoch <julian.enoch@ericsson.com>
Signed-off-by: Julian Enoch <julian.enoch@ericsson.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Fetch code has been moved to a ProtocolV2Parser, but ls-refs code is
still in UploadPack.
Moving it to the parser makes it easier to test, keeps the parsing
together and makes the two commands follow similar structure.
Change-Id: I573ce543e804ddeb9f83303b4af250b7cddc8cad
Signed-off-by: Ivan Frade <ifrade@google.com>
Also check that reflog is enabled by default.
Also check that a file in a folder is reset correctly.
Change-Id: I4bc6649928ce10d7b558901e465e8032b083ade0
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
The statement:
assertThat(recvStream.available(), is(0));
results in a warning from Eclipse:
The expression of type int is boxed into Integer
because recvStream.available() returns int, but the hamcrest is()
method takes an Integer.
Replace it with the equivalent JUnit assertion.
Also remove the suppression of another similar warning and fix that
in the same way.
Change-Id: I6f18b304a540bcd0a10aec7d3abc7dc6f047fe80
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
In order to support GPG-signed commits, add some methods which will
allow GPG signatures to be parsed out of RevCommit objects.
Later, we can add code to verify the signatures.
Change-Id: Ifcf6b3ac79115c15d3ec4b4eaed07315534d09ac
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use existing test utility methods instead of nested PrintWriter usage.
Change-Id: I324852c7971ae644fa499f377a31d1cf265c7fd9
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
ErrorProne warns [1] about implicit use of the platform default charset,
which can result in differing behaviour between JVM executions or
incorrect behavior if the encoding of the data source doesn't match
expectations.
[1] http://errorprone.info/bugpattern/DefaultCharset
Change-Id: I0fd489d352170339c3867355cd24324dfdbd4b59
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Client will stop sending haves when the number of haves sent reaches maxhaves.
Change-Id: I1e5b1525be4c67f20a81ca24a2770c20eb5c1271
Signed-off-by: Minh Thai <mthai@google.com>
The parsing code for protocol v2 fetch doesn't have any dependency on
the rest of UploadPack.
Move it to its own class. This makes testing easier (no need to
instantiate the full UploadPack), simplifies the code in UploadPack and
increases modularity.
At the moment, the parser needs to know about the reference database to
validate incoming references. This dependency could be easily removed
moving the validation later in the flow, after the parsing, where other
validations are already happening. Postponing that to keep this patch
about moving unmodified code around.
Change-Id: I7ad29a6b99caa7c12c06f5a7f30ab6a5f6e44dc7
Signed-off-by: Ivan Frade <ifrade@google.com>
* stable-5.1:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: Ie9d77a7716591246b87fb59ac85214417fe8309d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.0:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I7c2fc7d694d7d891cac96cb805c89c8bdf7c5cbf
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.11:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I5ebb1058927ea5a478eae9e44b7d96f123a22523
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.10:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I500dd4e63302abb19ad669988b3a427cef8ef3d4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I37e936de0c06d4b9f17724ac4f1feb83f6c15ae3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.8:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I17436237cd66ca1c2800ad5ab0142f4a2bc07328
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.7:
Bazel: Use hyphen instead of underscore in external repository names
Bazel: Format all build files with buildifier 0.15.0
ChangeIdUtilTest: Remove unused notestCommitDashV
Change-Id: I414ade902dc38b696c566dd604000e3d289f3973
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
At the moment there are two copies of the client shallow commit list:
one in the request and another in the clientShallowCommits member of
the class.
The verifyShallowCommit function was removing missing object ids
from the member but not the request list, and code afterwards was
using the request's version.
In practice, this didn't cause trouble because these shallow commits
are used as endpoint for a walk, and missing ids are just never reached.
Change-Id: I70a8f1fd46de135da09f16e5d954693c8438ffcb
Signed-off-by: Ivan Frade <ifrade@google.com>
This test was never being run. Since it was introduced it was
named "notest.." which meant it didn't run with JUnit3, and
since it is not annotated @Test it also doesn't run with JUnit4.
When compiling with Bazel 0.6.0, error-prone raises an error
that the public method is not annotated with @Ignore or @Test.
Given that the test has never been run anyway, we can just
remove it.
Bug: 525415
Change-Id: Ie9a54f89fe42e0c201f547ff54ff1d419ce37864
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.0:
Silence API warning for new ObjectIdSerializer introduced in 4.11.1
Ignore API warnings
Fix photon target platform to use photon version of org.eclipse.osgi
Update Photon orbit repository to R20180606145124
Suppress warning for trying to delete non-empty directory
Fix fetching with duplicate ref updates
Fetch(Process): should tolerate duplicate refspecs
FetchCommandTest: test add/update/delete fetch
Change-Id: I9e7fa37b100a7ea3cbe4104802d36c6f38df9e08
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.11:
Silence API warning for new ObjectIdSerializer introduced in 4.11.1
Ignore API warnings
Fix photon target platform to use photon version of org.eclipse.osgi
Update Photon orbit repository to R20180606145124
Suppress warning for trying to delete non-empty directory
Fix fetching with duplicate ref updates
Fetch(Process): should tolerate duplicate refspecs
FetchCommandTest: test add/update/delete fetch
Change-Id: I2497f99b7d0686a0e0f773c39c8725e813ce9d4a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.10:
Ignore API warnings
Fix photon target platform to use photon version of org.eclipse.osgi
Update Photon orbit repository to R20180606145124
Suppress warning for trying to delete non-empty directory
Fix fetching with duplicate ref updates
Fetch(Process): should tolerate duplicate refspecs
FetchCommandTest: test add/update/delete fetch
Change-Id: I47671108574fcd10fe016a58e38c038af54a6748
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Renamed and restructured tests for improved clarity.
Bug: 479266
Change-Id: Ic9d05ddf722bddd148fa9d9c19248dd53d97f1e4
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
* stable-4.9:
Ignore API warnings
Fix photon target platform to use photon version of org.eclipse.osgi
Update Photon orbit repository to R20180606145124
Suppress warning for trying to delete non-empty directory
Fix fetching with duplicate ref updates
Fetch(Process): should tolerate duplicate refspecs
FetchCommandTest: test add/update/delete fetch
Change-Id: I2a83c059b7014c2a9e6267c963422c7785b23f17
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
On recent VMs, collection.toArray(new T[0]) is faster than
collection.toArray(new T[collection.size()]). Since it is also more
readable, it should now be the preferred way of collection to array
conversion.
https://shipilev.net/blog/2016/arrays-wisdom-ancients/
Change-Id: I80388532fb4b2b0663ee1fe8baa94f5df55c8442
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
If packed refs are used, duplicate updates result in an exception
because JGit tries to lock the same lock file twice. With non-atomic
ref updates, this used to work, since the same ref would simply be
locked and updated twice in succession.
Let's be more lenient in this case and remove duplicates before
trying to do the ref updates. Silently skip duplicate updates
for the same ref, if they both would update the ref to the same
object ID. (If they don't, behavior is undefined anyway, and we
still throw an exception.)
Add a test that results in a duplicate ref update for a tag.
Bug: 529400
Change-Id: Ide97f20b219646ac24c22e28de0c194a29cb62a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Bug: 529314
Change-Id: I91eaeda8a988d4786908fba6de00478cfc47a2a2
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Change-Id: I2442394fb7eae5b3715779555477dd27b274ee83
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The TransportLocal object created in
newTransportLocalWithStrictValidation() closes original repository and
increments use of test internal "dst" repository, but never decrements
use. Because of this, pack file is not closed and during tearDown on
Windows system is unable to delete it.
Bug: 538068
Change-Id: I96df8e91abfee78c91cf26c2466718e9145a69db
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
In Git protocol v2, UploadPack and ReceivePack have the same
capabilities and can process any protocol v2 request. For example, a
client can sent a "fetch" command to the "/git-receive-pack" endpoint.
This makes it difficult for existing hook interfaces. For example,
PreUploadHook takes UploadPack, but a "fetch" command may be received by
ReceivePack.
To resolve this skew, this change introduce a different hook interface
for the protocol v2. The hook takes a request that is independent to the
handlers (UploadPack, ReceivePack). Also this makes it clear what
parameters the hook is counting on, instead of keep track of the hook
using getters from UploadPack / ReceivePack.
Bug: 534847
Change-Id: I71f3266584483db1e2b2edfc1a72d0bdf1bb6041
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
When fetching with protocol v2, git expects the shallow-info section to
appear before wanted-refs if both appear in the response. Teach
UploadPack to do this.
Change-Id: Ie26a91edcce5d27a1d727d7fba5c30e1144e118b
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
ResolveMerger.checkout() and cleanUp() check out files directly and
must honor CR/LF settings and also smudge filters.
Deprecate the 3-argument version of DirCacheCheckout.checkoutEntry().
It isn't used anymore anywhere in JGit (nor in EGit).
Bug: 537410
Change-Id: I062b35401c8bd5bc99deb2f68f91089a0643504c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
UploadPack already allows the client to send wanted OIDs as "want"
lines. Extend UploadPack to also allow the client to send wanted ref
names as "want-ref" lines when the fetch is done using protocol v2.
The corresponding Git commit is 516e2b76bd ("upload-pack: implement
ref-in-want", 2018-06-28).
To support a two-stage rollout, two configuration variables are
provided: uploadpack.allowrefinwant (default "false") allows clients to
specify "want-ref" in their requests, and uploadpack.advertiserefinwant
(default "true") makes UploadPack advertise this capability. If
uploadpack.allowrefinwant is true but uploadpack.advertiserefinwant is
false, UploadPack will not advertise that it supports "want-ref", but it
will support it.
Change-Id: I3c24077949640d453af90d81a7f48ce4b8ac9833
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Two submodules at the same path on different branches need not represent
the same repository, and two submodules at different paths can represent
the same one.
The C Git implementation uses the submodule name to internally manage
the submodule repositories under .git/modules. When a submodule
represents different repositories in different branches, it makes a
conflict inside .git/modules.
The current RepoCommand implementation uses submodule paths as the
submodule names. When the manifest file mounts different repositories to
the same path in different branches, this makes a situation described
above. To solve this issue, we can use the project name instead of
the path as the submodule name.
On the other hand, since repo v1.12.8~3^2 (repo: Support multiple
branches for the same project., 2013-10-11), a manifest file can mount
the same project to different paths. If we naively use the project
name as the submodule name, it makes a conflict in .git/modules, too.
This patch uses the project name as the submodule name basically, but
when the same project is mounted to different paths, it uses the project
name and path as the submodule name.
Change-Id: I09dc7d62ba59016fe28852d3139a56ef7ef49b8f
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Reported-by: JP Sugarbroad <jpsugar@google.com>
* stable-5.0:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: I9717444c42ea89195dfcde8bb10121b9bb178e87
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.11:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: I780d4a1624fbb97a2acdf5321b9c3244b0626444
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.10:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: I2b44d862869d4453c57db668fc7c925da591f671
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: Ie7029bc91621af32e7bfd2e0d76a424b991b1995
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.8:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: I6ab2b951dd94a9fc1c4f5283847a3e2ec37d0895
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.7:
Prepare 4.7.3-SNAPSHOT builds
JGit v4.7.2.201807261330-r
Delete all loose refs empty directories
Use java.nio to delete path to get detailed errors
GC: Remove empty references folders
Do not ignore path deletion errors
Change-Id: Iadc8275fbaa3d6f7d08a96ab66d49f392f6aab78
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Remove completely the empty directories under refs/<namespace>
including the first level partition of the changes, when they are
completely empty.
Bug: 536777
Change-Id: I88304d34cc42435919c2d1480258684d993dfdca
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When processing a fetch using protocol v2, UploadPack#fetchV2 sends an
extraneous flush pkt when also sending a packfile (#sendPack sending its
own flush pkt). Update that method to only send the flush pkt if the
packfile is not being sent.
Change-Id: I7117a264bccd2d7f3a048645fcb8425a9d78d526
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
In a0c9016abd ("upload-pack: send refs' objects despite "filter"",
2018-07-09), Git updated the "filter" option in the fetch-pack
upload-pack protocol to not filter objects explicitly specified in
"want" lines, even if they match the criterion of the filter. Update
JGit to match that behavior.
Change-Id: Ia4d74326edb89e61062e397e05483298c50f9232
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Don't try to delete folders if the merger chooses THEIRS, but all of
BASE, OURS, and THEIRS contain the folder.
Add a test for rebase with auto-stash and subdirectories that
verifies this case. The needless directory deletion and reporting
such directories in getModifiedFiles() was the root cause of bug
536880.
Note even with this fix, bug 536880 will not be fixed in all cases
yet. There may still be cases where the set of modified files ends
up containing directories. This will be dealt with in EGit where
this set is used. (See https://git.eclipse.org/r/#/c/126242/ .)
Bug: 536880
Change-Id: I62b4571a1c1d4415934a6cb4270e0c8036deb2e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
RacyGitTests depend on filesystem timer resolution. We wait for a file
system timer tick, remember that time as t1, modify a file and assume
that this file has a lastmodified of t1.
If this assumption is not fulfilled then ignore the test result.
Bug: 526111
Change-Id: Ia38b7d2f99171ef54b8f9fe5be343cf9fcfd3971
Currently SubmoduleAddCommand always uses the path as submodule name.
This patch lets the caller specify a submodule name.
SubmoduleUpdateCommand still does not make use of the submodule name
(see bug 535027) but Git does. To avoid triggering CVE-2018-11235,
do some validation on the name to avoid '..' path components.
[jn: fleshed out commit message, mostly to work around flaky CI]
Change-Id: I6879c043c6d7973556e2080387f23c246e3d76a5
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Make the method names more consistent and their semantics simpler:
hasRef and seekRef to look up a single exact reference by name and
hasRefsByPrefix and seekRefsByPrefix to look up multiple references by
name prefix.
In particular, splitting hasRef into two separate methods for its
different uses makes DfsReftableDatabase.isNameConflicting easier to
follow.
[jn: fleshed out commit message]
Change-Id: I71106068ff3ec4f7e14dd9eb6ee6b5fab8d14d0b
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Reftable implementation of RefDatabase.getRefsByPrefix() should be
more performant, as references are filtered directly by prefix;
instead of fetching the whole subtree then filter by prefix.
Change-Id: If4f5f8c08285ea1eaec9efb83c3d864cea7a1321
Signed-off-by: Minh Thai <mthai@google.com>
After packaging references, the folders containing these references are
not deleted. In a busy repository, this causes operations to slow down
as traversing the references tree becomes longer.
Delete empty reference folders after the loose references have been
packed.
To avoid deleting a folder that was just created by another concurrent
operation, only delete folders that were not modified in the last 30
seconds.
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ie79447d6121271cf5e25171be377ea396c7028e0
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
It is convenient for TestRepository to return fully parsed
objects from its commit()/tree()/blob() methods, so that test
code doesn't have to remember to parse them before making
assertions about them.
Update TestRepostiory to return fully parsed objects.
Adjust the tests that are affected by this change in behavior.
Change-Id: I09d03d0c80ad22cb7092f4a2eaed99d40a10af63
Signed-off-by: Terry Parker <tparker@google.com>
In a new RevWalk, if the first object parsed is one of the
shallow commits, the following happens:
1) RevCommit.parseCanonical() is called on a new "r1" RevCommit.
2) RevCommit.parseCanonical() immediately calls
RevWalk.initializeShallowCommits().
3) RevWalk.initializeShallowCommits() calls lookupCommit(id),
creating and adding a new "r2" version of this same object and
marking its parents empty.
4) RevCommit.parseCanonical() initializes the "r1" RevCommit's
fields, including the parents.
5) RevCommit.parseCanonical()'s caller uses the "r1" commit that
has parents, losing the fact that it is a shallow commit.
This change passes the current RevCommit as an argument to
RevWalk.initializeShallowCommits() so that method can set its
parents empty rather than creating the duplicate "r2" commit.
Change-Id: I67b79aa2927dd71ac7b0d8f8917f423dcaf08c8a
Signed-off-by: Terry Parker <tparker@google.com>
Test that the commit selection algorithm provides appropriate
coverage across all branches.
Change-Id: I82c71b52068f01e8cef2398aecfec8e144d9a68b
Signed-off-by: Terry Parker <tparker@google.com>
The same errors are suppressed when generating javadoc.
Suppress the errors during site generation.
Change-Id: I83bd1c10e5de82f47c351e7edf2c9230be4b1f21
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.0:
Prepare 5.0.1-SNAPSHOT builds
JGit v5.0.0.201806131550-r
JGit v5.0.0.201806131210-r
Downgrade Apache httpclient to 4.5.2.v20170210-0925
RefUpdateTest: Refactor to not use deprecated Repository#getAllRefs
Propagate failure of ssh command to caller of SshSupport
Make JGit describe behaves same as c-git for lightweight tags
Fix issues with LFS on GitHub (SSH)
Change-Id: I0471440919adfdbfc72996711d9e0bbd1f3cf477
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit now considers lightweight tags only if the --tags option is set
i.e. `git.describe().setAllTags(true)` has to be set, else the default
is now as in c git:
Only annotated tags are evaluated unless you pass true
equivalent to --tags (or --all) by the option setAllTags.
Hint: This (still) doesn't address any difference between c-git
`--all` and `!--all --tags` behavior;
perhaps this might be a follow up request
Bug: 423206
Change-Id: I9a3699756df0b9c6a7c74a7e8887dea0df17c8e7
Signed-off-by: Marcel Trautwein <me+eclipse@childno.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.0:
Use constant for ".lock"
Simplify locking of FileRepository's index snapshot
Refactor FileRepository.detectIndexChange()
Change-Id: Ifd427711359bcf38b2c877b2143d45bff0c9895a
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.0:
Teach UploadPack "filter" in protocol v2 fetch
Refactor test of capabilities output
Refactor v2 advertisement into own function
Refactor parsing of "filter" into its own method
Disallow unknown args to "fetch" in protocol v2
Teach UploadPack shallow fetch in protocol v2
Refactor unshallowCommits to local variable
Add protocol v2 support in http
Give info/refs services more control over response
Change-Id: I1683902222e076e1091795e94790a264550afb7b
Signed-off-by: Jonathan Nieder <jrn@google.com>
If the configuration variable uploadpack.allowfilter is true, advertise
that "filter" is supported, and support it if the client sends such an
argument.
Change-Id: I7de66c0a0ada46ff71c5ba124d4ffa7c47254c3b
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
A subsequent patch will dynamically generate the capability
advertisement, so the capability advertisements produced are not always
the same. Separate the checking of the advertisements into its own test
method.
Change-Id: I768d14b9d1a244d5d886c42ffd62ef3957b518fb
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
JGit's implementation of the fetch command of protocol v2, unlike its
implementation of ls-refs, currently tolerates unknown arguments.
Tighten fetch to not allow unrecognized arguments and add tests to
verify this behavior for both ls-refs and fetch.
Change-Id: I321161d568bd638252fab1a47b06b924d472a669
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add support for the "shallow" and "deepen" parameters in the "fetch"
command in the fetch-pack/upload-pack protocol v2. Advertise support for
this in the capability advertisement.
TODO: implement deepen-relative, deepen-since, deepen-not
Change-Id: I7ffd80d6c38872f9d713ac7d6e0412106b3766d7
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Provide a factory for comparators that use the default heuristics except
with a different ordering of PackSources.
Change-Id: I0809b64deb3d0486040076946fdbdad650d69240
There are several ways of comparing DfsPackDescriptions for different
purposes, such as object lookup search order and reftable ordering. Some
of these are later compounded into comparators on other objects, so they
appear in the code as Comparator<DfsReftable>, for example.
Put all the DfsPackDescription comparators in static methods on
DfsPackDescription itself. Stop implementing Comparable, to avoid giving
the impression that there is always one true and correct way of sorting
packs.
Change-Id: Ia5ca65249c13373f7ef5b8a5d1ad50a26577706c
Rather than requiring callers to do their own computations based on the
package-private "category" number, provide an actual
Comparator<PackSource> instance, and explicitly discourage usage of
default Enum comparison.
Construct the default comparator using a builder pattern based on
defining equivalence classes. This gives us the same behavior as the old
category field in PackSource, with an abstraction that does not leak the
implementation detail of comparing rank numbers.
Change-Id: I6757211397ab1bc181d61298e073f88b69dbefc3
Tests that even a commit with a huge message can be committed and read
if the WindowCache's streamFileThreshold is configured large enough.
Bug: 535092
Change-Id: Id8090c608625010caf11dff7971b47882b5fd20f
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The canonical implementation also doesn't. Compare current
code in remote.c, function get_stale_heads_cb.[1] Not handling
symrefs in this case was introduced in canonical git in [2]
in 2008.
[1] https://github.com/git/git/blob/v2.17.0/remote.c#L2259
[2] https://github.com/git/git/commit/740fdd27f0
Bug: 533549
Change-Id: If348d56bb4a96b8aa7141f7e7b5a0d3dd4e7808b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Callers of getAllRefs that only iterate over the `values()` of the
returned map can be trivially fixed to call getRefDatabase().getRefs()
instead.
Only fix those where the calling method is already declared to throw
IOException, to avoid potential API changes.
Change-Id: I2b05f785077a1713953cfd42df7bf915f889f90b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Callers should use getRefDatabase().peel(ref) instead since it
doesn't swallow the IOException.
Adapt all trivial callers to user the alternative.
DescribeCommand still uses the deprecated method and is not adapted in
this change since it will require more refactoring to add handling of
the IOException.
Change-Id: I14d4a95a5e0570548753b9fc5c03d024dc3ff832
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Make FileTreeIterator not enter ignored directories by default. We
only need to enter ignored directories if we do some operation against
git, and there is at least one tracked file underneath an ignored
directory.
Walking ignored directories should be avoided as much as possible as
it is a potential performance bottleneck. Some projects have a lot of
files or very deep hierarchies in ignored directories; walking those
may be costly (especially so on Windows). See for instance also bug
500106.
Provide a FileTreeIterator.setWalkIgnoredDirectories() operation to
force the iterator to iterate also through otherwise ignored
directories. Useful for tests (IgnoreNodeTest, CGitIgnoreTest), or
to implement things like "git ls-files --ignored".
Add tests in DirCacheCheckoutTest, and amend IndexDiffTest to test a
little bit more.
Bug: 388582
Change-Id: I6ff584a42c55a07120a4369fd308409431bdb94a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Callers should use getRefDatabase().getRefsByPrefix(R_TAGS) instead.
Adjust the tests accordingly.
Bug: 534731
Change-Id: Ib28ae365e42720268996ff46e34cae1745ad545c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Remove it from
* package private functions.
* try blocks
* for loops
this was done with the following python script:
$ cat f.py
import sys
import re
import os
def replaceFinal(m):
return m.group(1) + "(" + m.group(2).replace('final ', '') + ")"
methodDecl = re.compile(r"^([\t ]*[a-zA-Z_ ]+)\(([^)]*)\)")
def subst(fn):
input = open(fn)
os.rename(fn, fn + "~")
dest = open(fn, 'w')
for l in input:
l = methodDecl.sub(replaceFinal, l)
dest.write(l)
dest.close()
for root, dirs, files in os.walk(".", topdown=False):
for f in files:
if not f.endswith('.java'):
continue
full = os.path.join(root, f)
print full
subst(full)
Change-Id: If533a75a417594fc893e7c669d2c1f0f6caeb7ca
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Add support for the "ofs-delta" parameter in the "fetch" command in
the fetch-pack/upload-pack protocol v2.
Change-Id: I728cf986082fce4ddeb6a6435897692e15e60cc7
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add support for the "include-tag" parameter in the "fetch" command in
the fetch-pack/upload-pack protocol v2.
In order to determine which tags to include, only objects pointed to by
refs starting with "refs/tags/" are checked. This restriction is for
performance reasons and to match the behavior of Git (see add_ref_tag()
in builtin/pack-objects.c).
Change-Id: I7d70aa09bcc8a525218ff1559e286c2a610258ca
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
This makes binary detection exact in ResolveMerger and DiffFormatter
This has the same intention as
Id4342a199628d9406bfa04af1b023c27a47d4014 but preserves backward
compatibility of the signature of RawParseUtils.lineMap.
Change-Id: Ia24a4e716592bab3363ae24e3a46315a7511154f
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It often fails on my machine, both in maven and bazel.
This patch marks the test flaky[1] in bazel so that "bazel test" can
run it a few times before declaring failure.
[1] https://docs.bazel.build/versions/master/be/common-definitions.html#test.flaky
Bug: 534285
Change-Id: Ibe5414fefbffe4e8f86af7047608d51cf5df5c47
Eclipse reports these as errors, so remove them.
Change-Id: Ic53d8003f9faef38fe776af5a73794e7bb1dfc49
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add support for the "no-progress" parameter in the "fetch" command in
the fetch-pack/upload-pack protocol v2.
Change-Id: I6a6d6b1534f44845254b81d0e1f5c4ba2ac3d10b
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add support for the "thin-pack" parameter in the "fetch" command in
the fetch-pack/upload-pack protocol v2.
Change-Id: I39a37b2b66a16929137d35c718a3acf2afb6b0b5
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add basic support for the "fetch" command in the fetch-pack/upload-pack
protocol v2. This patch teaches "have" and "done".
The protocol specification (Documentation/technical/protocol-v2.txt in
the Git project) states:
want <oid>
Indicates to the server an object which the client wants to
retrieve. Wants can be anything and are not limited to
advertised objects.
It is unspecified whether the server should respect the
uploadpack.allowtipsha1inwant option etc. when serving packfiles. This
patch is conservative in that the server respects them.
Change-Id: I3dbec172239712ef9286a15b8407e86b87ea7863
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Add support for the "ref-prefix" parameter in the "ls-refs" command in
the fetch-pack/upload-pack protocol v2.
Change-Id: If9cf93b2646f75d50a11b5f482594f014d59a836
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Implement support for Git protocol v2's "ls-refs" command and its
"symrefs" and "peel" parameters.
This adds support for this command to UploadPack but the git://,
ssh://, and git:// transports do not make use of it yet. That will
have to wait for later patches.
Change-Id: I8abc6bcc6ed4a88c165677ff1245625aca01267b
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Add initial support for protocol v2 of the fetch-pack/upload-pack
protocol. This protocol is described in the Git project in
"Documentation/technical/protocol-v2.txt".
This patch adds support for protocol v2 (without any capabilities) to
UploadPack. Adaptations of callers to make use of this support will
come in subsequent patches.
[jn: split from a larger patch; tweaked the API to make UploadPack
handle parsing the extra parameters and config instead of requiring
each caller to do such parsing]
Change-Id: I79399fa0dce533fdc8c1dbb6756748818cee45b0
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Most pkt-lines (data-pkts) have the form
pkt-len pkt-payload
where pkt-len is a string of 4 hexadecimal digits representing the
size in bytes of the pkt-line. Since this size includes the size of
the pkt-len, no data-pkt has a length less than 4.
A pkt-line with a length field less than 4 can thus be used for
other purposes. In Git protocol v1, the only such pkt-line was
flush-pkt = "0000"
which was used to mark the end of a stream. Protocol v2 (see
Documentation/technical/protocol-v2.txt in git.git) introduces a
second special pkt-line type:
delim-pkt = "0001"
used to mark the end of a section within a stream, for example to
separate capabilities from the content of a command.
[jn: split out from a larger patch that made use of this support]
Change-Id: I10e7824fa24ed74c4f45624bd490bba978cf5c34
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
The existing RefDatabase#getRefs abstract method (to be implemented by
ref database backends) has the following issues:
- It returns a map with a key (the name of the ref with the prefix
removed) which is potentially superfluous (it can be derived by the
caller if need be) and confusing (in that the prefix is removed).
- The prefix is required to end with a '/', but some backends (e.g.
reftable) have fast search by prefix regardless of what the last
character of the prefix is.
Add a new method #getRefsByPrefix that does not have these issues. This
is non-abstract with a default implementation that uses #getRefs (for
backwards compatibility), but ref database backends can reimplement it.
This also prepares for supporting "ref-prefix" in the "ls-refs" command
in the fetch-pack/upload-pack protocol v2, which does not require that
the prefix end with a '/'.
Change-Id: I4c92f852e8c1558095dd460b5fd7b602c1d82df1
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Otherwise successful, non-conflicting merges will never get a
Gerrit Change-Id.
Bug: 358206
Change-Id: I9b599ad01d9f7332200c1d81a1ba6ce5ef990ab5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Various places on the client side of the push were creating unordered
maps and sets of ref names, resulting in ReceivePack processing commands
in an order other than what the client provided. This is normally not
problematic for clients, who don't typically care about the order in
which ref updates are applied to the storage layer.
However, it does make it difficult to write deterministic tests of
ReceivePack or hooks whose output depends on the order in which commands
are processed, for example if informational per-ref messages are written
to a sideband.[1]
Add a test that ensures the ordering of commands both internally in
ReceivePack and in the output PushResult.
[1] Real-world example:
https://gerrit-review.googlesource.com/c/gerrit/+/171871/1/javatests/com/google/gerrit/acceptance/git/PushPermissionsIT.java#149
Change-Id: I7f1254b4ebf202d4dcfc8e59d7120427542d0d9e
Previously @ was allowed e.g. in branch names, but not as the last
character. The case that @ is the last character was not handled.
Change-Id: Ic33870b22236f7a5ec7b54007f1b0cefd9354bfb
Getting attributes of files on Windows is an expensive operation.
Windows stores file attributes in the directory, so they are
basically available "for free" when a directory is listed. The
implementation of Java's Files.walkFileTree() takes advantage of
that (at least in the OpenJDK implementation for Windows) and
provides the attributes from the directory to a FileVisitor.
Using Files.walkFileTree() with a maximum depth of 1 is thus a
good approach on Windows to get both the file names and the
attributes in one go.
In my tests, this gives a significant speed-up of FileTreeIterator
over the "normal" way: using File.listFiles() and then reading the
attributes of each file individually. The speed-up is hard to
quantify exactly, but in my tests I've observed consistently 30-40%
for staging 500 files one after another, each individually, and up
to 50% for individual TreeWalks with a FileTreeIterator.
On Unix, this technique is detrimental. Unix stores file attributes
differently, and getting attributes of individual files is not costly.
On Unix, the old way of doing a listFiles() and getting individual
attributes (both native operations) is about three times faster than
using walkFileTree, which is implemented in Java.
Therefore, move the operation to FS/FS_Win32 and call it from
FileTreeIterator, so that we can have different implementations
depending on the file system.
A little performance test program is included as a JUnit test (to be
run manually).
While this does speed up things on Windows, it doesn't solve the basic
problem of bug 532300: the iterator always gets the full directory
listing and the attributes of all files, and the more files there are
the longer that takes.
Bug: 532300
Change-Id: Ic5facb871c725256c2324b0d97b95e6efc33282a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The class names imply that RecursiveMergerTest tests the RecursiveMerger
and ResolveMergerTest tests the ResolveMerger.
In fact, both of them include coverage of both strategies; the difference
is that RecursiveMergerTest is only testing criss-cross merges.
The tests cannot be combined into a single class because the criss-cross
test methods have additional data points.
Instead, rename the classes to more meaningful names.
Change-Id: I7ca8a03a3b7e351e2d4fcaca3b3186c098a3ca66
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Define strategiesUnderTest as an array of MergeStrategy using the
@DataPoints annotation, rather than two separate variables each
annotated as @DataPoint.
This makes the implementation consistent with RecursiveMergerTest.
Change-Id: I9f1d525b38cb59634ba054c7779dc4af1fc46e25
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The local variable 'remote' hides the class scope variable
of the same name.
Change-Id: I7410c33678677ce2a14691772466d91e8139e3fa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.11:
ObjectIdSerializer: Support serialization of known non-null ObjectId
Change-Id: Ie430fa2c5d13ae698d884a37d0d03884ebbf25ec
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Teach UploadPack to advertise the filter capability and support a
"filter" line in the request, accepting blob sizes only, if the
configuration variable "uploadpack.allowfilter" is true. This feature is
currently in the "master" branch of Git, and as of the time of writing,
this feature is to be released in Git 2.17.
This is incomplete in that the filter-by-sparse-specification feature
also supported by Git is not included in this patch.
If a JGit server were to be patched with this commit, and a repository
on that server configured with RequestPolicy.ANY or
RequestPolicy.REACHABLE_COMMIT_TIP, a Git client built from the "master"
branch would be able to perform a partial clone.
Change-Id: If72b4b422c06ab432137e9e5272d353b14b73259
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
The implementation of ObjectIdSerializer, added in change I7599cf8bd,
is not equivalent to the original implementation in Gerrit [1].
The Gerrit implementation provides separate methods to (de)serialize
instances of ObjectId that are known to be non-null. In these methods,
no "marker" is written to the stream. Replacing Gerrit's implementation
with ObjectIdSerializer [2] broke persistent caches because it started
writing markers where they were not expected [3].
Since ObjectIdSerializer is included in JGit 4.11 we can't change the
existing #write and #read methods. Keep those as-is, but extend the
Javadoc to clarify that they support possibly null ObjectId instances.
Add new methods #writeWithoutMarker and #readWithoutMarker to support
the cases where the ObjectId is known to be non-null and the marker
should not be written to the serialization stream.
Also:
- Replace the hard-coded `0` and `1` markers with constants that can
be linked from the Javadocs.
- Include the marker value in the "Invalid flag before ObjectId"
exception message.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/9792
[2] https://gerrit-review.googlesource.com/c/gerrit/+/165851
[3] https://gerrit-review.googlesource.com/c/gerrit/+/165952
Change-Id: Iaf84c3ec32ecf83efffb306fdb4940cc85740f3f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
In most of the tests, the temporary buffer is explicitly destroyed in
a finally block after being closed. This is not possible if using the
try-with-resource construct, because the variable is not accessible in
the finally block scope.
Change-Id: I3bab30695ddd12e1a0ae107989638428fe3ef551
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When an auto-closeable resources is not opened in try-with-resource,
the warning "should be managed by try-with-resource" is emitted by
Eclipse.
Fix the ones that can be silenced simply by moving the declaration of
the variable into a try-with-resource.
In cases where we explicitly call the close() method, for example in
tests where we are testing specific behavior caused by the close(),
suppress the warning.
Leave the ones that will require more significant refcactoring to fix.
They can be done in separate commits that can be reviewed and tested
in isolation.
Change-Id: I9682cd20fb15167d3c7f9027cecdc82bc50b83c4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Parameter negateFirstMatch is not honored anymore
Change-Id: Idff1a92643c1431c7e34a7730f8414135e1ac196
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The WorkingTreeSource produced an ObjectLoader that returned
inconsistent sizes: the file size in getSize(), but then a
correctly filtered smaller stream in openStream(). This resulted
either in an IOE "short read of block" or in an EOFException
depending on the resulting filtered size.
Fix this by ensuring that getSize() does return the size of the
filtered stream.
Bug: 530106
Change-Id: I7c7c85036047dc10030ed29c1d5a6c7f34f2bdff
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Replace hard-coded "UTF-8" string with the constant.
Change-Id: Ie812add2df28e984090563ec7c6e2c0366616424
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Respect merge=lfs and diff=lfs attributes where required to replace (in
memory) the content of LFS pointers with the actual blob content from
the LFS storage (and vice versa when staging/merging).
Does not implement general support for merge/diff attributes for any
other use case apart from LFS.
Change-Id: Ibad8875de1e0bee8fe3a1dffb1add93111534cae
Signed-off-by: Markus Duft <markus.duft@ssi-schaefer.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This is based on the ObjectIdSerialization class written by Shawn Pearce
for the Gerrit Code Review project in 2009 [1]. As mentioned in the
commit message there, it should be part of core JGit.
This implementation is slightly different to Shawn's version. Rather
than having separate methods for null/non-null ids, single methods are
implemented with @Nullable annotations.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/9792
Change-Id: I7599cf8bd1ecd546e2252783d6d672eb76804060
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Processing of negated rules, like !bin/ was not working correctly: they
were interpreted too broad, resulting in unexpected untracked files
which should actually be ignored
Bug: 409664
Change-Id: I0a422fd6607941461bf2175c9105a0311612efa0
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
**/ should match only directories, but not files
Change-Id: I885c83e5912cac5bff338ba657faf6bb9ec94064
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
* changes:
RepoCommand: generate relative submodule URLs from absolute URLs.
RepoCommand: don't record new commit if tree did not change
RepoCommand: persist unreadable submodules in .gitmodules
If a manifest file specifies an absolute URL on the same host on which
the superproject resides, rewrite the URLs to be relative.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id616611e5195998fb665c8e7806539a3a02e219a
In cases where a manifest file mixes different remotes, a Gerrit
server process may not have access to all remotes, and won't be able
to produce a full submodule tree.
Preserving this information in .gitmodules will let downstream clients
reconstruct the full tree.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I52f5d3f288e771dca0af2b4dd3f3fa0f940dcf15
* stable-4.10:
Fix ssh host name handling for Jsch
Jsch overrides the port in the URI with the one in ~/.ssh/config
Change-Id: I860fc61ceb12ae792b1ee7421046ecd32373b9f8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
Fix ssh host name handling for Jsch
Jsch overrides the port in the URI with the one in ~/.ssh/config
Change-Id: Iff9076f65e767bbe8df016337b631bdaeb40ad98
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If we give Jsch access to the ssh config file, we must _not_ resolve
the host name from the alias. Instead we must give the alias (i.e.,
the host name as is in the URI) to Jsch, so that it finds the same
ssh config entry.
Otherwise if the hostname in the URI, which is taken as an alias in
ssh config ("Host" line), is unequal to the "Hostname" line, and
there happens to be another ssh config entry with that translated
host name as alias, Jsch will pick up that second entry, and we end
up with a strange mixture of both.
Add tests for this case.
Bug: 531118
Change-Id: I249d8c073b0190ed110a69dca5b9be2a749822c3
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Jsch unconditionally overwrites the port from the ssh config
file (if a port is specified there), even if the URI explicitly does
give a different port.
Fix this, and add tests.
Change-Id: I7b014543c7ece26270e366db39d7647f82d64f0d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The InitCommand returns a Git that is instantiated with the newly
created Repository, but the Repository is not closed with the Git
resulting in resource leaks.
Create the Git with `closeRepo` set to true, such that the Repository
is also closed when the Git is closed.
Adjust the tests to use try-with-resource on the Git instance.
Change-Id: Ib26e7428c7d8840956d1edb09e53b93e23e6fe5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Android an Chrome have several repos with >300k refs. We sometimes see
negotiations of >100k rounds. This change provides a "minimal negotiation"
feature on the client side that limits how many "have" lines the client
sends. The client extracts the current SHA-1 values for the refs in its
wants set, and terminates negotiation early when all of those values have
been sent as haves. If a new branch is being fetched then that set will
be empty and the client will terminate after current default minimum
of two rounds.
This feature is gated behind a "fetch.useminimalnegotiation" configuration
flag, which defaults to false.
Change-Id: Ib12b095cac76a59da6e8f72773c4129e3b32ff2b
Signed-off-by: Terry Parker <tparker@google.com>
This would allow compact and GC process to clean up duplicate ref names in the reftables.
Change-Id: I2b9df0bf72dba63cc3525e374982e60559a776c2
Signed-off-by: Minh Thai <mthai@google.com>
When CleanCommand is collecting the files and folders to be deleted
it may happen that the list of directories contains obsolete entries.
E.g. a folder and its parent folder may be in the list. Only the
parent folder would be sufficient.
This was a reason for hitting FileNotFoundExceptions when finally
trying to delete the files and folders. Improve CleanCommand
to ignore files to be deleted which are already gone.
Bug: 514434
Change-Id: I10caa01bfb9cec5967dfdaea50c6e4a713eeeabd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
After packaging references, the folders containing these references are
not deleted. In a busy repository, this causes operations to slow down
as traversing the references tree becomes longer.
Delete empty reference folders after the loose references have been
packed.
To avoid deleting a folder that was just created by another concurrent
operation, only delete folders that were not modified in the last 30
seconds.
Signed-off-by: Hector Oswaldo Caballero <hector.caballero@ericsson.com>
Change-Id: Ie79447d6121271cf5e25171be377ea396c7028e0
This doesn't handle the really hard thing, which is merging spurious
conflicts inside .gitmodules files. That's OK: git.git doesn't
either. Users can resolve the conflict themselves and then commit
the merge.
Previously, jgit would crash when attempting to merge conflicting
submodule changes. Even if there was no conflict, after a merge which
adds submodules, the repository would have been missing empty
directories for newly-added submodules.
This patch fixes the crash, and adds the empty directories where
necessary. It ensures that the index is in a conflicted state when
submodule changes conflict.
Reported-by: Alexey Korobkov
Bug: 494551
Change-Id: I79db6798c2bdcc1159b5b2589b02da198dc906a1
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Commit fc7d407 corrected line endings for working tree files resulting
from merges when CRLF translations are to be done. However, that also
resulted in the file content being put as-is into the index, which is
wrong. The index must contain the file content with reverse CRLF
translations applied.
With core.autocrlf=true, the working tree file should have CR-LF, but
the index blob must still contain only LF.
Fix this oversight and apply the inverse translation when updating the
index, similar to what is done in AddCommand.
Bug: 499615
Change-Id: I3a33931318bdb580b2390f3450f91ea8f258a6a4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Consistently require 1.7.0. We ship 1.7.2 with our p2 repository but
there is no need to require 1.7.2 since it should be API compatible with
1.7.0.
Change-Id: I8467bb14316cb24daa79e89275332107d2716190
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Merges are performed using the raw text as stored in the git
repository. When we write the merge result, we must apply the
correct CRLF settings. Otherwise the line endings in the result
will be wrong.
Bug: 499615
Change-Id: I37a9b987e9404c97645d2720cd1c7c04c076a96b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* Section and key names in git config files are case-insensitive.
* If an include directive is invalid, include the line in the
exception message.
* If inclusion of the included file fails, put the file name into
the exception message so that the user knows in which file the
problem is.
Change-Id: If920943af7ff93f5321b3d315dfec5222091256c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Jsch caches keys (aka identities) specified in ~/.ssh/config via
IndentityFile only for the current Jsch Session. This results in
multiple password prompts for successive sessions.
Do the handling of IdentityFile exclusively in JGit, as it was before
4.9. JGit uses different Jsch instances per host and caches the
IdentityFile there, allowing it to be re-used in different sessions
for the same host.
* Add comments to explain this.
* Move the JschBugFixingConfig from OpenSshConfig to
JschConfigSessionFactory to have all these Jsch work-arounds
in one place.
* Make that config hide the IdentityFile config from Jsch to avoid
that Jsch overrides the JGit behavior.
Bug: 529173
Change-Id: Ib36c34a2921ba736adeb64de71323c2b91151613
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Instead of instantiating a new Random on each invocation of newLargeBlob,
create it once and reuse it.
This fixes a warning raised by Spotbugs about the Random object being
created and only used once.
Change-Id: I5b8e6ccbbc92641811537808aed9eae2034c1133
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Change-Id: Iaaefc2cbafbf083d6ab158b1c378ec69cc76d282
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
Strings#convertGlob: fix escaping of patterns like [\[].
Change-Id: I18d55537002b3153db35f8a6b60f2f5317d17248
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Originally the patterns were escaped twice leading
to wrong matching results.
Bug: 528886
Change-Id: I26e201b4b0ef51cac08f940b76f381260fa925ca
Signed-off-by: Dmitry Pavlenko <pavlenko@tmatesoft.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
These are ignored by C git when parsing:
$ git config -f - --list <<EOF
[foo "x\0y"]
bar = baz
[foo "x\qy"]
bar = baz
[foo "x\by"]
bar = baz
[foo "x\ny"]
bar = baz
[foo "x\ty"]
bar = baz
EOF
foo.x0y.bar=baz
foo.xqy.bar=baz
foo.xby.bar=baz
foo.xny.bar=baz
foo.xty.bar=baz
This behavior is different from value parsing, where an invalid escape
sequence is an error (which JGit already does as well):
$ git config -f - --list <<EOF
[foo]
bar = x\qy
EOF
fatal: bad config line 2 in standard input
Change-Id: Ifd40129b37d9a62df3d886d8d7e22f766f54e9d1
* stable-4.9:
PackInserter: Ensure objects are written at the end of the pack
ObjectInserter: Add warning about mixing read-back with writes
Change-Id: I308e7c1c6b72e8d4d9b5d0f4f51e9815fc92d7d7
When interleaving reads and writes from an unflushed pack, we forgot to
reset the file pointer back to the end of the file before writing more
new objects. This had at least two unfortunate effects:
* The pack data was potentially corrupt, since we could overwrite
previous portions of the file willy-nilly.
* The CountingOutputStream would report more bytes read than the size
of the file, which stored the wrong PackedObjectInfo, which would
cause EOFs during reading.
We already had a test in PackInserterTest which was supposed to catch
bugs like this, by interleaving reads and writes. Unfortunately, it
didn't catch the bug, since as an implementation detail we always read a
full buffer's worth of data from the file when inflating during
readback. If the size of the file was less than the offset of the object
we were reading back plus one buffer (8192 bytes), we would completely
accidentally end up back in the right place in the file.
So, add another test for this case where we read back a small object
positioned before a large object. Before the fix, this test exhibited
exactly the "Unexpected EOF" error reported at crbug.com/gerrit/7668.
Change-Id: I74f08f3d5d9046781d59e5bd7c84916ff8225c3b
Previously, Config was using the same method for both escaping and
parsing subsection names and config values. The goal was presumably code
savings, but unfortunately, these two pieces of the git config format
are simply different.
In git v2.15.1, Documentation/config.txt says the following about
subsection names:
"Subsection names are case sensitive and can contain any characters
except newline (doublequote `"` and backslash can be included by
escaping them as `\"` and `\\`, respectively). Section headers cannot
span multiple lines. Variables may belong directly to a section or to
a given subsection."
And, later in the same documentation section, about values:
"A line that defines a value can be continued to the next line by
ending it with a `\`; the backquote and the end-of-line are stripped.
Leading whitespaces after 'name =', the remainder of the line after
the first comment character '#' or ';', and trailing whitespaces of
the line are discarded unless they are enclosed in double quotes.
Internal whitespaces within the value are retained verbatim.
Inside double quotes, double quote `"` and backslash `\` characters
must be escaped: use `\"` for `"` and `\\` for `\`.
The following escape sequences (beside `\"` and `\\`) are recognized:
`\n` for newline character (NL), `\t` for horizontal tabulation (HT,
TAB) and `\b` for backspace (BS). Other char escape sequences
(including octal escape sequences) are invalid."
The main important differences are that subsection names have a limited
set of supported escape sequences, and do not support newlines at all,
either escaped or unescaped. Arguably, it would be easy to support
escaped newlines, but C git simply does not:
$ git config -f foo.config $'foo.bar\nbaz.quux' value
error: invalid key (newline): foo.bar
baz.quux
I468106ac was an attempt to fix one bug in escapeValue, around leading
whitespace, without having to rewrite the whole escaping/parsing code.
Unfortunately, because escapeValue was used for escaping subsection
names as well, this made it possible to write invalid config files, any
time Config#toText is called with a subsection name with trailing
whitespace, like {foo }.
Rather than pile hacks on top of hacks, fix it for real by largely
rewriting the escaping and parsing code.
In addition to fixing escape sequences, fix (and write tests for) a few
more issues in the old implementation:
* Now that we can properly parse it, always emit newlines as "\n" from
escapeValue, rather than the weird (but still supported) syntax with a
non-quoted trailing literal "\n\" before the newline. In addition to
producing more readable output and matching the behavior of C git,
this makes the escaping code much simpler.
* Disallow '\0' entirely within both subsection names and values, since
due to Unix command line argument conventions it is impossible to pass
such values to "git config".
* Properly preserve intra-value whitespace when parsing, rather than
collapsing it all to a single space.
Change-Id: I304f626b9d0ad1592c4e4e449a11b136c0f8b3e3
It was already increased in 61a943e and 661232b but is still not
enough to take into account snapshot versions that are 100 or more
commits ahead of tag, i.e. 4.9.2.201712150930-r.105-gc1d37ca27
Change-Id: Ibeff73adae06b92fe5bb9c5eced9e4c6a08c437c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The Config class must be safe to run against untrusted input files.
Reading arbitrary local system paths using include.path is risky for
servers, including Gerrit Code Review.
This was fixed on master [1] by making "readIncludedConfig" a noop
by default. This allows only FileBasedConfig, which originated from
local disk, to read local system paths.
However, the "readIncludedConfig" method was only introduced in [2]
which was needed by [3], both of which are only on the master branch.
On the stable branch only Config supports includes. Therefore this
commit simply disables the include functionality.
[1] https://git.eclipse.org/r/#/c/113371/
[2] https://git.eclipse.org/r/#/c/111847/
[3] https://git.eclipse.org/r/#/c/111848/
Bug: 528781
Change-Id: I9a3be3f1d07c4b6772bff535a2556e699a61381c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The Config class must be safe to run against untrusted input files.
Reading arbitrary local system paths using include.path is risky for
servers, including Gerrit Code Review. Return null by default to
incide the include should be ignored.
Only FileBasedConfig which originated from local disk should be trying
to read local system paths. FileBasedConfig already overrides this
method with its own implementation.
Change-Id: I2ff31753868aa1bbac4a6843a4c23e50bd6f46f3
Path.getFileName() may return null if the path has zero elements.
Enclose the dereference in a null-check.
Change-Id: I7ea3d3f07edc13a80b593d28e8fd512a4e1ed56b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
There are several different version ranges specified in the various
manifest files.
Align them all to the same range: [4.12,5.0.0)
Change-Id: I02205b8b8546c9f53ed431b5fd9abf6ddcda4423
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Instead of hard-coding the charset strings "US-ASCII", "UTF-8", and
"ISO-8859-1", use the corresponding constants from StandardCharsets.
UnsupportedEncodingException is not thrown when the StandardCharset
constants are used, so remove the now redundant handling.
Because the encoding names are no longer hard-coded strings, also
remove redundant $NON-NLS warning suppressions.
Also replace existing usages of the constants with static imports.
Change-Id: I0a4510d3d992db5e277f009a41434276f95bda4e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
ConfigTest#pathToString is not visible to FileBasedConfigTest when
bulding with bazel.
Move it to FileUtils rather than messing about with the bazel build
rules to make it visible.
Change-Id: Idcfd4822699dac9dc4a426088a929a9cd31bf53f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Relative include.path are now resolved against the config's parent
directory. include.path starting with ~/ are resolved against the
user's home directory
Change-Id: I91911ef404126618b1ddd3589294824a0ad919e6
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Change-Id: I37a2ef611aef97faf1b891a9660c1745435a915d
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
GC: Delete stale temporary packs and indexes
Change-Id: I49b37845ee8a465404b801a2d8de0205a2e7ba30
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When a GC operation is interrupted, temporary packs and indexes can be
left on the pack folder. In big, busy repositories this can lead to
significant amounts of wasted disk space if this interruption is done
with a certain frequency.
Remove stale temporary packs and indexes at the end of the GC process so
they do not accumulate. To avoid interfering with a possible concurrent
JGit GC process in the same repository, only delete temporary files that
are older than one day.
Change-Id: If9b6c1e57fac8a6a0ecc0a703089634caba4caae
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
* stable-4.9:
Ignore warning for minor version change without API change
Silence boxing warning
Prepare 4.5.5-SNAPSHOT builds
JGit v4.5.4.201711221230-r
Fix LockFile semantics when running on NFS
Honor trustFolderStats also when reading packed-refs
Prepare 4.5.4-SNAPSHOT builds
JGit v4.5.3.201708160445-r
Change-Id: Icc33d2e36f140e8714fce088379673a8834ae9de
- this is a new warning option in Eclipse 4.7 and higher
- we always change version of all bundles in a release to keep release
engineering simple
Change-Id: Ic7523d77b67b2802f1bab3bc70af250d712a034f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
Yet another work-around for a Jsch bug: timeouts
Change-Id: I7cf227c62a3c06f91cee1a6c61719b6fe50da883
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Jsch 0.1.54 passes on the values from ~/.ssh/config for
"ServerAliveInterval" and "ConnectTimeout" as read from
the config file to java.net.Socket.setSoTimeout(). That
method expects milliseconds, but the values in the config
file are seconds!
The missing conversion in Jsch means that the timeout is
set way too low, and if the server doesn't respond within
that very short time frame, Jsch kills the connection and
then throws an exception with a message such as "session is
down" or "timeout in waiting for rekeying process".
As a work-around, do the conversion to milliseconds in the
Jsch-facing Config interface of OpenSshConfig. That way Jsch
already gets these values as milliseconds.
Bug: 526867
Change-Id: Ibc9b93f7722fffe10f3e770dfe7fdabfb3b97e74
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
A tombstone will prevent a delayed reference update from resurrecting the
deleted reference.
Change-Id: Id9f4df43d435a299ff16cef614821439edef9b11
Signed-off-by: Minh Thai <mthai@google.com>
Do not use 0 as the unset value for minUpdateIndex, as input reftables
may have minUpdateIndex starting at 0.
Change-Id: Ie040a6b73d4a5eba5521e51d0ee4580713c84a3e
Signed-off-by: Minh Thai <mthai@google.com>
So far, in order to get the pack directory it was necessary to resolve
it from the object directory. This resolution is already done when
creating the object directory, so simplify the call by just adding a
getter to the pack directory.
Change-Id: I69e783141dc6739024e8b3d5acc30843edd651a7
Signed-off-by: Hector Caballero <hector.caballero@ericsson.com>
When invoking File.toPath(), an (unchecked) InvalidPathException may be
thrown which should be converted to a checked IOException.
For now, we will replace File.toPath() by FileUtils.toPath() only for
code which can already handle IOExceptions.
Change-Id: I0f0c5fd2a11739e7a02071adae9a5550985d4df6
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
* changes:
BitmapWalker: do not revisit objects in bitmap
Use bitmaps for non-commit reachability checks
Make PackWriterBitmapWalker public
UploadPackTest: construct commits in test method
Currently, unless RequestPolicy#ANY is used, UploadPack rejects all
non-commit "want" lines unless they were advertized. This is fine,
except when "uploadpack.allowreachablesha1inwant" is true
(corresponding to RequestPolicy#REACHABLE_COMMIT), in which case one
would expect that "want"-ing anything reachable would work.
(There is no restriction that "want" lines must only contain commits -
it is allowed for refs to directly point to trees and blobs, and
requesting for them using "want" lines works.)
This commit has been written to avoid performance regressions as much
as possible. In the usual (and currently working) case where the only
unadvertized things requested are commits, we do a standard RevWalk in
order to avoid incurring the cost of loading bitmaps. However, if
unadvertized non-commits are requested, bitmaps are used instead, and
if there are no bitmaps, a WantNotValidException is thrown (as is
currently done).
Change-Id: I68ed4abd0e477ff415c696c7544ccaa234df7f99
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
JGit's delta handling code requires the target to be a single byte
array. Any attempt to inflate a delta larger than fits in the 2GiB
limit will fail with some form of array index exceptions. Check for
this overflow early and abort pack parsing.
Change-Id: I5bb3a71f1e4f4e0e89b8a177c7019a74ee6194da
This new warning was introduced in Eclipse 4.7 Oxygen [1].
The only instances of the warning are in test code that is asserting
that some class does not compare equal to Strings. As in the Gerrit
project [2] these asserts are arguably overkill, but arguably also
a reasonable test of an equals implementation. Ignore the warning in
these cases.
Note that if the project is opened in an earlier version of Eclipse,
a warning "Unsupported @SuppressWarnings" will be emitted.
[1] https://www.eclipse.org/eclipse/news/4.7/M6/
[2] https://gerrit-review.googlesource.com/#/c/gerrit/+/110339/
Change-Id: I08ea33d71e6009cf0f37e6492a475931f447256b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
In a subsequent commit, more tests will be added. This commit allows
those tests to reuse fields.
Change-Id: Icbd17d158cfe3ba4dacbd8a11a67f9e7607b41b3
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* stable-4.9:
PackInserter: Implement newReader()
Move some strings from DfsText to JGitText
FileRepository: Add pack-based inserter implementation
ObjectDirectory: Factor a method to close open pack handles
ObjectDirectory: Remove last modified check in insertPack
Change-Id: Ifc9ed6f5d8336bc978818a64eae122bceb933e5d
Applications that use ObjectInserters to create lots of individual
objects may prefer to avoid cluttering up the object directory with
loose objects. Add a specialized inserter implementation that produces a
single pack file no matter how many objects. This inserter is loosely
based on the existing DfsInserter implementation, but is simpler since
we don't need to buffer blocks in memory before writing to storage.
An alternative for such applications would be to write out the loose
objects and then repack just those objects later. This operation is not
currently supported with the GC class, which always repacks existing
packs when compacting loose objects. This in turn requires more
CPU-intensive reachability checks and extra I/O to copy objects from old
packs to new packs.
So, the choice was between implementing a new variant of repack, or not
writing loose objects in the first place. The latter approach is likely
less code overall, and avoids unnecessary I/O at runtime.
The current implementation does not yet support newReader() for reading
back objects.
Change-Id: I2074418f4e65853b7113de5eaced3a6b037d1a17
This makes detection of binaries exact for ResolveMerger and
DiffFormatter: they will classify files as binary regardless of where
the '\0' occurs in the text.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id4342a199628d9406bfa04af1b023c27a47d4014
This method creates a RawText from a blob, but avoids reading the blob
if the start contains null bytes. This should reduce the amount of
garbage that Gerrit produces for changes with binaries.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Idd202d20251f2d1653e5f1ca374fe644c2cf205f
* stable-4.9:
Avoid bad rounding "1 year, 12 months" in date formatter
Ensure that ~ in ssh config is replaced before Jsch sees it
Change-Id: If6ca55f9447aaea3d7c2d36c03520d5e6dd5193e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Round first, then calculate the labels. This avoids "x years, 12 months"
and instead produces "x+1 years".
One test case has been added for the original example the bug was found
with, and one assertion has been moved from an existing test case to the
new test case, since it also triggered the bug.
Bug: 525907
Change-Id: I3270af3850c4fb7bae9123a0a6582f93055c9780
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This ensure DfsReftableDatabase is tested by the same test suites that
use/test InMemoryRepository. It also simplifies the logic of
InMemoryRepository and brings its compatibility story closer to any
other DFS repository that uses reftables for its reference storage.
Change-Id: I881469fd77ed11a9239b477633510b8c482a19ca
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Terry Parker <tparker@google.com>
Do tilde replacement for values from the ssh config file that are
file names in all cases to make sure that they are already replaced
when Jsch tries to get the values.
Previously, OpenSshConfig did tilde replacement only for the
IdentityFile in the JGit-facing "Host" interface and left the
replacement in the Jsch-facing "Config" interface to Jsch.
But on Windows the JGit notion of what should be used to replace the
tilde differs from Jsch's replacement. Jsch always replaces the tilde
by the value of the system property "user.home", whereas JGit also
considers some environment variables like %HOME%. This can lead to
rather surprising failures as in the case of bug 526175 where
%HOME% != user.home.
Prior to commit 9d24470 (i.e.,prior to JGit 4.9.0) this problem never
occurred because Jsch was completely unaware of the ssh config file
and all host and IdentityFile handling happened exclusively in JGit.
Bug: 526175
Change-Id: I1511699664ffea07cb58ed751cfdb79b15e3a99e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Allow creating symbolic references with link, and deleting them or
switching to ObjectId with unlink. How this happens is up to the
individual RefDatabase.
The default implementation detaches RefUpdate if a symbolic reference
is involved, supporting these command instances on RefDirectory.
Unfortunately the packed-refs file does not support storing symrefs,
so atomic transactions involving more than one symref command are
failed early.
Updating InMemoryRepository is deferred until reftable lands, as I
plan to switch InMemoryRepository to use reftable for its internal
storage representation.
Change-Id: Ibcae068b17a2fc6d958f767f402a570ad88d9151
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Terry Parker <tparker@google.com>
Round first, then calculate the labels. This avoids "x years, 12 months"
and instead produces "x+1 years".
One test case has been added for the original example the bug was found
with, and one assertion has been moved from an existing test case to the
new test case, since it also triggered the bug.
Bug: 525907
Change-Id: I3270af3850c4fb7bae9123a0a6582f93055c9780
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Shadow commits in the RevWalk in the UploadPack object may cause the
UNINTERESTING flag not being carried over to their parents commits since
they were marked NO_PARENTS during the assumeShallow or
initializeShallowCommits call.
A new RevWalk needs to be created for this reason, but instead of
creating a new RevWalk from Repository, we can reuse the ObjectReader in
the RevWalk of UploadPack to load objects.
Change-Id: Ic3fee0512d35b4f555c60e696a880f8b192e4439
Signed-off-by: Zhen Chen <czhen@google.com>
Per git-config(1), core.logAllRefUpdates auto-creates reflogs for HEAD
and for refs under heads, notes, tags, and for HEAD. Add notes and
remove stash from ReflogWriter#shouldAutoCreateLog. Explicitly force
writing reflogs for refs/stash at call sites, now that this is
supported.
Change-Id: I3a46d2c2703b7c243e0ee2bbf6948279800c485c
Even if a repository has core.logAllRefUpdates=true, ReflogWriter does
not create reflog files unless the refs are under a hard-coded list of
prefixes, or unless the forceWrite bit is set. Expose the forceWrite bit
on a per-update basis in RefUpdate/BatchRefUpdate/ReceiveCommand,
creating RefLogWriters as necessary.
Change-Id: Ifc851fba00f76bf56d4134f821d0576b37810f80
The ReflogWriter constructor just took a Repository and called
getDirectory() on it to figure out the reflog dirs, but not all
Repository instances use this storage format for reflogs, so it's
incorrect to attempt to use ReflogWriter when there is not a
RefDirectory directly involved. In practice, ReflogWriter was mostly
only used by the implementation of RefDirectory, so enforcing this is
mostly just shuffling around calls in the same internal package.
The one exception is StashDropCommand, which writes to a reflog lock
file directly. This was a reasonable implementation decision, because
there is no general reflog interface in JGit beyond using
(Batch)RefUpdate to write new entries to the reflog. So to implement
"git stash drop <N>", which removes an arbitrary element from the
reflog, it's fair to fall back to the RefDirectory implementation.
Creating and using a more general interface is well beyond the scope of
this change.
That said, the old behavior of writing out the reflog file even if
that's not the reflog format used by the given Repository is clearly
wrong. Fail fast in this case instead.
Change-Id: I9bd4b047bc3e28a5607fd346ec2400dde9151730
This test was never being run. Since it was introduced it was
named "notest.." which meant it didn't run with JUnit3, and
since it is not annotated @Test it also doesn't run with JUnit4.
When compiling with Bazel 0.6.0, error-prone raises an error
that the public method is not annotated with @Ignore or @Test.
Given that the test has never been run anyway, we can just
remove it.
Bug: 525415
Change-Id: Ie9a54f89fe42e0c201f547ff54ff1d419ce37864
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Git has a rather elaborate mechanism to specify HTTP configuration
options per URL, based on pattern matching the URL against "http"
subsection names.[1] The URLs used for this matching are always the
original URLs; redirected URLs do not participate.
* Scheme and host must match exactly case-insensitively.
* An optional user name must match exactly.
* Ports must match exactly after default ports have been filled in.
* The path of a subsection, if any, must match a segment prefix of
the path of the URL.
* Matches with user name take precedence over equal-length path
matches without, but longer path matches are preferred over
shorter matches with user name.
Implement this for JGit. Factor out the HttpConfig from TransportHttp
and implement the matching and override mechanism.
The set of supported settings is still the same; JGit currently
supports only followRedirects, postBuffer, and sslVerify, plus the
JGit-specific maxRedirects key.
Add tests for path normalization and prefix matching only on segment
separators, and use the new mechanism in SmartClientSmartServerSslTest
to disable sslVerify selectively for only the test server URLs.
Compare also bug 374703 and bug 465492. With this commit it would be
possible to set sslVerify to false for only the git server using a
self-signed certificate instead of having to switch it off globally
via http.sslVerify.
[1] https://git-scm.com/docs/git-config
Change-Id: I42a3c2399cb937cd7884116a2a32fcaa7a418fcb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This will be used later when adding for support for recursing
submodules on push.
Change-Id: Ie2a183e5404a32046de9f6524e6ceeec37919671
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
With atomic ref updates using packed refs, JGit did not fire a
RefsChangedEvent. This resulted in a user-visible regression in
EGit: the UI would not update after a "Fetch from upstream...".
Presumably it would also make Gerrit miss out on ref changes?
Strengthen the BatchRefUpdateTest by also asserting the expected
number of RefsChangedEvents, and ensure modCnt is incremented in
RefDirectory.commitPackedRefs() when refs really changed (as opposed
to some internal housekeeping operation, such as packing loose refs).
Bug: 521296
Change-Id: Ia985bda1d99f45a5f89c8020ca4845e7a66e743e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Much of the time the caller can specify a RefSpec succinctly using a
string, and doesn't care about calling setters. Add a convenience method
for this case, and use it where applicable in JGit core.
Change-Id: Ic3fac7fc568eee4759236a5264d2e7e5f9b9716d
An application can choose to invoke setAdvertisedRefs multiple times,
for example several AdvertiseRefsHook installed in a chain. Each of
these invocations populates the advertisedHaves collection with the
unique set of ObjectIds.
This can lead to a server over-advertising with ".have" lines if the
first hook pushes in a lot of references, and the second hook filters
this to a subset. ReceivePack will advertise the unique objects from
the first hook using ".have" lines, which may lead to a huge
advertisement sent to the client.
This can also contribute to a very slow connectivity check after the
pack is parsed as ReceivePack calls markUninteresting on every commit
in advertisedHaves. This may require expanding a lot of subtrees to
mark all trees as uninteresting as well. On a very big repository
this can lead to a many-second stall.
Clear the advertisedHaves collection any time the refs are updated.
Add a test to verify the correct set of objects was sent.
Change-Id: I97f6998d0597251444a2e846a3ea1f461bae96f9
Matthias Sohn