* changes:
Compare getting all refs except specific refs with seek and with filter
Add getsRefsByPrefixWithSkips (excluding prefixes) to ReftableDatabase
Add seekPastPrefix method to RefCursor
We sometimes want to get all the refs except specific prefixes,
similarly to getRefsByPrefix that gets all the refs of a specific
prefix.
We now create a new method that gets all refs matching a prefix except a
set of specific prefixes.
One use-case is for Gerrit to be able to get all the refs except
refs/changes; in Gerrit we often have lots of refs/changes, but very
little other refs. Currently, to get all the refs except refs/changes we
need to get all the refs and then filter the refs/changes, which is very
inefficient. With this method, we can simply skip the unneeded prefix so
that we don't have to go over all the elements.
RefDirectory still uses the inefficient implementation, since there
isn't a simple way to use Refcursor to achieve the efficient
implementation (as done in ReftableDatabase).
Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I8c5db581acdeb6698e3d3a2abde8da32f70c854c
org.eclipse.jgit.http.apache uses several features that exist only
since httpclient 4.4, but its MANIFEST.MF still had a lower bound of
4.3.0. Bump this to 4.4.0 for all packages from httpclient. 4.3.0 for
the packages from httpcore is fine.
Do a similar clean-up in the other bundles using packages from Apache
httpclient (http.test, lfs, lfs.server, lfs.server.test)
Bug: 570451
Change-Id: Iffdde2a9bd0d65db2e5201a08cffbf03597e2866
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.
Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.
Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.
No preemptive authentication will be done if the connection is
redirected to a different host.
Add tests.
Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.
Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.
Make the global static factory reference volatile.
Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.
Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This would allow other JGit users to access and reuse the constants.
Change-Id: I1608802f45586af5f8582afa592e26679e9cebe3
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It seems Eclipse 4.18 reports them as error whereas earlier versions
ignored this maybe since we don't require javadoc comments for all the
test bundles.
Change-Id: I3f4d42ce681ea5c2b4b302991d2641290ac8561d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This reverts commit f802f06e7f.
I had misunderstood how protocol V2 works. This implementation only
works if the negotiation during fetch is done in one round.
Fixing this is substantial work in BasePackFetchConnection. Basically
I think I'd have to change back negotiate to the V0 version, and have
a doFetch() that does
if protocol V2
doFetchV2()
else
doFetchV0()
with doFetchV0 the old code, and doFetchV2 completely new.
Plus there would need to be a HTTP test case requiring several
negotiation rounds.
This is a couple of days work at least, and I don't know when I will
have the time to revisit this. So although the rest of the code is
fine I prefer to back this out completely and not leave a only half
working implementation in the code for an indeterminate time.
Bug: 553083
Change-Id: Icbbbb09882b3b83f9897deac4a06d5f8dc99d84e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.
A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.
In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.
BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)
BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (simplified ACK handling, delimiters, section
headers).
In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.
Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. Do the same for the SSH transport tests.
Bug: 553083
Change-Id: Ice9866aa78020f5ca8f397cde84dc224bf5d41b4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Avoid trying other authentication methods on SocketException or on
InterruptedIOException. SocketException is rather fatal, such as
nothing listening on the peer's port, connection reset, or it could
be a connection time-out.
Time-outs enforced by Timeout{Input,Output}Stream may result in
InterruptedIOException being thrown.
In both cases, it makes no sense to try other authentication methods,
and doing so may wrongly report "authentication not supported" or
"cannot open git-upload-pack" or some such instead of reporting a
time-out.
Bug: 563138
Change-Id: I0191b1e784c2471035e550205abd06ec9934fd00
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Bazel buildtools project includes in addition to buildifier also unused
deps and buildozer utilities, that detect unused dependencies and fix
them by applying the removal to the build files. This change is created
by installing unused_deps from buildtools@HEAD and running:
$ unused_deps //...
and applying the suggested modifications.
Change-Id: Iad74ec2fa719475b29391586f40b13ae30477004
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Since version 4.13 JUnit has an assertThrows method. Remove the
implementation in MoreAsserts and use the one from JUnit.
CQ: 21439
Change-Id: I086baa94aa3069cebe87c4cbf91ed1534523c6cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
As UploadPackErrorHandler's Javadoc says, UploadPackServlet should have
called uploadWithExceptionPropagation and let UploadPackErrorHandler to
handle the exception. Fix UploadPackServlet.
Change-Id: I1f9686495fcf3ef28598ccdff3e6f76a16c8bca3
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Since [1] the gerrit project includes jgit as a submodule, and has this
warning enabled, resulting in 100s of warnings in the console.
Also enable the warning here, and fix them.
At the same time, add missing braces around adjacent and nearby one-line
blocks.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/227897
Change-Id: I81df3fc7ed6eedf6874ce1a3bedfa727a1897e4c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This introduces ReftableBatchRefUpdate and ReftableDatabase, as
generic classes, with some code moved to DfsReftableBatchRefUpdate and
DfsReftableDatabase.
Clarify thread-safety requirements by asserting locked status in
accessors, and acquiring locks in callers. This does not fix threading
problems, because ReftableBatchRefUpdate already wraps the whole
transaction in a lock.
This also fixes a number of bugs in ReftableBatchRefUpdate:
* non-atomic updates should not bail on first failure
* isNameConflicting should also check for conflicts between names that
are added and removed in the BatchRefUpdate.
Change-Id: I5ec91173ea9a0aa19da444c8c0b2e0f4e8f88798
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.5:
bazel: fix running http tests
Set parameter name in parameterized http tests
Format BUILD files with buildifier
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Idb0848dd33a76328b24908dc86db335cca742a1c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.4:
Format BUILD files with buildifier
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Idf3abd80ad3b00188f655e638d9908228770911f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Other than Maven bazel seems unable to cope with abstract
AllFactoriesHttpTestCase having no test methods, hence tag this class
with @Ignore.
Change-Id: I9dfe43f882ad073b284648e24844b51877d87776
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This shows the class name of the HTTP factory used in each test run.
Change-Id: I7c1df20f0e138dc7e3120fe87e80d40ab17dd8c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Iaea77a89fcd821df5dbb24f5e3e39e6d1dfc0bd4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Using buildifier from master branch and the command:
$ buildifier -r -lint fix -warnings all .
Change-Id: I52a18c09a1f192e2141f4156f9c1aabbbdbc910a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: I0d8b7f79177a20dc00c89e2cf0005eb3d3039532
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Iea37a8e39e9d4872dc607c9222dcf191ce4e4757
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Jonathan Tan