Commit Graph

5712 Commits

Author SHA1 Message Date
Matthias Sohn c2990810e9 Merge "reftable: add random suffix to table names" 2021-02-05 19:43:42 -05:00
Alina Djamankulova f077158acf TransportGitAnon: remove unnecessary socket bind to a local address
before connecting.

A socket gets bound on connect in the next line.

Signed-off-by: Alina Djamankulova <adjama@google.com>
Change-Id: I69a423c592e2fdd582b3c40099137b4ef3d05b39
2021-02-05 18:14:59 -05:00
Thomas Wolf 91ddc0e284 IO: fix IO.readFully(InputStream, byte[], int)
This would run into an endless loop if the offset given was not zero.
Fix the logic to exit the read loop when the buffer is full.

Luckily all existing uses of this method call it only with offset zero.

Change-Id: I0ec2a4fb43efe4a605d06ac2e88cf155d50e2f1e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-31 10:31:10 +01:00
Jonathan Nieder 59420886e9 Merge "Move reachability checker generation into the ObjectReader object" 2021-01-29 01:52:13 -05:00
Terry Parker dbd05433ec Move reachability checker generation into the ObjectReader object
Reachability checkers are retrieved from RevWalk and ObjectWalk objects:
* RevWalk.createReachabilityChecker()
* ObjectWalk.createObjectReachabilityChecker()

Since RevWalks and ObjectWalks are themselves directly instantiated
in hundreds of places (e.g. UploadPack...) overriding them in a
consistent way requires overloading 100s of methods, which isn't
feasible. Moving reachability checker generation to a more central
place solves that problem.

The ObjectReader object seems a good place from which to get
reachability checkers, because reachability checkers return
information about relationships between objects. ObjectDatabases
delegate many operations to ObjectReaders, and reachability bitmaps
are attached to ObjectReaders.

The Bitmapped and Pedestrian reachability checker objects were
package private in the org.eclipse.jgit.revwalk package. This change
makes them public and moves them to the
org.eclipse.jgit.internal.revwalk package. Corresponding tests are
also moved.

Motivation:
1) Reachability checking algorithms need to scale. One of the
   internal Android repositories has ~2.4 million refs/changes/*
   references, causing bad long tail performance in reachability
   checks.
2) Reachability check performance is impacted by repository
   topography: number of refs, number of objects, amounts of
   related vs. unrelated history.
3) Reachability check performance is also affected by per-branch
   access (Gerrit branch permissions) since different users can
   see different branches.
4) Reachability check performance isn't affected by any state in a
   RevWalk or ObjectWalk.

I don't yet know if a single algorithm will work for all cases in #2
and #3. We may need to evolve the ReachabilityChecker interfaces
over time to solve the Gerrit branch permissions case, or use
Gerrit-specific identity information to solve that in an efficient
way.

This change takes the existing public API and moves it to the
ObjectReader/whole repository level, which is where we can do
consistent customizations for #2 and #3. We intend to upstream the
best of whatever works, but anticipate the need for multiple rounds
of experimentation.

Change-Id: I9185feff43551fb387957c436112d5250486833d
Signed-off-by: Terry Parker <tparker@google.com>
2021-01-28 22:17:26 -08:00
Jonathan Tan c29ec3447d Merge changes I36d9b63e,I8c5db581,I2c02e89c
* changes:
  Compare getting all refs except specific refs with seek and with filter
  Add getsRefsByPrefixWithSkips (excluding prefixes) to ReftableDatabase
  Add seekPastPrefix method to RefCursor
2021-01-27 13:36:43 -05:00
Gal Paikin a6b90b7ec5 Add getsRefsByPrefixWithSkips (excluding prefixes) to ReftableDatabase
We sometimes want to get all the refs except specific prefixes,
similarly to getRefsByPrefix that gets all the refs of a specific
prefix.

We now create a new method that gets all refs matching a prefix except a
set of specific prefixes.

One use-case is for Gerrit to be able to get all the refs except
refs/changes; in Gerrit we often have lots of refs/changes, but very
little other refs. Currently, to get all the refs except refs/changes we
need to get all the refs and then filter the refs/changes, which is very
inefficient. With this method, we can simply skip the unneeded prefix so
that we don't have to go over all the elements.

RefDirectory still uses the inefficient implementation, since there
isn't a simple way to use Refcursor to achieve the efficient
implementation (as done in ReftableDatabase).

Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I8c5db581acdeb6698e3d3a2abde8da32f70c854c
2021-01-27 02:22:45 -05:00
Terry Parker b79882586d Wrap the Files.list returned Stream in a try-with-resources block
Adds a new FileUtils.hasFiles(Path) helper method to correctly handle
the Files.list returned Stream.

These errors were found by compiling the code using JDK11's
javac compiler.

Change-Id: Ie8017fa54eb56afc2e939a2988d8b2c5032cd00f
Signed-off-by: Terry Parker <tparker@google.com>
2021-01-26 16:04:13 -08:00
Gal Paikin 68b95afc70 Add seekPastPrefix method to RefCursor
This method will be used by the follow-up change. This useful if we want
to go over all the changes after a specific ref.

For example, the new method allows us to create a follow-up that would
go over all the refs until we reach a specific ref (e.g refs/changes/),
and then we use seekPastPrefix(refs/changes/) to read the rest of the refs,
thus basically we return all refs except a specific prefix.

When seeking past a prefix, the previous condition that created the
RefCursor still applies. E.g, if the cursor was created by
seekRefsWithPrefix, we can skip some refs but we will not return refs
that are not starting with this prefix.

Signed-off-by: Gal Paikin <paiking@google.com>
Change-Id: I2c02e89c877fe90da8619cb8a4a9a0c865f238ef
2021-01-26 21:47:28 +01:00
Han-Wen Nienhuys b832b068d2 reftable: add random suffix to table names
In some circumstances (eg. compacting a stack that has deletions), the
result may have a {min, max} range that already exists. In these
cases, we would rename onto an already existing file, which does not
work on Windows. By adding a random suffix, we disambiguate the files,
and avoid this failure scenario.

Change-Id: I0273f99bb845cfbdbd8cdd582b55d3c310505d29
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2021-01-25 16:55:39 +01:00
Thomas Wolf 84dbc2d431 TemporaryBuffer: fix toByteArray(limit)
Heap always copied whole blocks, which leads to AIOOBEs. LocalFile
didn't overwrite the method and thus caused NPEs.

Change-Id: Ia37d4a875df9f25d4825e6bc95fed7f0dff42afb
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-22 23:00:01 +01:00
Matthias Sohn 5e2c976233 Merge "[spotbugs]: Fix potential NPE in FileSnapshot constructor" 2021-01-18 02:44:13 -05:00
Thomas Wolf 58f4e6e7f8 TransportHttp: support preemptive Basic authentication
If the caller knows already HTTP Basic authentication will be needed
and if it also already has the username and password, preemptive
authentication is a little bit more efficient since it avoids the
initial 401 response.

Add a setPreemptiveBasicAuthentication(username, password) method
to TransportHttp. Client code could call this for instance in a
TransportConfigCallback. The method throws an IllegalStateException
if it is called after an HTTP request has already been made.

Additionally, a URI can include userinfo. Although it is not
recommended to put passwords in URIs, JGit's URIish and also the
Java URL and URI classes still allow it. The underlying HTTP
connection may omit these fields though. If present, take these
fields as additional source for preemptive Basic authentication if
setPreemptiveBasicAuthentication() has not been called.

No preemptive authentication will be done if the connection is
redirected to a different host.

Add tests.

Bug: 541327
Change-Id: Id00b975e56a15b532de96f7bbce48106d992a22b
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 471ad49546 TransportHttp: shared SSLContext during fetch or push
TransportHttp makes several HTTP requests. The SSLContext and socket
factory must be shared over these requests, otherwise authentication
information may not be propagated correctly from one request to the
next. This is important for authentication mechanisms that rely on
client-side state, like NEGOTIATE (either NTLM, if the underlying HTTP
library supports it, or Kerberos). In particular, SPNEGO cannot
authenticate on a POST request; the authentication must come from the
initial GET request, which implies that the POST request must use the
same SSLContext and socket factory that was used for the GET.

Change the way HTTPS connections are configured. Introduce the concept
of a GitSession, which is a client-side HTTP session over several HTTPS
requests. TransportHttp creates such a session and uses it to configure
all HTTP requests during that session (fetch or push). This gives a way
to abstract away the differences between JDK and Apache HTTP connections
and to configure SSL setup outside.

A GitSession can maintain state and thus give all HTTP requests in a
session the same socket factory.

Introduce an extension interface HttpConnectionFactory2 that adds a
method to obtain a new GitSession. Implement this for both existing
HTTP connection factories. Change TransportHttp to use the new
GitSession to configure HTTP connections.

The old methods for disabling SSL verification still exist to support
possibly external connection and connection factory implementations
that do not make use of the new GitSession yet.

Bug: 535850
Change-Id: Iedf67464e4e353c1883447c13c86b5a838e678f1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:45 +01:00
Thomas Wolf 224aaa0be7 TransportHttp: make the connection factory configurable
Previously, TransportHttp always used the globally set connection
factory. This is problematic if that global factory is changed in
the middle of a fetch or push operation. Initialize the factory to
use in the constructor, then use that factory for all HTTP requests
made through this transport. Provide a setter and a getter for it
so that client code can customize the factory, if needed, in a
TransportConfigCallback.

Once a factory has been used on a TransportHttp instance it cannot
be changed anymore.

Make the global static factory reference volatile.

Change-Id: I7c6ee16680407d3724e901c426db174a3125ba1c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-14 16:23:44 +01:00
Thomas Wolf dd3846513b Tag message must not include the signature
Signatures on tags are just tacked onto the end of the message.
Getting the message must not return the signature. Compare [1]
and [2] in C git, which both drop a signature at the end of an
object body.

[1] https://github.com/git/git/blob/21bf933/builtin/tag.c#L173
[2] https://github.com/git/git/blob/21bf933/ref-filter.c#L1276

Change-Id: Ic8a1062b8bc77f2d7c138c3fe8a7fd13b1253f38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-10 10:19:40 -05:00
Thomas Wolf fb3ae37e26 Protocol V2: don't log spurious ACKs in UploadPack
UploadPack may log ACKs in protocol V2 that it doesn't send (if it
got a "done" from the client), or may log ACKs twice. That makes
packet log analysis difficult.

Add a new constructor to PacketLineOut to omit all logging from an
instance, and use it in UploadPack.

Change-Id: Ic29ef5f9a05cbcf5f4858a4e1b206ef0e6421c65
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-06 12:17:23 +01:00
Thomas Wolf fc9f866a17 Merge "Protocol V2: respect MAX_HAVES only once we got at least one ACK" 2021-01-05 17:59:27 -05:00
Christian Halstrick 5aaaad5cc1 Merge "FileSnapshot: don't try to read file attributes twice" 2021-01-05 02:11:48 -05:00
Thomas Wolf 0d7d98620f Protocol V2: respect MAX_HAVES only once we got at least one ACK
The negotiation in the git protocol contains a cutoff: if the client
has sent more than MAX_HAVES "have" lines without getting an ACK, it
gives up and sends a "done". MAX_HAVES is 256.

However, this cutoff must kick in only if at least one ACK has been
received. Otherwise the client may give up way too early, which makes
the server send all its history. See [1].

This was missed when protocol V2 was implemented for fetching in JGit
in commit 0853a241.

Compare also C git commit 0b07eecf6ed.[2] C git had the same bug.[3][4]

[1] https://github.com/git/git/blob/6c430a647cb9/Documentation/technical/pack-protocol.txt#L385
[2] https://github.com/git/git/commit/0b07eecf6ed
[3] https://lore.kernel.org/git/b7f5bfb9-61fb-2552-4399-b744428728e4@suse.cz/
[4] https://lore.kernel.org/git/20200422084254.GA27502@furthur.local/

Bug: 553083
Change-Id: I1f4e2cc16b5eed6971d981d472329185abb9e4a9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-04 10:00:28 +01:00
Matthias Sohn 74d5a1c172 RepositoryCache: declare schedulerLock final
This fixes errorprone error [SynchronizeOnNonFinalField]: Synchronizing
on non-final fields is not safe: if the field is ever updated, different
threads may end up locking on different objects.

Change-Id: I42fe5bde825151693e2da2d5b6cd6e1d34038dbc
2021-01-03 19:59:35 -05:00
David Ostrovsky d9143287b7 Enable git wire protocol version 2 on server side per default
Bug: 563145
Change-Id: Id5030c2b85466da0a8ccf3d78ae78df16d64ffc5
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2021-01-03 16:25:01 +01:00
Matthias Sohn ef04b3b883 [spotbugs]: Fix potential NPE in FileSnapshot constructor
File#getParent can return null which caused this spotbugs warning.

FS.FileStoreAttributes#get already gets the parent directory if the
passed File is not a directory and checks for null. Hence there is no
need to get the parent directory in the FileSnapshot constructor.

Change-Id: I77f71503cffb05970ab8d9ba55b69c96c53098b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2021-01-03 16:08:59 +01:00
Matthias Sohn 877ce01d29 FileSnapshot: don't try to read file attributes twice
If file doesn't exist set state to MISSING_FILE immediately. Doing that
by calling File#lastModified and File#length effectively does the same
since they set the value to 0 if the file doesn't exist.

Log an error if a different exception than NoSuchFileException is
caught.

Change-Id: I0d4396b9f80446692a088d17522d64f735ce6708
2021-01-03 15:33:36 +01:00
Thomas Wolf 0853a2410f Client-side protocol V2 support for fetching
Make all transports request protocol V2 when fetching. Depending on
the transport, set the GIT_PROTOCOL environment variable (file and
ssh), pass the Git-Protocol header (http), or set the hidden
"\0version=2\0" (git anon). We'll fall back to V0 if the server
doesn't reply with a version 2 answer.

A user can control which protocol the client requests via the git
config protocol.version; if not set, JGit requests protocol V2 for
fetching. Pushing always uses protocol V0 still.

In the API, there is only a new Transport.openFetch() version that
takes a collection of RefSpecs plus additional patterns to construct
the Ref prefixes for the "ls-refs" command in protocol V2. If none
are given, the server will still advertise all refs, even in protocol
V2.

BasePackConnection.readAdvertisedRefs() handles falling back to
protocol V0. It newly returns true if V0 was used and the advertised
refs were read, and false if V2 is used and an explicit "ls-refs" is
needed. (This can't be done transparently inside readAdvertisedRefs()
because a "stateless RPC" transport like TransportHttp may need to
open a new connection for writing.)

BasePackFetchConnection implements the changes needed for the protocol
V2 "fetch" command (stateless protocol, simplified ACK handling,
delimiters, section headers).

In TransportHttp, change readSmartHeaders() to also recognize the
"version 2" packet line as a valid smart server indication.

Adapt tests, and run all the HTTP tests not only with both HTTP
connection factories (JDK and Apache HttpClient) but also with both
protocol V0 and V2. The SSH tests are much slower and much more
focused on the SSH protocol and SSH key handling. Factor out two
very simple cloning and pulling tests and make those run with
protocol V2.

Bug: 553083
Change-Id: I357c7f5daa7efb2872f1c64ee6f6d54229031ae1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 21:22:30 +01:00
Thomas Wolf 0f442d7083 Use Map interface instead of ConcurrentHashMap class
On Android, the co-variant override of ConcurrentHashMap.keySet()
introduced in Java 8 was undone. [1] If compiled Java code calls that
co-variant override directly, one gets a NoSuchMethodError exception
at run-time on Android.

Making the code call that method via Map.keySet() side-steps this
problem.

This is similar to bug 496262, where the same problem cropped up when
compiling with Java 8 against a Java 7 target, but here we cannot use
bootclasspath. We build against Java 8, not against the Android version
of it.

Recent Android versions should have some bytecode "magic" that adds the
co-variant override in bytecode (see the commit referenced in [1]), but
on older Android version this problem may still occur. (Or perhaps the
"magic" is ineffective...) There are two pull requests on Github for
this problem, both from 2020, [2][3] while the Android commit [1] is
from March 2018. Apparently people still occasionally run into this
problem in the wild.

[1] 0e8b937ded/ojluni/src/main/java/java/util/concurrent/ConcurrentHashMap.java (1244)
[2] https://github.com/eclipse/jgit/pull/104
[3] https://github.com/eclipse/jgit/pull/100

Change-Id: I7c07e0cc59871cb7fe60795e22867827fa9c2458
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2021-01-01 12:49:05 -05:00
Thomas Wolf 5b1a6e0e38 Fix NPE in DirCacheCheckout
If a file exists in head, merge, and the working tree, but not in
the index, and we're doing a force checkout, the checkout must be
an "update", not a "keep".

This is a follow-up on If3a9b9e60064459d187c7db04eb4471a72c6cece.

Bug: 569962
Change-Id: I59a7ac41898ddc1dd90e86b09b621a41fdf45667
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-30 10:51:14 +01:00
Christian Halstrick 086f474054 Merge "Added check for null on DirCacheEntry in checkoutEntry method" 2020-12-30 03:05:00 -05:00
Matthias Sohn 3482e50134 [spotbugs] Fix incorrect lazy initialization in SystemReader
This fixes two warnings of type LI_LAZY_INIT_STATIC.

Change-Id: I26a7a48aed9d0a0547e908a56b7014a7620fadd8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:01 +01:00
Matthias Sohn 13e0ffbd3d [spotbugs] Don't use class from java.util.concurrent for locking
Use a dedicated Lock object to lock the scheduler in
RepositoryCache#configureEviction to fix spotbugs warning
JLM_JSR166_UTILCONCURRENT_MONITORENTER.

Change-Id: I003dcf0ed1a0a3f4eea5d8a2f51a07473d28a928
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:01 +01:00
Matthias Sohn bceb202319 [spotbugs] Fix potential NPE in FS.FileStoreAttributes#get
Path#getParent can return null, return fallback filestore attributes in
that case.

Change-Id: Ic09484d527bc87b27964b625e07373b82412f2da
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:53:00 +01:00
Matthias Sohn 0132666d5a [spotbugs] Fix FileReftableStack#equals to check for null
This fixes spotbugs warning NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT.

This implementation violated the contract defined by
java.lang.Object.equals() because it did not check for null being passed
as the argument. All equals() methods should return false if passed a
null value.

Change-Id: I607f6979613d390aae2f3546b587f63133d6d73c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn fa0e77e8f9 [spotbugs] FileReftableDatabase: extract lock to local variable
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#compactFully.

Change-Id: I370578ad9a027c5c9709d60a1dfafdac0cfca908
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 39cbc574d1 [spotbugs] DfsReftableDatabase: extract lock to local variable
This fixes UL_UNRELEASED_LOCK_EXCEPTION_PATH raised by spotbugs in
#DfsReftableDatabase and #clearCache.

Change-Id: Ifd3189288d2a8e64139c02cd105eb335fa2f68cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 1fd0a49ce0 [spotbugs] Silence NP_BOOLEAN_RETURN_NULL in IgnoreNode#checkIgnored
Also mark the return value @Nullable to enable null analysis in Eclipse.

Change-Id: I5b286d657d432f4b32afd4dd370f76892b115422
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:59 +01:00
Matthias Sohn 65ac9f2c2f Revert "Remove unused API problem filters"
This reverts commit 022b02dea1.

Removing this API warning filter was wrong since we intentionally
removed the config constant CONFIG_REFSTORAGE_REFTREE.

Change-Id: Ie068e539157cc2312efc8a07feabeb0dd2f75096
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-22 10:52:10 +01:00
Matthias Sohn 022b02dea1 Remove unused API problem filters
Change-Id: Id533cf598cd37c277b4de06fdd8ae74cfeede4d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 18:42:00 +01:00
Matthias Sohn 23bc9dc71d [spotbugs] Fix potential NPE in FS#write
Path#getParent can return null.

Change-Id: I01f13ac426dda4c007cc5caab546a0c9be62ce76
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 18:42:00 +01:00
Matthias Sohn 90b046e7ab [spotbugs] Fix potential NPE in WorkingTreeIterator#isModified
File#list can return null. Fix the potential NPE by using Files#list
which is also faster since it retrieves directory entries lazily while
File#list retrieves them eagerly.

Change-Id: Idf4bda398861c647587e357326b8bc8b587a2584
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 18:42:00 +01:00
Matthias Sohn a2bb540f29 [spotbugs] Fix potential NPE in FileRepository#convertToReftable
File#listFiles can return null. Use Files#list which does not return
null and should be faster since it's returning directory entries lazily
while File#listFiles fetches them eagerly.

Change-Id: I3bfe2a52278244fc469143692c06b05d9af0d0d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn 15998622fa [spotbugs] silence warnings for intended use of == to compare strings
Change-Id: Ib6967ad4deb5cf233d1f1d714cd094da5fad48e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn a56624349e [spotbugs] silence short-circuit warning in PackOutputStream#write
Change-Id: I47b7a7991afae0dd1e678bd5e1f8e81599791e5f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn 5b16f32de0 FetchProcess#execute: fix formatting
Change-Id: I133af64e6b165bdc58b8d7c805f0c68b6919baf2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn c8c0556f76 [errorprone] FetchProcess: ensure exception isn't suppressed
If TransportException is thrown in the finally block of execute()
ensure that the exception handled in the previous catch block isn't
suppressed.

Change-Id: I670acdfb4d36e7a419a9a79ae9faab2e085a43ee
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn 9680407e45 [errorprone] WalkFetchConnection: ensure exception isn't suppressed
If TransportException is thrown in the finally block of
downloadPackedObject() ensure that the exception handled in the previous
catch block isn't suppressed.

Change-Id: I23982a2b215e38f681cc1719788985e60232699a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-17 17:18:11 +01:00
Matthias Sohn 5dfe4c096e Remove unused API problem filters
Change-Id: I982f21dfaec4e35818eca7a4793e24a2cec4e467
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 02:00:35 +01:00
Han-Wen Nienhuys 7386f73bb0 Remove reftree and ketch
This was experimental code and never used in production.

Change-Id: Ia3da7f2b82d9e365cec2ccf9397cbc47439cd150
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 02:00:35 +01:00
Matthias Sohn 3fc8da3a3c LooseObjects: fix formatting
Change-Id: I992e5066d1dc4603e5bae991b26605d319f5cdc2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 02:00:17 +01:00
Martin Fick ecad2dc0a6 Split out loose object handling from ObjectDirectory
The ObjectDirectory class manages the interactions for the entire object
database, this includes loose objects, packfiles, alternates, and
shallow commits. To help reduce the complexity of this class, abstract
some of the loose object specific details into a class which understands
just this, leaving the ObjectDirectory to focus more on the interactions
between the different mechanisms.

Change-Id: I39f3a74d6308f042a2a2baa57769f4acde5ba5e0
Signed-off-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-15 00:18:32 +01:00
Matthias Sohn 415788df28 Merge branch 'stable-5.9'
* stable-5.9:
  Fix stamping to produce stable file timestamps

Change-Id: Icd092cd9b883556fcbd115c17346a9d88dc172ce
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-14 16:03:35 +01:00
David Ostrovsky 5d925ecbb3 Fix stamping to produce stable file timestamps
Change-Id: I628ab5feb4a70f81ec832f1b81d1ad3a9caca615
2020-12-14 15:45:29 +01:00
Matthias Sohn ff0590812a PackDirectory: fix formatting
Change-Id: I10586981323529c7e9041110ebb58033e7180194
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-12 23:32:09 +01:00
Martin Fick a84a2d22ab Split out packfile handling from ObjectDirectory
The ObjectDirectory class manages the interactions for the entire object
database, this includes loose objects, packfiles, alternates, and
shallow commits. To help reduce the complexity of this class, abstract
some of the packfile specific details into a class which understands
just this, leaving the ObjectDirectory to focus more on the interactions
between the different mechanisms.

Change-Id: I5cc87b964434b0afa860b3fe23867a77b3c3a4f2
Signed-off-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-12 23:15:21 +01:00
Thomas Wolf e3ac56e2d0 TagCommand: propagate NO_CHANGE information
Some clients may wish to allow NO_CHANGE lightweight tag updates
without setting the force flag. (For instance EGit does so.)
Command-line git does not allow this.

Propagate the RefUpdate result via the RefAlreadyExistsException.
That way a client has the possibility to catch it and check the
failure reason without having to parse the exception message, and
take appropriate action, like ignoring the exception on NO_CHANGE.

Change-Id: I60e7a15a3c309db4106cab87847a19b6d24866f6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-08 22:20:46 +01:00
Thomas Wolf 29e1270768 TagCommand: make -f work with lightweight tags for NO_CHANGE
JGit treated a NO_CHANGE RefUpdate as an error in all cases. But when
updating a lightweight tag, this is a successful result if -f was
specified.

Change-Id: Iddfa6d6a6dc8bf8fed81138a008ebc32d5f960bd
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-08 22:20:45 +01:00
Thomas Wolf 41b9159795 TagCommand: support signing annotated tags
Add the two config constants from C git that can switch on signing
of annotated tags. Add them to the GpgConfig, and implement actually
signing a tag in TagCommand.

The interactions between command line options for "git tag" and config
options is a bit murky in C git. There are two config settings for it:

* tag.gpgSign is the main option, if set to true, it kicks in if
  neither -s nor -u are given on the command line.
* tag.forceSignAnnotated signs only tags created via "git tag -m",
  but only if command-line option "-a" is not present. It applies
  even if tag.gpgSign is set explicitly to false.

Giving -s or -u on the command line also forces an annotated tag
since lightweight tags cannot be signed.

Bug: 386908
Change-Id: Ic8a1a44b5f12f47d5cdf3aae2456c1f6ca9ef057
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-07 09:33:57 +01:00
Thomas Wolf 5abd8a4feb Enable GpgSigner to also sign tags
Factor out a common ObjectBuilder as super class of CommitBuilder
and TagBuilder, and make the GpgSigner work on ObjectBuilder.

In order not to break API, add the new method for signing an
ObjectBuilder in a new interface GpgObjectSigner.

The signature for a tag is just tacked onto the end of the tag
message. The message of a signed tag must end in LF.

Bug: 386908
Change-Id: I5e021e3c927f4051825cd7355b129113b949455e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-12-07 09:04:33 +01:00
Tudor Matrescu aa3a1ecd13 Added check for null on DirCacheEntry in checkoutEntry method
Observed the error when trying to force checkout from a branch
that had no changes on it. When the 'keep()' method from 'DirCacheCheckout'
method was called the 'DirCacheEntry e' was null and was passed like
this to the 'checkoutEntry()' method where the 'getObjectId()' is
being called on the 'e' object

Change-Id: If3a9b9e60064459d187c7db04eb4471a72c6cece
2020-12-03 02:59:10 -05:00
Matthias Sohn 9034c7d423 Prepare 5.11.0-SNAPSHOT builds
Change-Id: I91e5532526775191fbd34f81e2ef777cba605e3b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:57:16 +01:00
Matthias Sohn 205ee010f6 Merge branch 'stable-5.9'
* stable-5.9:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: I3cf6c71e8310c1d7ef333d4a9c23fa41f45118c4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:34:14 +01:00
Matthias Sohn 9a1065afec Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: I08caea979ac4c1298b453e6e5558bccb86fb0181
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:31:11 +01:00
Matthias Sohn 8f422e9a9a Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: I638ee6409f25ac8bcb2e365b1c37939b520b289f
2020-12-02 15:28:33 +01:00
Matthias Sohn f621c31f4c Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: If5f001c414e677bb2bac59714421f0191b23ea1f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:26:00 +01:00
Matthias Sohn c9d871f15d Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: I986029816ef66fbfae1a59bd97179392320a485c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:21:32 +01:00
Matthias Sohn e62a93955b Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: Iede8686198332d6271771bef6eb00c25f1103979
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 15:16:11 +01:00
Matthias Sohn e8948056fa Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.10-SNAPSHOT builds
  JGit v5.3.9.202012012026-r
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: I319b54ceffe095add60420c6ae83eac0ba9c14b6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 14:26:37 +01:00
Matthias Sohn 0a63e045c9 Prepare 5.3.10-SNAPSHOT builds
Change-Id: I6f131ad04574bd0d569ae6a59c29ea987be0efb2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 13:09:57 +01:00
Matthias Sohn aa5d1e425c JGit v5.3.9.202012012026-r
Change-Id: I75e03ef8630d7a369e97be0f797253b968575354
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 02:27:44 +01:00
Matthias Sohn 5bb71dd2d4 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: If1d8034b5e0cbc004a11a31b228cb5732efb390d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 02:19:11 +01:00
Matthias Sohn babc56ef0f Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.16-SNAPSHOT builds
  JGit v5.1.15.202012011955-r
  Fix PackInvalidException when fetch and repack run concurrently

Change-Id: Id83e29e567646a3945a5b817860ea8f7c3e6e5cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 02:17:41 +01:00
Matthias Sohn 5e0cfce5ad Prepare 5.1.16-SNAPSHOT builds
Change-Id: I50e59e1e73a92fa4fe366398fb8141f5e2e289c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 02:02:48 +01:00
Matthias Sohn f72a001250 JGit v5.1.15.202012011955-r
Change-Id: Icb4f04a40ab366cbacbb3fdf0db1748f27277fda
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 01:53:39 +01:00
Petr Hrebejk 2fbbd6d890 Fix PackInvalidException when fetch and repack run concurrently
We are running several servers with jGit. We need to run repack from
time to time to keep the repos performant. I.e. after push we test how
many small packs are in the repo and when a threshold is reached we run
the repack.

After upgrading jGit version we've found that if someone does the clone
at the time repack is running the clone sometimes (not always) fails
because the repack removes .pack file used by the clone. Server
exception and client error attached.

I've tracked down the cause and it seems to be introduced between jGit
5.2 (which we upgraded from) and 5.3 and being caused by this commit:
Move throw of PackInvalidException outside the catch -
afef866a44

The problem is that when the throw was inside of the try block the last
catch block catched the exception and called openFailed(false) method.
It is true that it called it with invalidate = false, which is wrong.
The real problem though is that with the throw outside of the try block
the openFail is not called at all and the fields activeWindows and
activeCopyRawData are not set to 0. Which affects the later called tests
like: if (++activeCopyRawData == 1 && activeWindows == 0).

The fix for this is relatively simple keeping the throw outside of the
try block and still having the invalid field set to true. I did
exhaustive testing of the change running concurrent clones and pushes
indefinitely and with the patch applied it never fails while without the
patch it takes relatively short to get the error.

See: https://www.eclipse.org/lists/jgit-dev/msg04014.html

Bug: 569349
Change-Id: I9dbf8801c8d3131955ad7124f42b62095d96da54
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-12-02 01:43:25 +01:00
Matthias Sohn 180dfdefa4 Merge branch 'stable-5.9'
* stable-5.9:
  Add constants for parsing git wire protocol version

Change-Id: I1e4174ae8bbefcb71a191dcbf32027d4b35c2b37
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-30 16:54:02 +01:00
David Ostrovsky 23389a6323 Add constants for parsing git wire protocol version
This would allow other JGit users to access and reuse the constants.

Change-Id: I1608802f45586af5f8582afa592e26679e9cebe3
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-29 22:46:51 +01:00
Matthias Sohn 4f2065d145 Merge branch 'stable-5.10'
* stable-5.10:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  PacketLineIn: ensure that END != DELIM
  Update API warning filters
  Remove unused imports

Change-Id: I25f50c3807a4e6b22a264320ea7ed3758e2a75ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 10:59:01 +01:00
Matthias Sohn d4fe93f55b Merge branch 'stable-5.9' into stable-5.10
* stable-5.9:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  PacketLineIn: ensure that END != DELIM
  Update API warning filters
  Remove unused imports

Change-Id: Icf415ac5bab2f69f71189c942424ee69f8a64d4e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 10:10:44 +01:00
Matthias Sohn 5cd485e5dd Merge branch 'stable-5.8' into stable-5.9
* stable-5.8:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  PacketLineIn: ensure that END != DELIM
  Update API warning filters
  Remove unused imports

Change-Id: I70b399eb3df02aa2fb112e86e844cf5a0daa5515
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 09:38:42 +01:00
Matthias Sohn d1801402fe Merge branch 'stable-5.7' into stable-5.8
* stable-5.7:
  Remove unused imports
  Silence API warnings
  Remove erraneously merged source features
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I9b94938f5c09bd726e8e368c98c56da8280fd0b2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 09:07:30 +01:00
Matthias Sohn f65271f843 Silence API warnings
Change-Id: I5de476935e79f8f0beefc27885cbc3ffe31c0aed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 02:26:23 +01:00
Matthias Sohn 99f5329c38 Merge branch 'stable-5.6' into stable-5.7
* stable-5.6:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Ie24d381f295cccfb99068c7ed5817179da29c1db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 01:03:02 +01:00
Matthias Sohn 480b00f1c7 Merge branch 'stable-5.5' into stable-5.6
* stable-5.5:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I00c8ddad0059a20a978743dfb7ad1b513dc7fef6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 01:01:31 +01:00
Matthias Sohn 95846863cb Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Icc34f809a3bb019d8d640b9bdb71363e617942e2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 00:40:26 +01:00
Matthias Sohn a74c0cd333 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.9-SNAPSHOT builds
  JGit v5.3.8.202011260953-r
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I82576ad0f61cf3ff11b54691b32666c61401ad9b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-27 00:29:53 +01:00
Lee Worrall 9ebbfe93bb Add support for reading symrefs from pack capabilities
A SymbolicRef is added to the advertised refs for any symref in
capabilities whose target is an advertised ref; this may replace an
existing entry, such as HEAD.

When cloning, if any advertised HEAD is symbolic then use the target
rather than looking for an advertised ref with a matching objectId.

Add --symref option to LsRemote command.

Bug: 514052
Change-Id: Idfb48e6f6e8dcfe57a6896883fe6d84d533aa9d0
Signed-off-by: Lee Worrall <worrall.la@gmail.com>
2020-11-26 21:14:08 +01:00
Matthias Sohn fdcdfc37a3 Prepare 5.3.9-SNAPSHOT builds
Change-Id: I69f181453c79cef2b4f43fac38d9836917cdc973
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 16:30:32 +01:00
Matthias Sohn 0e9cf495d9 JGit v5.3.8.202011260953-r
Change-Id: I7b128942ef224335f415f867c2d9d5da7498ed8b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 15:52:53 +01:00
Matthias Sohn 3f85d3b75c Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: Id9386846a202b5ae98dd602744963f8897ddaa8c
2020-11-26 15:37:19 +01:00
Matthias Sohn 65598b9516 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.15-SNAPSHOT builds
  JGit v5.1.14.202011251942-r
  GC#deleteOrphans: log warning for deleted orphaned files
  GC#deleteOrphans: handle failure to list files in pack directory
  Ensure that GC#deleteOrphans respects pack lock
  Update API warning filters
  Remove unused imports

Change-Id: I91cfe2820c40d2d773cbf018cc2a6c36b062801e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 15:34:24 +01:00
Matthias Sohn 8c2dc8b6c0 Prepare 5.1.15-SNAPSHOT builds
Change-Id: I70246b66c76e865aef4e3adada3a507750ca7c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 15:02:01 +01:00
Matthias Sohn dac540a37f JGit v5.1.14.202011251942-r
Change-Id: Ibe124988be39feaa029c3770777126dd87b18abc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 01:42:17 +01:00
Matthias Sohn 0f1735de0b GC#deleteOrphans: log warning for deleted orphaned files
Change-Id: Ie245bf5c8c924dfb1f0f40b8bcdcb1e6f5815526
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 01:34:42 +01:00
Matthias Sohn 3996408cde GC#deleteOrphans: handle failure to list files in pack directory
- log an error
- either there is no list or it is incomplete hence return immediately

Change-Id: Ieee5378ca06304056b9ccc30c1acd5f52360052d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 01:17:49 +01:00
Matthias Sohn fde7a271a4 Ensure that GC#deleteOrphans respects pack lock
If pack or index files are guarded by a pack lock (.keep file)
deleteOrphans() should not touch the respective files protected by the
lock file. Otherwise it may interfere with PackInserter concurrently
inserting a new pack file and its index.

The problem was caused by the following race.

All mentioned files are located in  "objects/pack/".
File endings relevant in "pack" dir:
  .pack
  .keep
  .idx
  .bitmap

When ReceivePack receives a pack file it executes the following steps:

ReceivePack.service():
  receivePackAndCheckConnectivity():
    receivePack():
      receive the pack
      parse the pack, returns packLock (.keep file)
      PackInserter.flush():
        write tmpPck file:  "insert_<random>.pack"
        write tmpIdx file:  "insert_<random>.idx"
        real pack name: "pack-<SHA1>.pack"
        real index name: "pack-<SHA1>.idx"
        atomic rename tmpPack to realPack
        atomic rename tmpIdx to tmpIdx
  execute commands
  unlock pack by removing .keep file
  trigger auto gc if enabled

When PackInserter.flush() renames the temporary pack to the final
"pack-xxx.pack" file  the temporary pack index file "insert_xxx.idx"
has no matching .pack file with the same base name for a short interval.
If deleteOrphans() ran during that interval it deduced the pack index
file was orphaned. Subsequently the missing pack index caused
MissingObjectExceptions since objects contained in the pack couldn't be
looked up anymore.

Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=13544
Change-Id: I559c81e4b1d7c487f92a751bd78b987d32c98719
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-26 01:17:47 +01:00
Matthias Sohn 63f16c53c8 Prepare 5.10.0-SNAPSHOT builds
Change-Id: Iabc845edf565ac706344f68f854384faa63b6f43
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-25 22:30:34 +01:00
Matthias Sohn 2486be2706 JGit v5.10.0.202011251205-m3
Change-Id: Id58d8eb49f550e62b04053a71366f7eabd9f9a53
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2020-11-25 18:05:35 +01:00
Thomas Wolf 9f3616dcb4 PacketLineIn: ensure that END != DELIM
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.

Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
2020-11-20 23:30:24 +01:00
Thomas Wolf e2663a8b85 PacketLineIn: ensure that END != DELIM
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.

Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
(cherry picked from commit 7da0f0a8f3)
2020-11-19 21:07:25 +01:00
Thomas Wolf 7da0f0a8f3 PacketLineIn: ensure that END != DELIM
Just allocate the two string objects directly. The previously used
new StringBuilder(0).toString() returns the same object for both END
and DELIM when run on Java 15, which breaks the wire protocol since
then END and DELIM cannot be distinguished anymore.

Bug: 568950
Change-Id: I9d54d7bf484948c24b51a094256bd9d38b085f35
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-11-19 13:23:05 +01:00
Thomas Wolf e84881ea6b Allow to resolve a conflict by checking out a file
DirCacheEditor unconditionally applied a PathEdit to all stages in the
index. This gives wrong results if one wants to check out a file from
some commit to resolve a conflict: JGit would update the working tree
file multiple times (once per stage), and set all stages to point to
the checked-out blob.

C git replaces the stages by the entry for the checked-out file.

To support this, add a DirCacheEntry.setStage() method so that
CheckoutCommand can force the stage to zero. In DirCacheEditor, keep
only the zero stage if the PathEdit re-set the stage.

Bug: 568038
Change-Id: Ic7c635bb5aaa06ffaaeed50bc5e45702c56fc6d1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2020-11-17 14:27:28 +01:00