Currently when a GPG key is looked up using a user identity the first
key from the keyring that has this user identity is returned.
The code was changed to instead return the first signing [S] key in this
keyring and only return the master key if no such signing key was found.
If the master key also does not have the signing flag set null is
returned instead.
Bug: 552288
Change-Id: I194862991d13c2c7ff34a60a54a227167f88f53b
Signed-off-by: Roan Hofland <roan.hofland@hotmail.com>
Currently when a subkey is configured for signing via the git
user.signingkey configuration option the first key from the keyring for
this subkey would be returned for use (master key). The code has been
changed to return the requested key from the keyring instead.
Bug: 552288
Change-Id: I1c1cdf64c1667316a274ff9d829fc2b563797f2a
Signed-off-by: Roan Hofland <roan.hofland@hotmail.com>
Support the core.hooksPath git config. This can be an absolute or
relative path of a directory where to find git hooks; a relative
path is resolved relative to the directory the hook will run in.
Bug: 500266
Change-Id: I671999a6386a837e897c31718583c91d8035f3ba
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Since [1] the gerrit project includes jgit as a submodule, and has this
warning enabled, resulting in 100s of warnings in the console.
Also enable the warning here, and fix them.
At the same time, add missing braces around adjacent and nearby one-line
blocks.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/227897
Change-Id: I81df3fc7ed6eedf6874ce1a3bedfa727a1897e4c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This introduces ReftableBatchRefUpdate and ReftableDatabase, as
generic classes, with some code moved to DfsReftableBatchRefUpdate and
DfsReftableDatabase.
Clarify thread-safety requirements by asserting locked status in
accessors, and acquiring locks in callers. This does not fix threading
problems, because ReftableBatchRefUpdate already wraps the whole
transaction in a lock.
This also fixes a number of bugs in ReftableBatchRefUpdate:
* non-atomic updates should not bail on first failure
* isNameConflicting should also check for conflicts between names that
are added and removed in the BatchRefUpdate.
Change-Id: I5ec91173ea9a0aa19da444c8c0b2e0f4e8f88798
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
On changing a ref, the old SHA1 is not updated in the object => ref
mapping. This means search by object ID may still turn up a ref from
deeper within the stack. To fix this, check all refs produced by the
merged iterator against the merged reftables.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I41e9cd395b0608eedeeaead0a9fd997238d747c9
The flag enabling sideband-all is used in two places: in UploadPack
for advertisement and in the protocol parser to read it from the
request.
This leds to problems in distributed deployments where the two requests of
a fetch can go to different servers with different configurations.
Use the existing allowsidebandall to accept the sideband-all request
(and respond to it) and introduce a new "advertisesidebandall" to toggle
the advertising of the feature.
Change-Id: I892d541bc3f321606c89bad1d333b079dce6b5fa
Signed-off-by: Ivan Frade <ifrade@google.com>
If the SubModuleWalk isn't closed its TreeWalk's ObjectReader won't
be closed. Re-loading the DirCache during an IndexDiff is not only
inefficient but could also give strange results if an external
process had modified the index in the meantime: file diffs would
be based on a "before" state, but submodule diffs on an "after"
state.
Change-Id: Iab948c08ac342138b37263c9028d80b84101f6d6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add to statistics the amount and size of packfiles offloaded to HTTP
download.
Change-Id: I895a7219ecac2794368bfc4fdfae74c1238deed9
Signed-off-by: Ivan Frade <ifrade@google.com>
The object identifying packfiles to send them via packfile-uri contains
only the uri and the hash. This is the information that goes through the
wire. It would be useful to know also the size of those packfile, for
example to track how many bytes have been offloaded to HTTP.
Add size field the CachedPackUriProvider.PackInfo object.
Change-Id: If6b921b48a4764d936141c777879b148cc80bbd3
Signed-off-by: Ivan Frade <ifrade@google.com>
* changes:
UploadPack: Create a method that propagates an exception as-is
UploadPack: Consolidate the sideband handling code to one place
UploadPack: Introduce ErrorWriter
Exception handling can be isolated from UploadPack. This makes it
possible to make the exception handler pluggable.
Change-Id: Ieebbd6711963c7f2e47a98783b4ad815793721c7
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
This consolidates the sideband stream creation code and the error
handling code for the sideband-allowed part in the Git protocol to one
place.
Change-Id: I0e3e94564f50d1be32006f9d8bcd1ef1ce6bf07e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
ErrorWriter writes an error message to the user. The implementation is
swapped once it detects that the client supports sideband. By default it
uses the protocol level ERR packet, which was introduced recently.
In total the error output is done in two different places;
UploadPack#upload and UploadPack#sendPack. These will be consolidated in
the next change.
Change-Id: Ia8d72e31170bbeafc8ffa8ddb92702196af8a587
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Also correctly parse the "always" value (allowed in canonical git
since git 2.12.0[1]). Adapt the ReflogWriter.
[1] https://github.com/git/git/commit/341fb2862
Bug: 551664
Change-Id: I051c76ca355a2ac8d6092de65f44b18bf9aeb125
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.5:
Prepare 5.5.2-SNAPSHOT builds
JGit v5.5.1.201910021850-r
Prepare 5.3.7-SNAPSHOT builds
JGit v5.3.6.201910020505-r
Prepare 5.1.13-SNAPSHOT builds
JGit v5.1.12.201910011832-r
Do not rely on ArrayIndexOutOfBoundsException to detect end of input
Change-Id: Iabebdc4786b52ec678caa798975428fda4ad7bd7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.4:
Prepare 5.3.7-SNAPSHOT builds
JGit v5.3.6.201910020505-r
Prepare 5.1.13-SNAPSHOT builds
JGit v5.1.12.201910011832-r
Do not rely on ArrayIndexOutOfBoundsException to detect end of input
Change-Id: Ia1070fd170651ce827bc6b876c6764a44ffe60eb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Prepare 5.3.7-SNAPSHOT builds
JGit v5.3.6.201910020505-r
Prepare 5.1.13-SNAPSHOT builds
JGit v5.1.12.201910011832-r
Do not rely on ArrayIndexOutOfBoundsException to detect end of input
Change-Id: Ia621d06a9489ee276c793de9dd4a77f4ff19e2ac
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.1.13-SNAPSHOT builds
JGit v5.1.12.201910011832-r
Do not rely on ArrayIndexOutOfBoundsException to detect end of input
Change-Id: I3f11a83e177daefa0a2e91173f70f9547067f713
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
# By Matthias Sohn (2) and Saša Živkov (1)
* stable-5.1:
Prepare 5.1.13-SNAPSHOT builds
JGit v5.1.12.201910011832-r
Do not rely on ArrayIndexOutOfBoundsException to detect end of input
Change-Id: Iaae4b171eaa0081f9142489de8df94ab455d65f7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
MergedReftable is not used as an AutoCloseable, because closing tables
is currently handled by DfsReftableStack#close.
Encode that a MergedReftable is a list of ReftableReaders. The previous
code suggested that we could form nested trees of MergedReftables,
which is not how we use reftables.
Change-Id: Icbe2fee8a5a12373f45fc5f97d8b1a2b14231c96
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
In the Config#StringReader we relied on ArrayIndexOutOfBoundsException
to detect the end of the input. Creation of exception with (deep) stack
trace can significantly degrade performance in case when we read
thousands of config files, like in the case when Gerrit reads all
external ids from the NoteDb.
Use the buf.length to detect the end of the input.
Change-Id: I12266f25751373a870ce3fa623cf2a95d882d521
Otherwise the paths modified by a cherry-pick with conflicts won't be
reported as modified via WorkingTreeModifiedEvents.
Change-Id: I875b67c0d2f68efdf90a9c32b80a2e074ed3570d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This makes the intended use of the classes more clear. It also
simplifies generic functions that write reftables: they only need a
ReftableWriter as argument, as the stream is carried within the
ReftableWriter.
Change-Id: Idbb06f89ae33100f0c0b562cc38e5b3b026d5181
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The functionality in ReftableStack is specific to DFS.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: If6003d104b1ecb0f3ca7e9c3815b233fa0abf077
This makes maxUpdateIndex() available in MergedReftable, so we can
know generically at which index to create the next reftable in a
stack.
Change-Id: Ia2314bc57c8b5dd7e69d5e61096fdce1d35abd11
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Older JGit stored only milliseconds timestamps in the index. Newer
JGit may get finer timestamps from the file system. This leads to
slow index diffs when a new JGit runs against an index produced
by older JGit because many timestamps will differ and JGit will
then do many content checks. See [1].
Handle this migration case by only comparing milliseconds if the
index entry has only millisecond precision.
The inverse may also occur; also compare only milliseconds if the
file timestamp has only millisecond precision.
Do the same also for microsecond resolution. On Windows, NTFS may
provide 100ns resolution and may be used by external programs writing
the index, but Java's WindowsFileAttributes may provide only
microseconds.
File timestamp precision in Java depends not only on the Java APIs
used by different JGit versions but may also change when running the
same Java code on different VMs. And of course the resolution may
vary among operating and file systems. Moreover, timestamp precision
in the index depends on the program that wrote the index. Canonical
git may use a different resolution, maybe even different between git
versions.
[1] https://www.eclipse.org/forums/index.php/t/1100344/
Change-Id: Idfd08606c883cb98787b2138f9baf0cc89a57b56
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The removed code was trying to avoid mistakenly reporting differences
when core.autocrlf was set to "input" but a file had already been
committed with CR-LF. It did that by running the blob from the cache
through a CRLF-to-LF filter because older JGit would also run the file
from the working tree through such a filter.
The real fix for this case was done in commit 60cf85a. Since then files
are not normalized if they have already been committed with CR-LF and
this old fix attempt from bug 372834 is no longer needed.
Change-Id: Ib4facc153d81325cb48b4ee956a596b423f36241
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This removes one use of DFS specific code in this class.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I3ef6a4b98357cc6dc480892244ddc51d2fd751a2
This lets us write reftables generically with functions that take
just ReftableWriter argument
Change-Id: I7285951f62f9bd4c78e8f0de194c077d51fa4e51
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
allRefs determined the end of the ref block without accounting for
index or log blocks. This could cause other blocks to be interpreted
as ref blocks, leading to "invalid block" error messages.
Change-Id: I7b9323e7d5e0e7d64535b3ec1efd576aed1e9870
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
If CheckStat is MINIMAL or timestamps have no nanosecond part
WorkingTreeIterator.compareMetaData only checks the second part of
timestamps and ignores nanoseconds which may have ended up in the index
by using native git.
If
fileLastModified.getEpochSecond() == cacheLastModified.getEpochSecond()
we currently proceed comparing fileLastModified and cacheLastModified
with full precision which is wrong since we determined that we detected
reduced timestamp resolution.
Fix this and also handle smudged index entries for CheckStat.MINIMAL.
Change-Id: I6149885903ac63d79b42d234cc02aa4e19578f3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.5:
bazel: fix running http tests
Set parameter name in parameterized http tests
Format BUILD files with buildifier
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Idb0848dd33a76328b24908dc86db335cca742a1c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.4:
Format BUILD files with buildifier
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Idf3abd80ad3b00188f655e638d9908228770911f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Format BUILD files with buildifier
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Iaea77a89fcd821df5dbb24f5e3e39e6d1dfc0bd4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
[error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
Bazel: Format BUILD files with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: I0d8b7f79177a20dc00c89e2cf0005eb3d3039532
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
We don't need to update time atomically since it's only used to order
cache entries in LRU order.
Change-Id: I756fa6d90b180c519bf52925f134763744f2c1f1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Bazel: Fix warning about deprecated lib.bzl
Format lib/BUILD with buildifier
Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Bazel: Bump skylib library version to 0.8.0
Use bazelisk to switch between used bazel version
Bazel: Require minimum bazel version 0.17.1
Fix wrong placeholder index in error message packInaccessible
JGitText: Remove unused externalized strings
RepoText: Remove unused externalized string
CLI: Remove unused externalized strings
Change-Id: Iea37a8e39e9d4872dc607c9222dcf191ce4e4757
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since commit c3b0dec509fe136c5417422f31898b5a4e2d5e02, Git has
disallowed writing a non-commit to refs/heads/* refs. JGit still
allows that, which can put users in a bad situation. For example,
git push origin v1.0:master
pushes the tag object v1.0 to refs/heads/master, instead of the
intended commit object v1.0^{commit}.
Prevent that by validating that the target of the ref points to a
commit object when pushing to refs/heads/*.
Git performs the same check at a lower level (in the RefDatabase). We
could do the same here, but for now let's start conservatively by
handling it in pushes first.
[jn: fleshed out commit message]
Change-Id: I8f98ae6d8acbcd5ef7553ec732bc096cb6eb7c4e
Signed-off-by: Yunjie Li <yunjieli@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.2:
Prepare 5.1.12-SNAPSHOT builds
JGit v5.1.11.201909031202-r
Prepare 4.11.10-SNAPSHOT builds
JGit v4.11.9.201909030838-r
Bazel: Update bazlets to the latest master revision
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: I7e88c7c7d202f1e3fb8e143277650aa5fefff439
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Prepare 5.1.12-SNAPSHOT builds
JGit v5.1.11.201909031202-r
Prepare 4.11.10-SNAPSHOT builds
JGit v4.11.9.201909030838-r
Bazel: Update bazlets to the latest master revision
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: Iac5dc9683cea97db04d20f27c10f2e103d3ae7b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.0:
Prepare 4.11.10-SNAPSHOT builds
JGit v4.11.9.201909030838-r
Bazel: Update bazlets to the latest master revision
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: I172136a031ff0730e575327cafb3527c9650a71d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.11:
Prepare 4.11.10-SNAPSHOT builds
JGit v4.11.9.201909030838-r
Bazel: Update bazlets to the latest master revision
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: Ifb6a4dbea2f48fd2ffa66eb737d61920aefedfbd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.10:
Bazel: Update bazlets to the latest master revision
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: If672b4f0c350f4e8ff7e1e706485cffd8137236d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
BatchRefUpdate: repro racy atomic update, and fix it
Delete unused FileTreeIteratorWithTimeControl
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Silence API warnings
Change-Id: Id5bf44645655fca40ad22bb1f1ad20a7c2e8f6db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
PackedBatchRefUpdate was creating a new packed-refs list that was
potentially unsorted. This would be papered over when the list was
read back from disk in parsePackedRef, which detects unsorted ref
lists on reading, and sorts them. However, the BatchRefUpdate also
installed the new (unsorted) list in-memory in
RefDirectory#packedRefs.
With the timestamp granularity code committed to stable-5.1, we can
more often accurately decide that the packed-refs file is clean, and
will return the erroneous unsorted data more often. Unluckily timed
delays also cause the file to be clean, hence this problem was
exacerbated under load.
The symptom is that refs added by a BatchRefUpdate would stop being
visible directly after they were added. In particular, the Gerrit
integration tests uses BatchRefUpdate in its setup for creating the
Admin group, and then tries to read it out directly afterward.
The tests recreates one failure case. A better approach would be to
revise RefList.Builder, so it detects out-of-order lists and
automatically sorts them.
Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=548716 and
https://bugs.chromium.org/p/gerrit/issues/detail?id=11373.
Bug: 548716
Change-Id: I613c8059964513ce2370543620725b540b3cb6d1
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add the constant, and implement hashing of known host names in
OpenSshServerKeyDatabase. Add a test verifying that the hashing
works.
Bug: 548492
Change-Id: Iabe82b666da627bd7f4d82519a366d166aa9ddd4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* changes:
dfs: Take size as long instead of int
dfs: Read at the aligned position
dfs: Add a position argument
dfs: Move the deeply nested code to its own method
* stable-5.5:
Return a new instance from openSystemConfig and openUserConfig
Change-Id: I3ba9fa8ceced6e9f883fc23afb0e2c812391beac
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.4:
Return a new instance from openSystemConfig and openUserConfig
Change-Id: If576513ebbd80c7f01b83be5ab2edd436f5739e3
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.3:
Return a new instance from openSystemConfig and openUserConfig
Change-Id: I5f2f957ff27b7941ab29b2dc1c885fbf0093705a
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.2:
Return a new instance from openSystemConfig and openUserConfig
Change-Id: I6491549ab13aad2a4c3f8444a090a94a378eccdb
Signed-off-by: Jonathan Nieder <jrn@google.com>
* stable-5.1:
Return a new instance from openSystemConfig and openUserConfig
Change-Id: I93afa16b14e3e2d127705e3378fb703d6c329694
Signed-off-by: Jonathan Nieder <jrn@google.com>
Move the handling of cached user and system config to getSystemConfig
and getUserConfig methods and revert the implementation of
openSystemConfig and openUserConfig to the old stateless
implementation.
This ensures the open methods respect the passed-in parent config, which
may be different on each invocation. Additionally, returning a new
instance matches the behavior of the previous implementation of the
default system reader, which downstream callers may be depending on.
Move the implementation of the new caching methods getSystemConfig and
getUserConfig up to SystemReader. This avoids that we break the ABI for
subclasses of SystemReader.
Also see [1] which fixed a similar problem with Gerrit's custom
SystemReader.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/225458
Change-Id: If54a2491932d8fc914d4649cb73c9e837c5b8ad0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
These warnings were missed to address in a0048208 which introduced them.
Change-Id: Ia2d15fdce72c10378d020682b80fe7fc548c0d4c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In order to support implementations of CachedPackUriProvider (which need
to supply, among other things, the checksum of the packfile
corresponding to a CachedPack), in a004820858 ("UploadPack: support
custom packfile-to-URI mapping", 2019-08-20), the method getCheckSum()
was added to PackIndex. However, there is no way to access the PackIndex
from a DfsCachedPack.
Therefore, add an accessor for the DfsPackFile stored in the
DfsCachedPack. Now, a user who has a DfsCachedPack can use
DfsCachedPack#getPackFile then DfsPackFile#getPackIndex then
PackIndex#getCheckSum to obtain the checksum of a pack.
Change-Id: Ia010c016f6cac0f058ee20eff4c10f57338bfefc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
When firstParent is set, RevWalk traverses only the first parent of a
commit, even though that commit is UNINTERESTING. Since we want the
maximal UNINTERESTING set, we shouldn't prune any parents here. This
issue is apparent only when some of the commits being traversed are
unparsed, since walker.carryFlagsImpl() propagates the UNINTERESTING
flag to all parsed ancestors, masking the issue.
Therefore teach RevWalk to traverse all parents when a commit is
UNINTERESTING and not only the first parent. Since this issue is
masked by commit parsing, also test situations when the commits
involved are unparsed.
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: I95e2ad9ae8f1f50fbecae674367ee7e0855519b1
* stable-5.4:
Prepare 5.4.3-SNAPSHOT builds
JGit v5.4.2.201908231537-r
Prepare 5.3.5-SNAPSHOT builds
JGit v5.3.4.201908231101-r
Prepare 5.1.11-SNAPSHOT builds
JGit v5.1.10.201908230655-r
Use AtomicReferences to cache user and system level configs
Fix copy-paste typo in CloneCommand#cleanup
SystemReader: Use correct constructor of FileBasedConfig
Change-Id: I241dd1314f5535147ad641576a25f49cae6fe62d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It's expected that jgit should work without native git installation.
In such case Security Manager can be configured to deny access to the
files outside of git repository. JGit tries to find cygwin
installation. If Security manager restricts access to some folders
in PATH, it should be considered that those folders are absent
for jgit.
Also JGit tries to detect if symbolic links are supported by OS. If
security manager forbids creation of symlinks, it should be assumed
that symlinks aren't supported.
Bug: 550115
Change-Id: Ic4b243cada604bc1090db6cc1cfd74f0fa324b98
Signed-off-by: Nail Samatov <sanail@yandex.ru>
* stable-5.3:
Prepare 5.3.5-SNAPSHOT builds
JGit v5.3.4.201908231101-r
Prepare 5.1.11-SNAPSHOT builds
JGit v5.1.10.201908230655-r
Use AtomicReferences to cache user and system level configs
Fix copy-paste typo in CloneCommand#cleanup
SystemReader: Use correct constructor of FileBasedConfig
Change-Id: I4422632766fc5554a20f75346c5480b7fb611484
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.1.11-SNAPSHOT builds
JGit v5.1.10.201908230655-r
Use AtomicReferences to cache user and system level configs
Fix copy-paste typo in CloneCommand#cleanup
Change-Id: I3ababc0f9b540cf1785925395d32a85d30aed40d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Prepare 5.1.11-SNAPSHOT builds
JGit v5.1.10.201908230655-r
Use AtomicReferences to cache user and system level configs
Fix copy-paste typo in CloneCommand#cleanup
Change-Id: Ie383633de334e65901ed4c6deb2d59cfcce98e16
This ensures that only one instance of user and one instance of system
config is set.
Change-Id: Idd00150f91d2d40af79499dd7bf8ad5940f87c4e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
deleteChildren was called on directory instead of gitDir, leading to a
potential null pointer exception if the git directory existed initially.
Bug: 550340
Change-Id: Iafc3b2961253a99862a59e81c7371f7bc564b412
Signed-off-by: Adrien Bustany <adrien-xx-eclipse@bustany.org>
The merge done in change If0c5010a2 resolved a conflict incorrectly
and reverted the fix that was done in change Id0bcdc93b.
Change-Id: I0f5fde33d1f366817f2b966eb42535f7bd3b063e
Reported-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.2:
Add missing @since tag on FileTreeIterator#getLastModifiedInstant
Prepare 5.1.10-SNAPSHOT builds
JGit v5.1.9.201908210455-r
Avoid sign extension when comparing mtime with Instant#getEpochSecond
Fix deprecation in DirCache caused by Instant based DirCacheEntry
Change-Id: If6d5f4dfd9fc8e8c09e29aa11b1004057eafeb9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Add missing @since tag on FileTreeIterator#getLastModifiedInstant
Prepare 5.1.10-SNAPSHOT builds
JGit v5.1.9.201908210455-r
Avoid sign extension when comparing mtime with Instant#getEpochSecond
Fix deprecation in DirCache caused by Instant based DirCacheEntry
Change-Id: Id824c0b8b14dad5947ae9da1f90c3471e07b400f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ensure we use the same type when comparing seconds since the epoch.
This does not prevent that in 2038 timestamps in seconds since the epoch
stored in a 32 bit integer will overflow. Integer.MAX_VALUE translates
to 2038-01-19T03:14:07Z. After this date we'll have an issue since we
store seconds since the epoch in a 32 bit integer in some places.
Bug: 319142
Change-Id: If0c03003d40b480f044686e2f7a2f62c9f4e2fe1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Replace the two int variables smudge_s and smudge_ns by an Instant and
use the new method DirCacheEntry.mightBeRacilyClean(Instant).
Change-Id: Id70adbb0856a64909617acf65da1bae8e2ae934a
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Teach UploadPack to take a provider of URIs corresponding to cached
packs. When fetching, if the client supports the packfile-uri feature,
and if such a cached pack were to be streamed, instead send the
corresponding URI.
This packfile-uri feature is implemented in the jt/fetch-cdn-offload
branch of Git. There is interest in this feature [1], but it is not yet
merged.
[1] https://public-inbox.org/git/cover.1552073690.git.jonathantanmy@google.com/
Change-Id: I9a32dae131c9c56ad2ff4a8a9638ae3b5e44dc15
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* stable-5.4:
Fix NPE in RebaseTodoFile#parseComments
Fix NPE in ObjectIdOwnerMap#get
Fix NPE in CommitOnlyTest#getHead
FileUtils#lastModifiedInstant should not log error if path doesn't exist
Cache user global and system-wide git configurations
Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
Add missing dependencies for running FS_POSIXTest in Eclipse
Fix javadoc for SystemReader#getInstance
Upgrade tycho-extras to 1.4.0
Improve retry handling when saving FileStoreAttributes fails
Ensure FSTest uses MockSystemReader
Make supportsAtomicCreateNewFile return true as default
Update orbit to R20190602212107-2019-06 to enable backports from master
Handle InvalidPathException in FS_POSIX#createNewFileAtomic
Ensure root cause of lock creation failures is logged
Implement toString in MockSystemReader and MockConfig
LocalDiskRefTreeDatabaseTest shall use MockSystemReader
Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
Ensure we use MockSystemReader in tests
Override FileBasedConfig's save method in MockConfig
Remove FileBasedConfig.load(boolean) introduced in d45219ba
Disable debug log for FS in org.eclipse.jgit.test
Bazel: enable logging for tests in org.eclipse.jgit.test
LockFile: log exception if creation of lock file failed
Stop using deprecated Constants.CHARACTER_ENCODING
Change-Id: I709de5edb626536529a99220aae7751b127c9bff
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Fix NPE in RebaseTodoFile#parseComments
Fix NPE in ObjectIdOwnerMap#get
Fix NPE in CommitOnlyTest#getHead
FileUtils#lastModifiedInstant should not log error if path doesn't exist
Cache user global and system-wide git configurations
Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
Add missing dependencies for running FS_POSIXTest in Eclipse
Fix javadoc for SystemReader#getInstance
Improve retry handling when saving FileStoreAttributes fails
Ensure FSTest uses MockSystemReader
Make supportsAtomicCreateNewFile return true as default
Update orbit to R20190602212107-2019-06 to enable backports from master
Handle InvalidPathException in FS_POSIX#createNewFileAtomic
Ensure root cause of lock creation failures is logged
Implement toString in MockSystemReader and MockConfig
LocalDiskRefTreeDatabaseTest shall use MockSystemReader
Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
Ensure we use MockSystemReader in tests
Override FileBasedConfig's save method in MockConfig
Remove FileBasedConfig.load(boolean) introduced in d45219ba
Disable debug log for FS in org.eclipse.jgit.test
Bazel: enable logging for tests in org.eclipse.jgit.test
LockFile: log exception if creation of lock file failed
Stop using deprecated Constants.CHARACTER_ENCODING
Change-Id: I43c2ab8b44c3e87d48e4072907ad169c81e3ffe0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Fix NPE in RebaseTodoFile#parseComments
Fix NPE in ObjectIdOwnerMap#get
Fix NPE in CommitOnlyTest#getHead
FileUtils#lastModifiedInstant should not log error if path doesn't exist
Cache user global and system-wide git configurations
Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
Add missing dependencies for running FS_POSIXTest in Eclipse
Fix javadoc for SystemReader#getInstance
Improve retry handling when saving FileStoreAttributes fails
Ensure FSTest uses MockSystemReader
Make supportsAtomicCreateNewFile return true as default
Update orbit to R20190602212107-2019-06 to enable backports from master
Handle InvalidPathException in FS_POSIX#createNewFileAtomic
Ensure root cause of lock creation failures is logged
Implement toString in MockSystemReader and MockConfig
LocalDiskRefTreeDatabaseTest shall use MockSystemReader
Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
Ensure we use MockSystemReader in tests
Override FileBasedConfig's save method in MockConfig
Remove FileBasedConfig.load(boolean) introduced in d45219ba
Disable debug log for FS in org.eclipse.jgit.test
Bazel: enable logging for tests in org.eclipse.jgit.test
LockFile: log exception if creation of lock file failed
Stop using deprecated Constants.CHARACTER_ENCODING
Change-Id: If0c5010a2cf151ebebb2f2088fac3ee02c5007b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Fix NPE in RebaseTodoFile#parseComments
Fix NPE in ObjectIdOwnerMap#get
Fix NPE in CommitOnlyTest#getHead
FileUtils#lastModifiedInstant should not log error if path doesn't exist
Cache user global and system-wide git configurations
Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
Add missing dependencies for running FS_POSIXTest in Eclipse
Fix javadoc for SystemReader#getInstance
Improve retry handling when saving FileStoreAttributes fails
Ensure FSTest uses MockSystemReader
Make supportsAtomicCreateNewFile return true as default
Update orbit to R20190602212107-2019-06 to enable backports from master
Handle InvalidPathException in FS_POSIX#createNewFileAtomic
Ensure root cause of lock creation failures is logged
Implement toString in MockSystemReader and MockConfig
LocalDiskRefTreeDatabaseTest shall use MockSystemReader
Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
Ensure we use MockSystemReader in tests
Override FileBasedConfig's save method in MockConfig
Remove FileBasedConfig.load(boolean) introduced in d45219ba
Disable debug log for FS in org.eclipse.jgit.test
Bazel: enable logging for tests in org.eclipse.jgit.test
LockFile: log exception if creation of lock file failed
Stop using deprecated Constants.CHARACTER_ENCODING
Change-Id: I48c585f3c9287be7d6ddb6b01a1955444e13fa31
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Previously, the API did not enforce ordering of writes. Misuse of
this API would lead to data effectively being lost.
Guard against that with IllegalArgumentException, and add a test.
Change-Id: I04f55c481d60532fc64d35fa32c47037a03988ae
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Small reftables omit the log index. Currently,
ReftableWriter#shouldHaveIndex does this if there is a single-block
log, but other writers could decide on different criteria.
In the case that the log index is missing, we have to linearly search
for the right block. It is never appropriate to use binary search on
blocks for log data, as the blocks are compressed and therefore
irregularly sized.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id59874edf6bf45c7dec502d9465888e077ffe198
So far the git configuration and the system wide git configuration were
always reloaded when jgit accessed these global configuration files to
access global configuration options which are not in the context of a
single git repository. Cache these configurations in SystemReader and
only reload them if their file metadata observed using FileSnapshot
indicates a modification.
Change-Id: I092fe11a5d95f1c5799273cacfc7a415d0b7786c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The existing javadoc was copied from another method and not adapted.
Change-Id: I39a7e5d719b2c379de9bd1a4710a55a73700c6f0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The "Location" header in a redirect response may contain a relative
URI. Resolve it against the URI the request was made.
Bug: 550033
Change-Id: I29de07dfbbbc794090821b7c190cb2cf662c5a60
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The algorithm for finding keys was already improved in commit db0eb9f8,
but that wasn't quite correct yet.
If there is no pubring.kbx but a private-keys-v1.d directory and a
pubring.gpg, GPG also uses pubring.gpg in combination with the
private-keys-v1.d directory. GPG has three ways to locate public and
private key pairs:
* pubring.kbx and private-keys-v1.d (GPG >= 2.1)
* pubring.gpg and private-keys-v1.d (GPG >= 2.1)
* pubring.gpg and secring.gpg (GPG < 2.1)
See [1] and [2]. pubring.kbx may not exist if the user migrated from
an older GPG installation and didn't run the agent. Since we don't
know which GPG version the user has we must try secring.gpg also if
we found the public key in pubring.gpg, but didn't find the secret
key in the private key directory. Note that GPG < 2.1 also may have
a private key directory, used by the agent. But it may also _not_ have
that directory.
[1] https://lists.gnupg.org/pipermail/gnupg-users/2015-December/054881.html
[2] https://www.gnupg.org/faq/whats-new-in-2.1.html#nosecring
Bug: 549439
Change-Id: I6088014b16c585b6a3408bb31dba3c116e6b583d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
- fix handling of interrupts in FileStoreAttributes#saveToConfig
- increase retry wait time to 100ms
- don't wait after last retry
- dont retry if failure is caused by another exception than
LockFailedException
Change-Id: I108c012717d2bcce71f2c6cb9cf0879de704ebc2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Practically we wouldn't have 2GB+ objects in the DfsBlockCache, but by
making it long, we can clean up some long-to-integer conversions.
Change-Id: I1217f5f273a1420d80e2307ac9ff4a52460237a2
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
The position is anyway aligned in BlockBasedFile, so this is no-op.
Change-Id: Iba037e0ecff339393dd2c03fc5ae4fe858031e4f
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
This makes DfsBlockCache methods more unified. Also this reduces a magic
number embedded in DfsBlockCache.
Change-Id: I61e6c93ca283c0395738103bd2d94091edbccd4e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
The method org.eclipse.jgit.util.FS.supportsAtomicCreateNewFile()
should default to true as mentioned in docs [1]
org.eclipse.jgit.util.FS_POSIX.supportsAtomicCreateNewFile() method
will set the value to false if the git config
core.supportsatomiccreatenewfile is not set.
It should default to true if the configuration is undefined.
[1]
4169a95a65/org.eclipse.jgit/src/org/eclipse/jgit/util/FS_POSIX.java (L372)
Bug: 544164
Change-Id: I16ccf989a89da2cf4975c200b3228b25ba4c0d55
Signed-off-by: Vishal Devgire <vishaldevgire@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
We can't add this method to the super class StoredConfig since that
abstracts from filesystem storage. MockSystemReader.MockConfig is a
StoredConfig and is also used by tests for dfs based storage. Hence
remove this leaky abstraction.
This implies we always use the fallback FileStoreAttributes which means
a config file modification is considered racy within the first 2
seconds. This should not be an issue since typically configs change
rarely and re-reading a config within the racy period is relatively
cheap since configs are small.
Change-Id: Ia2615addc24a7cadf3c566ee842c6f4f07e159a5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If no OpenPGP key is found in pubring.kbx, try the legacy secring.gpg.
This appears to be consistent with GPG[1].
[1] https://lists.gnupg.org/pipermail/gnupg-users/2015-December/054881.html
Bug: 549439
Change-Id: I1557fd9b1f555a9b521fcd57cd3caccbdbacbeda
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.4: (82 commits)
Export all packages of o.e.j.ant and o.e.j.archive bundles
Do not require test bundles to export all packages
Fix API problem filters
Increase severity of AmbiguousMethodReference to ERROR
[error prone] suppress AmbiguousMethodReference in AnyLongObjectId
[error prone] fix ReferenceEquality warning in CommitBuilder
[error prone] suppress NonAtomicVolatileUpdate warning in SimpleLruCache
[error prone] fix ReferenceEquality warning in CommitGraphPane#authorFor
[error prone] fix ReferenceEquality warning in RevWalk#isMergedInto
[error prone] fix ReferenceEquality warning in RefUpdate#updateImpl
[error prone] fix ReferenceEquality warning in static equals methods
[error prone] suppress AmbiguousMethodReference in AnyObjectId
[error prone] fix "FutureReturnValueIgnored" error in FS
Fix formatting and add missing braces in Repository#stripWorkDir
Repository: fix reference comparison of Files
MergeAlgorithm: Suppress Error Prone warning about reference equality
Fix NarrowingCompoundAssignment warnings from Error Prone
FS_POSIX: handle Files.getFileStore() failures
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
GlobalBundleCache: Fix ClassNewInstance warning from Error Prone
IncorrectObjectTypeException: Fix typos in constructors' Javadoc
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Error Prone: Increase severity of NonOverridingEquals to ERROR
Error Prone: Increase severity of ImmutableEnumChecker to ERROR
GitDateParser#ParseableSimpleDateFormat: Make formatStr private final
BatchRefUpdateTest: Suppress ImmutableEnumChecker warning
PacketLineIn: Suppress comparison warnings for END and DELIM
FileSnapshot#toString: Suppress ReferenceEquality warnings
Blame: Suppress ReferenceEquality warning for RevCommit instances
Fix API problem filters
pgm: add missing optional dependency to org.tukaani:xz
NetscapeCookieFile: Make hash static and group overloaded write
NetscapeCookieFile: Javadoc fixes
Config: Handle reference-equality warning (and empty javadoc)
Error Prone: Increase severity of ShortCircuitBoolean to ERROR
ObjectWalk: Prefer boolean operators over logical operators in comparisons
BasePackFetchConnection: Prefer boolean operators over logical operators in comparisons
PackWriter: Prefer boolean operators over logical operators in comparisons
Change-Id: I825fd55bcb5345fb7afe066bf54ca50325f40acb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Comparing with UTF_8 constant in StandardCharsets doesn't require to use
equals.
Change-Id: I6c73a929367f32c9e76ce99f6c0af268480d9230
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It's not important to update time field, scalability is more important
than perfect LRU ordering of cache entries.
Change-Id: I22466c580cd3613b81e1989130b2724af9d6c466
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Implement a helper method suppressing the ReferenceEquality error prone
warning and use it to fix this warning in static equals methods where
this comparison is used to implement fast path of static equals
implementation.
See https://errorprone.info/bugpattern/ReferenceEquality
Change-Id: I33538a3406007d24efec3a504e031ca1069572ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Move the implementation of the static equals() method to a new method
and suppress the error. Deprecate the old method to signal that we
intend to remove it in the next major release.
See https://errorprone.info/bugpattern/AmbiguousMethodReference
Change-Id: I5e29c97f4db3e11770be589a6ccd785e2c9ac7f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Error Prone reports:
[ReferenceEquality] Comparison using reference equality instead of
value equality
The END_EDIT instance is used as a marker, and thus it's OK to use
a reference equality comparison. Factor the comparison to a method
and add a suppression.
Change-Id: I7d9dc1fa21f46c984787056b0b5d163e313026a6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
[NarrowingCompoundAssignment] Compound assignments from long to int
hide lossy casts
and
[NarrowingCompoundAssignment] Compound assignments from int to byte
hide lossy casts
See https://errorprone.info/bugpattern/NarrowingCompoundAssignment
Fix the warnings by adding explicit casts or changing types as
necessary.
Now that all occurrences of the warning are fixed, increase its
severity to ERROR.
Change-Id: Idb3670e6047b146ae37daee07212ff9455512623
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Android unconditionally throws a SecurityException;[1] getFileStore()
is not supported. Catch the exception and don't attempt the hard-
linking atomic file mechanism.
[1] https://android.googlesource.com/platform/libcore/+/21e6175e25
Bug: 548947
Change-Id: Idfba2d9dbcbc80ea15ab2ae7889e5142444c1581
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.3:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Iec3ad6ccc194582cb844310dc172c3103dae4457
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of
StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: I238adfd3080a5fed9d64c3c757297da6ea893918
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Ia57385b2a60f48a5317c8d723721c235d7043a84
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Increase the safety factor to 2.5x for extra safety if max of measured
timestamp resolution and measured minimal racy threshold is < 100ms, use
1.25 otherwise since for large filesystem resolution values the
influence of finite resolution of the system clock should be negligible.
Before, not yet using the newly introduced minRacyThreshold measurement,
the threshold was 1.1x FS resolution, and we could issue the
following sequence of events,
start
create-file
read-file (currentTime)
end
which had the following timestamps:
create-file 1564589081998
start 1564589082002
read 1564589082003
end 1564589082004
In this case, the difference between create-file and read is 5ms,
which exceeded the 4ms FS resolution, even though the events together
took just 2ms of runtime.
Reproduce with:
bazel test --runs_per_test=100 \
//org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_FileSnapshotTest
The file system timestamp resolution is 4ms in this case.
This code assumes that the kernel and the JVM use the same clock that
is synchronized with the file system clock. This seems plausible,
given the resolution of System.currentTimeMillis() and the latency for
a gettimeofday system call (typically ~1us), but it would be good to
justify this with specifications.
Also cover a source of flakiness: if the test runs under extreme load,
then we could have
start
create-file
<long delay>
read
end
which would register as an unmodified file. Avoid this by skipping the
test if end-start is too big.
[msohn]:
- downported from master to stable-5.1
- skip test if resolution is below 10ms
- adjust safety factor to 1.25 for resolutions above 100ms
Change-Id: I87d2cf035e01c44b7ba8364c410a860aa8e312ef
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since we now measure file time resolution we can use it to replace the
hard coded wait time of 25ms. FileSnapshot#equals will return true until
the mtime of the old (o) and the new FileSnapshot (n) differ by at least
one file time resolution.
Change-Id: Icb713a80ce9eb929242ed083406bfb6650c72223
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Cache FileStoreAttributeCache entries since looking up FileStore for a
file may be expensive on some platforms.
Implement a simple LRU cache based on ConcurrentHashMap using a simple
long counter to order access to cache entries.
Change-Id: I4881fa938ad2f17712c05da857838073a2fc4ddb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
Use the fallback timestamp resolution as already described in the
javadoc of these methods. Using zero file timestamp resolution doesn't
make sense.
Change-Id: Iaad2a0f99c3be3678e94980a0a368181b6aed38c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
To enable persisting the minimal racy threshold per FileStore add a
new config option to the user global git configuration:
- Config section is "filesystem"
- Config subsection is concatenation of
- Java vendor (system property "java.vendor")
- Java version (system property "java.version")
- FileStore's name, on Windows we use the attribute volume:vsn instead
since the name is not necessarily unique.
- separated by '|'
e.g.
"AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
The same prefix is used as for filesystem timestamp resolution, so
both values are stored in the same config section
- The config key for minmal racy threshold is "minRacyThreshold" as a
time value, supported time units are those supported by
DefaultTypedConfigGetter#getTimeUnit
- measure for 3 seconds to limit runtime which depends on hardware, OS
and Java version being used
If the minimal racy threshold is configured for a given FileStore the
configured value is used instead of measuring it.
When the minimal racy threshold was measured it is persisted in the user
global git configuration.
Rename FileStoreAttributeCache to FileStoreAttributes since this class
is now declared public in order to enable exposing all attributes in one
object.
Example:
[filesystem "AdoptOpenJDK|11.0.3|/dev/disk1s1"]
timestampResolution = 7000 nanoseconds
minRacyThreshold = 3440 microseconds
Change-Id: I22195e488453aae8d011b0a8e3276fe3d99deaea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
By running FileSnapshotTest#detectFileModified we found that the sum of
measured filesystem timestamp resolution and measured clock resolution
may yield a too small interval after a file has been modified which we
need to consider racily clean. In our tests we didn't find this behavior
on all systems we tested on, e.g. on MacOS using APFS and Java 8 and 11
this effect was not observed.
On Linux (SLES 15, kernel 4.12.14-150.22-default) we collected the
following test results using Java 8 and 11:
In 23-98% of 10000 test runs (depending on filesystem type and Java
version) the test failed, which means the effective interval which needs
to be considered racily clean after a file was modified is larger than
the measured file timestamp resolution.
"delta" is the observed interval after a file has been modified but
FileSnapshot did not yet detect the modification:
"resolution" is the measured sum of file timestamp resolution and clock
resolution seen in Java.
Java version filesystem failures resolution min delta max delta
1.8.0_212-b04 btrfs 98.6% 1 ms 3.6 ms 6.6 ms
1.8.0_212-b04 ext4 82.6% 3 ms 1.1 ms 4.1 ms
1.8.0_212-b04 xfs 23.8% 4 ms 3.7 ms 3.9 ms
1.8.0_212-b04 zfs 23.1% 3 ms 4.8 ms 5.0 ms
11.0.3+7 btrfs 98.1% 3 us 0.7 ms 4.7 ms
11.0.3+7 ext4 98.1% 6 us 0.7 ms 4.7 ms
11.0.3+7 xfs 98.5% 7 us 0.1 ms 8.0 ms
11.0.3+7 zfs 98.4% 7 us 0.7 ms 5.2 ms
Mac OS
1.8.0_212 APFS 0% 1 s
11.0.3+7 APFS 0% 6 us
The observed delta is not distributed according to a normal gaussian
distribution but rather random in the observed range between "min delta"
and "max delta".
Run this test after measuring file timestamp resolution in
FS.FileAttributeCache to auto-configure JGit since it's unclear what
mechanism is causing this effect.
In FileSnapshot#isRacyClean use the maximum of the measured timestamp
resolution and the measured "delta" as explained above to decide if a
given FileSnapshot is to be considered racily clean. Add a 30% safety
margin to ensure we are on the safe side.
Change-Id: I1c8bb59f6486f174b7bbdc63072777ddbe06694d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
These are useful to avoid typos, and also for tab completion.
Change-Id: I0f2d267e46b36bc40297c9657c447f3fd8b9f831
Signed-off-by: David Turner <dturner@twosigma.com>
In a subsequent patch, in some cases, PackWriter#writePack will be
responsible for both the "packfile-uris" and "packfile" sections,
meaning that (in these cases) it must write the "packfile" section
header itself.
In preparation for that patch, move the writing of the "packfile"
section header closer to the invocation of PackWriter#writePack when the
entire fetch response is configured to use the sideband. This means that
"packfile" is written *after* objects are counted (and progress messages
sent to the client in sideband 2) when the "sideband-all" feature is
used (whether "packfile-uris" is used or not), and written *before*
objects are counted otherwise.
Having code to write "packfile" in two places is unfortunate but
necessary. When "sideband-all" is not used, object counting has to
happen after "packfile" is written, because "packfile" activates the
sideband that allows counting progress to be transmitted. When
"packfile-uris" is used, object counting has to happen before "packfile"
is written, because object counting determines whether to send
"packfile-uris" or "packfile". When "sideband-all" is used but
"packfile-uris" is not used, either way works; this commit uses
"packfile-uris" behavior in this case.
Also make the naming of the sideband-activating methods in PacketLineOut
more consistent.
Change-Id: Ifbfd26cc26af10c41b77758168833702d6983df1
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>