Error Prone reports:
[ReferenceEquality] Comparison using reference equality instead of
value equality
The END_EDIT instance is used as a marker, and thus it's OK to use
a reference equality comparison. Factor the comparison to a method
and add a suppression.
Change-Id: I7d9dc1fa21f46c984787056b0b5d163e313026a6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
[NarrowingCompoundAssignment] Compound assignments from long to int
hide lossy casts
and
[NarrowingCompoundAssignment] Compound assignments from int to byte
hide lossy casts
See https://errorprone.info/bugpattern/NarrowingCompoundAssignment
Fix the warnings by adding explicit casts or changing types as
necessary.
Now that all occurrences of the warning are fixed, increase its
severity to ERROR.
Change-Id: Idb3670e6047b146ae37daee07212ff9455512623
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Android unconditionally throws a SecurityException;[1] getFileStore()
is not supported. Catch the exception and don't attempt the hard-
linking atomic file mechanism.
[1] https://android.googlesource.com/platform/libcore/+/21e6175e25
Bug: 548947
Change-Id: Idfba2d9dbcbc80ea15ab2ae7889e5142444c1581
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-5.3:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Iec3ad6ccc194582cb844310dc172c3103dae4457
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of
StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: I238adfd3080a5fed9d64c3c757297da6ea893918
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Fix OpenSshConfigTest#config
FileSnapshot: fix bug with timestamp thresholding
In LockFile#waitForStatChange wait in units of file time resolution
Cache FileStoreAttributeCache per directory
Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Persist minimal racy threshold and allow manual configuration
Measure minimum racy interval to auto-configure FileSnapshot
Reuse FileUtils to recursively delete files created by tests
Fix FileAttributeCache.toString()
Add test for racy git detection in FileSnapshot
Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
Add missing javadoc in org.eclipse.jgit.junit
Enhance RepeatRule to report number of failures at the end
Fix FileSnapshotTests for filesystem with high timestamp resolution
Retry deleting test files in FileBasedConfigTest
Measure filesystem timestamp resolution already in test setup
Refactor FileSnapshotTest to use NIO APIs
Measure stored timestamp resolution instead of time to touch file
Handle CancellationException in FileStoreAttributeCache
Fix FileSnapshot#saveNoConfig
Use Instant for smudge time in DirCache and DirCacheEntry
Use Instant instead of milliseconds for filesystem timestamp handling
Workaround SecurityException in FS#getFsTimestampResolution
Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
FS: ignore AccessDeniedException when measuring timestamp resolution
Add debug trace for FileSnapshot
Use FileChannel.open to touch file and set mtime to now
Persist filesystem timestamp resolution and allow manual configuration
Increase bazel timeout for long running tests
Bazel: Fix lint warning flagged by buildifier
Update bazlets to latest version
Bazel: Add missing dependencies for ArchiveCommandTest
Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
Add support for nanoseconds and microseconds for Config#getTimeUnit
Optionally measure filesystem timestamp resolution asynchronously
Delete unused FileTreeIteratorWithTimeControl
FileSnapshot#equals: consider UNKNOWN_SIZE
Timeout measuring file timestamp resolution after 2 seconds
Fix RacyGitTests#testRacyGitDetection
Change RacyGitTests to create a racy git situation in a stable way
Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Fix non-deterministic hash of archives created by ArchiveCommand
Update Maven plugins ecj, plexus, error-prone
Update Maven plugins and cleanup Maven warnings
Make inner classes static where possible
Fix API problem filters
Change-Id: Ia57385b2a60f48a5317c8d723721c235d7043a84
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Increase the safety factor to 2.5x for extra safety if max of measured
timestamp resolution and measured minimal racy threshold is < 100ms, use
1.25 otherwise since for large filesystem resolution values the
influence of finite resolution of the system clock should be negligible.
Before, not yet using the newly introduced minRacyThreshold measurement,
the threshold was 1.1x FS resolution, and we could issue the
following sequence of events,
start
create-file
read-file (currentTime)
end
which had the following timestamps:
create-file 1564589081998
start 1564589082002
read 1564589082003
end 1564589082004
In this case, the difference between create-file and read is 5ms,
which exceeded the 4ms FS resolution, even though the events together
took just 2ms of runtime.
Reproduce with:
bazel test --runs_per_test=100 \
//org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_FileSnapshotTest
The file system timestamp resolution is 4ms in this case.
This code assumes that the kernel and the JVM use the same clock that
is synchronized with the file system clock. This seems plausible,
given the resolution of System.currentTimeMillis() and the latency for
a gettimeofday system call (typically ~1us), but it would be good to
justify this with specifications.
Also cover a source of flakiness: if the test runs under extreme load,
then we could have
start
create-file
<long delay>
read
end
which would register as an unmodified file. Avoid this by skipping the
test if end-start is too big.
[msohn]:
- downported from master to stable-5.1
- skip test if resolution is below 10ms
- adjust safety factor to 1.25 for resolutions above 100ms
Change-Id: I87d2cf035e01c44b7ba8364c410a860aa8e312ef
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since we now measure file time resolution we can use it to replace the
hard coded wait time of 25ms. FileSnapshot#equals will return true until
the mtime of the old (o) and the new FileSnapshot (n) differ by at least
one file time resolution.
Change-Id: Icb713a80ce9eb929242ed083406bfb6650c72223
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Cache FileStoreAttributeCache entries since looking up FileStore for a
file may be expensive on some platforms.
Implement a simple LRU cache based on ConcurrentHashMap using a simple
long counter to order access to cache entries.
Change-Id: I4881fa938ad2f17712c05da857838073a2fc4ddb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
Use the fallback timestamp resolution as already described in the
javadoc of these methods. Using zero file timestamp resolution doesn't
make sense.
Change-Id: Iaad2a0f99c3be3678e94980a0a368181b6aed38c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
To enable persisting the minimal racy threshold per FileStore add a
new config option to the user global git configuration:
- Config section is "filesystem"
- Config subsection is concatenation of
- Java vendor (system property "java.vendor")
- Java version (system property "java.version")
- FileStore's name, on Windows we use the attribute volume:vsn instead
since the name is not necessarily unique.
- separated by '|'
e.g.
"AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
The same prefix is used as for filesystem timestamp resolution, so
both values are stored in the same config section
- The config key for minmal racy threshold is "minRacyThreshold" as a
time value, supported time units are those supported by
DefaultTypedConfigGetter#getTimeUnit
- measure for 3 seconds to limit runtime which depends on hardware, OS
and Java version being used
If the minimal racy threshold is configured for a given FileStore the
configured value is used instead of measuring it.
When the minimal racy threshold was measured it is persisted in the user
global git configuration.
Rename FileStoreAttributeCache to FileStoreAttributes since this class
is now declared public in order to enable exposing all attributes in one
object.
Example:
[filesystem "AdoptOpenJDK|11.0.3|/dev/disk1s1"]
timestampResolution = 7000 nanoseconds
minRacyThreshold = 3440 microseconds
Change-Id: I22195e488453aae8d011b0a8e3276fe3d99deaea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
By running FileSnapshotTest#detectFileModified we found that the sum of
measured filesystem timestamp resolution and measured clock resolution
may yield a too small interval after a file has been modified which we
need to consider racily clean. In our tests we didn't find this behavior
on all systems we tested on, e.g. on MacOS using APFS and Java 8 and 11
this effect was not observed.
On Linux (SLES 15, kernel 4.12.14-150.22-default) we collected the
following test results using Java 8 and 11:
In 23-98% of 10000 test runs (depending on filesystem type and Java
version) the test failed, which means the effective interval which needs
to be considered racily clean after a file was modified is larger than
the measured file timestamp resolution.
"delta" is the observed interval after a file has been modified but
FileSnapshot did not yet detect the modification:
"resolution" is the measured sum of file timestamp resolution and clock
resolution seen in Java.
Java version filesystem failures resolution min delta max delta
1.8.0_212-b04 btrfs 98.6% 1 ms 3.6 ms 6.6 ms
1.8.0_212-b04 ext4 82.6% 3 ms 1.1 ms 4.1 ms
1.8.0_212-b04 xfs 23.8% 4 ms 3.7 ms 3.9 ms
1.8.0_212-b04 zfs 23.1% 3 ms 4.8 ms 5.0 ms
11.0.3+7 btrfs 98.1% 3 us 0.7 ms 4.7 ms
11.0.3+7 ext4 98.1% 6 us 0.7 ms 4.7 ms
11.0.3+7 xfs 98.5% 7 us 0.1 ms 8.0 ms
11.0.3+7 zfs 98.4% 7 us 0.7 ms 5.2 ms
Mac OS
1.8.0_212 APFS 0% 1 s
11.0.3+7 APFS 0% 6 us
The observed delta is not distributed according to a normal gaussian
distribution but rather random in the observed range between "min delta"
and "max delta".
Run this test after measuring file timestamp resolution in
FS.FileAttributeCache to auto-configure JGit since it's unclear what
mechanism is causing this effect.
In FileSnapshot#isRacyClean use the maximum of the measured timestamp
resolution and the measured "delta" as explained above to decide if a
given FileSnapshot is to be considered racily clean. Add a 30% safety
margin to ensure we are on the safe side.
Change-Id: I1c8bb59f6486f174b7bbdc63072777ddbe06694d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
These are useful to avoid typos, and also for tab completion.
Change-Id: I0f2d267e46b36bc40297c9657c447f3fd8b9f831
Signed-off-by: David Turner <dturner@twosigma.com>
In a subsequent patch, in some cases, PackWriter#writePack will be
responsible for both the "packfile-uris" and "packfile" sections,
meaning that (in these cases) it must write the "packfile" section
header itself.
In preparation for that patch, move the writing of the "packfile"
section header closer to the invocation of PackWriter#writePack when the
entire fetch response is configured to use the sideband. This means that
"packfile" is written *after* objects are counted (and progress messages
sent to the client in sideband 2) when the "sideband-all" feature is
used (whether "packfile-uris" is used or not), and written *before*
objects are counted otherwise.
Having code to write "packfile" in two places is unfortunate but
necessary. When "sideband-all" is not used, object counting has to
happen after "packfile" is written, because "packfile" activates the
sideband that allows counting progress to be transmitted. When
"packfile-uris" is used, object counting has to happen before "packfile"
is written, because object counting determines whether to send
"packfile-uris" or "packfile". When "sideband-all" is used but
"packfile-uris" is not used, either way works; this commit uses
"packfile-uris" behavior in this case.
Also make the naming of the sideband-activating methods in PacketLineOut
more consistent.
Change-Id: Ifbfd26cc26af10c41b77758168833702d6983df1
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
We should not list the complete cache but only show the cache entry at
hand.
Change-Id: I22be2a4dcbf0145155e23f2389bfcf5662cf23a6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Repeat the test 10000 times to get statistics if measured
fsTimestampResolution is working in practice to detect racy git
situations.
Add a class to compute statistics for this test. Log delta between
lastModified and time when FileSnapshot failed to detect modification.
This happens if the racy git limit determined by measuring filesystem
timestamp resolution and clock resolution is too small. If it would be
correct FileSnapshot would always detect modification or mark it
modified if time since modification is smaller than the racy git limit.
Change-Id: Iabe7af1a7211ca58480f8902d4fa4e366932fc77
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The newly introduced ProtocolV2HookChain is implemented using lists
instead of arrays.
Update PostUploadHookChain to keep the hook chains implementation
consistent.
Change-Id: I5ae0c923f117ac48558a989464f5d5d868d81f76
Signed-off-by: Ivan Frade <ifrade@google.com>
The newly introduced ProtocolV2HookChain is implemented using lists
instead of arrays.
Update PostUploadHookChain to keep hook chain implementations
consistent.
Change-Id: Ic5694feab943e8949896b93103dbf427716c9bd7
Signed-off-by: Ivan Frade <ifrade@google.com>
Set synchronized to make the config access consistent.
> Inconsistent synchronization of
org.eclipse.jgit.transport.JschConfigSessionFactory.config; locked 80%
of time
In order to make JschConfigSessionFactory threadsafe, synchronize this
method as well.
Change-Id: I32d1bfc2e98363d254992144e795ce72fe1e8846
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Measure granularity of timestamps stored in the filesystem by setting
and then getting lastModified timestamp until the read value changed.
Increase increment exponentially to limit number of iterations starting
with 1 microsecond since Java's FileTime (up to Java 12) truncates
timestamps to 1 microsecond resolution. The chosen algorithm yields 2000
steps between 1 ms and 2.5 s.
Also measure clock resolution and add that for the total timestamp
resolution. This avoids systematic measurement errors introduced by
doing IO to touch a file.
Change-Id: I9b37138619422452373e298d9d8c7cb2c384db3f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
We should not use configuration when creating FileSnapshot when
accessing FileBasedConfig.
Change-Id: Ic521632870f18bb004751642b9d30648dd94049a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A caller cannot install a second hook in the UploadPack without
overwriting whatever is already there.
Offer a method to get the current protocol v2 hook, so it can be chained
with new hooks.
Change-Id: Icb06f94ec52b8c8714f509b5b8622d6db42960fb
Signed-off-by: Ivan Frade <ifrade@google.com>
UploadPack only supports one protocol-v2 hook. There are already cases
where more than one is needed.
Offer a Chain class to compose ProtocolV2Hooks, as other hooks do. It
looks like a single hook but it calls all its members.
Change-Id: Idd173ca7df6672079ac0de03c67f77abac376538
Signed-off-by: Ivan Frade <ifrade@google.com>
This enables higher file timestamp resolution on filesystems like ext4,
Mac APFS (1ns) or NTFS (100ns) providing high timestamp resolution on
filesystem level.
Note:
- on some OSes Java 8,9 truncate milliseconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes truncates timestamp resolution to microseconds when
converting the internal representation to FileTime exposed in the API,
see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution
Change-Id: I25ffff31a3c6f725fc345d4ddc2f26da3b88f6f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
On Android FS#getFsTimestampResolution always throws a
SecurityException, handle this by falling back to the fallback timestamp
resolution.
Bug: 548947
Change-Id: I0ee6cb3c20e189bdc8d488434a930427ad6f2df2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It seems on cygwin creating a file under a writable directory can fail
with AccessDeniedException. Log a warning in this case and fallback to
worst case timestamp resolution of 2 seconds.
Bug: 548648
Change-Id: Ic50c31ce9dc9ccadd4db5247df929418ac62d45c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Checking lastModified is time critical hence debug trace is the only way
to analyze issues since debugging is impractical.
Also add configuration for buffering of log4j output to reduce runtime
impact when debug trace is on. Limit buffer to 1MiB and comment this
configuration out since we may not always want to use buffering.
Change-Id: Ib1a0537b67c8dc3fac994a77b42badd974ce6c97
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use options
- StandardOpenOption.CREATE to create touched file if not existing
- StandardOpenOption.SYNC to enforce synch of data and meta data changes
- StandardOpenOption.WRITE
Also set mtime explicitly in FileUtils#touch to the current system time.
This should fix that the previous implementation didn't work on
- locally cached Windows network share (CSC-CACHE filesystem) mapped as
a drive
- nfsv4 mounts on Linux
and that it didn't create unborn file like Linux command "touch".
Apache common's and Guava's touch() use the same approach.
Immediately after creating the probe file used to measure timestamp
resolution touch it. This ensures we always use the local system clock
when measuring filesystem timestamp resolution. This should prevent that
clock skew could influence the measured timestamp resolution in case of
a mounted network filesystem.
Bug: 548598
Change-Id: Iaeaf5967963f582395a195aa637b8188bfadac60
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
To enable persisting filesystem timestamp resolution per FileStore add a
new config section to the user global git configuration:
- Config section is "filesystem"
- Config subsection is concatenation of
- Java vendor (system property "java.vm.vendor")
- runtime version (system property "java.vm.version")
- FileStore's name
- separated by '|'
e.g.
"AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
The prefix is needed since some Java versions do not expose the full
timestamp resolution of the underlying filesystem. This may also
depend on the underlying operating system hence concrete key values
may not be portable.
- Config key for timestamp resolution is "timestampResolution" as a time
value, supported time units are those supported by
DefaultTypedConfigGetter#getTimeUnit
If timestamp resolution is already configured for a given FileStore
the configured value is used instead of measuring the resolution.
When timestamp resolution was measured it is persisted in the user
global git configuration.
Example:
[filesystem "AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"]
timestampResolution = 1 seconds
If locking the git config file fails retry saving the resolution up to 5
times in order to workaround races with another thread.
In order to avoid stack overflow use the fallback filesystem timestamp
resolution when loading FileBasedConfig which creates itself a
FileSnapshot to help checking if the config changed.
Note:
- on some OSes Java 8,9 truncate to milliseconds or seconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes up to Java 12 truncates timestamp resolution to
microseconds when converting the internal representation to FileTime
exposed in the API, see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution up to
Java 12
Hence do not attempt to manually configure a higher timestamp resolution
than supported by the Java version being used at runtime.
Bug: 546891
Bug: 548188
Change-Id: Iff91b8f9e6e5e2295e1463f87c8e95edf4abbcf8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Allow the client to specify "sideband-all" in a fetch v2 request,
indicating that the whole response is to be multiplexed (with a sideband
indicator on every non-flush and non-delim pkt) instead of only the
packfile being multiplexed. This allows, for example, progress messages
to be sent at any point in the response.
This implements the "sideband-all" feature documented in
Documentation/technical/protocol-v2.txt in Git.
Change-Id: I3e7f21c88ff0982b1b7ebb09c9ad6c742c4483c8
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
It is difficult to track what's happening with the pckOut instance
field, so replace it with a local variable in #upload instead.
Change-Id: Ibd9225b28334b7133eccdc6d82b26fc96cbde299
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Use brackets in all "if" statements and remove the "final" from local
variables and method arguments.
https://wiki.eclipse.org/EGit/Contributor_Guide#Coding_standards
Change-Id: I185f3112848fc1218cd7adb9828488f03fa4ddfc
Signed-off-by: Ivan Frade <ifrade@google.com>
In order to avoid blocking on the main thread during measurement
interactive applications like EGit may want to measure the filesystem
timestamp resolution asynchronously.
In order to enable measurement in the background call
FileStoreAttributeCache.setAsyncfileStoreAttrCache(true)
before the first access to cached FileStore attributes.
Bug: 548188
Change-Id: I8c9a2dbfc3f1d33441edea18b90e36b1dc0156c7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add a unittest.
In commit I5485db55 ("Fix FileSnapshot's consideration of file size"),
the special casing of UNKNOWN_SIZE was forgotten.
This change, together with I493f3b57b ("Measure file timestamp
resolution used in FileSnapshot") introduced a regression that would
occasionally surface in Gerrit integration tests marked UseLocalDisk,
with the symptom that creating the Admin user in NoteDb failed with a
LOCK_FAILURE.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I7ffd972581f815c144f810481103c7985af5feb0
Fixes PDE API checks complaining: the methods were added
in JGit 5.5.0.
Change-Id: I9ff860c3408c6bb3891fa0da7547394d0fe9d0b6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
It was reported that measuring file timestamp resolution may hang
indefinitely on nfs. Hence timeout this measurement at the known worst
filesystem timestamp resolution (FAT) of 2 seconds.
Bug: 548188
Change-Id: I17004b0aa49d5b0e76360a008af3adb911b289c0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
All other exceptions are handled in a wrapped sendPack method.
Consolidate the error handling code.
Change-Id: Ieac0ce64960534d009d1e6b025130b021b744794
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
By doing this, exceptions thrown by sendPack are also covered by the
same code.
Change-Id: I3509f2d832af1410f307e931577e4d07e32b014e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
When another exception is thrown while handling another exception, that
exception can be attached to the original exception since Java 7
(Throwable#getSuppressed). Attach the secondary exception to the
original exception instead of throwing it away.
Change-Id: Ia093b8207714f2638e0343bc45a83d4342947505
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
RevWalk does not currently provide a --first-parent equivalent and the
feature has been requested.
Add a field to the RevWalk class to specify whether walks should
traverse first parents only. Modify Generator implementations to support
the feature.
Change-Id: I4a9a0d5767f82141dcf6d08659d7cb77c585fae4
Signed-off-by: Dave Borowitz <dborowitz@google.com>
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
If a client clones with "--filter=blob:none", the checkout that "git
clone" automatically does causes the client to fetch all blobs at HEAD.
When fetching from a non-bitmapped repository, this will fail if an
object walk is ever needed, because JGit currently rejects such requests
- see the commit message of d3021788d2 ("Use bitmaps for non-commit
reachability checks", 2017-11-10) for more information.
Rejecting such requests in the absence of bitmaps is probably
overzealous: it is true that the server would prefer to have bitmaps in
this case, but there might be a small proportion of repos (for example,
very small repos or newly created ones) that do not have bitmaps, yet
the server would still like to have partial clones for them.
So, allow such requests, performing the object walk reachability check
if necessary. Limit this to servers with "uploadpack.allowFilter"
configured, so that servers wanting to support partial clone have this
functionality, and servers that do not support partial clone do not have
to pay the object walk reachability check cost.
Change-Id: I51964bafec68696a799625d627615b4f45ddbbbf
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Error Prone reports:
[ClassNewInstance] Class.newInstance() bypasses exception checking;
prefer getDeclaredConstructor().newInstance()
See https://errorprone.info/bugpattern/ClassNewInstance
This was the only occurrence of the warning in the code base; now it's
fixed, increase the severity to ERROR to prevent future occurrences.
Change-Id: Ic04d1c5d2bd458bbb4bb399d6ce9d147bd48d0b1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The placeholders in manifest and plugin.properties did not match. To
avoid similar issues, all placeholders have been changed to
Bundle-Vendor and Bundle-Name now.
Bug:548503
Change-Id: Ibd4b9bc237b323e614506b97e5fbc99416365040
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
The deleted code is not required as removed files are deleted correctly in
doCheckout() anyway.
The deleted code failed in case a non-empty directory had to be deleted.
file.delete() returned false, triggering an exception.
Bug: 479266
Change-Id: I011bb3882ff0c35b238aa3eccad7889041210277
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
Creating a folder failed in case a file with the same name already
existed.
Bug: 479266
Change-Id: Ia987660ec0968ad4081dbd5a60e80660539497e3
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Archives created by the ArchiveCommand didn't produce deterministic
archive hashes. For RevCommits RevWalk.parseTree returns the root tree
instead of the RevCommit hence retrieving the commit's timestamp didn't
work. Instead use RevWalk.parseAny and extract the tree manually.
Archive entries store timestamps with 1 second resolution hence we need
to wait longer when creating the same archive twice and compare archive
hashes. Otherwise hash comparison in tests wouldn't fail without this
patch.
Bug: 548312
Change-Id: I437d515de51cf68265584d28a8446cebe6341b79
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
update Maven plugins
- jacoco-maven-plugin to 0.8.4
- japicmp-maven-plugin to 0.14.1
- maven-compiler-plugin to 3.8.1
- maven-deploy-plugin to 3.0.0-M1
- maven-enforcer-plugin to 3.0.0-M2
- maven-install-plugin to 3.0.0-M1
- maven-jar-plugin to 3.1.2
- maven-javadoc-plugin to 3.1.0
- maven-jxr-plugin to 3.0.0
- maven-pmd-plugin to 3.12.0
- maven-resources-plugin to 3.1.0
- maven-shade-plugin to 3.2.1
- maven-source-plugin to 3.1.0
- maven-surefire-plugin to 3.0.0-M3
- spotbugs-maven-plugin to 3.1.12
- tycho to 1.3.0
- tycho-pack200a-plugin to 1.3.0
- tycho-pack200b-plugin to 1.3.0
Cleanup Maven warnings
- pin version of all used Maven plugins
- remove deprecated way to declare minimum Maven version
Change-Id: If23e2e2bb03e5e1e7b1eb9d4924a8faa0aa3704e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Replace javax.xml.bind.DatatypeConverter, that is not available any
more in Java 11 and later with Hex utility from non optional Bouncy
Castle library.
Bug: 540790
Change-Id: I9903c00ecc1a434e9795b8ba9267f02628fdc0e9
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
As reported by Error Prone:
An inner class should be static unless it references members of its
enclosing class. An inner class that is made non-static unnecessarily
uses more memory and does not make the intent of the class clear.
See https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: Ib99d120532630dba63cf400cc1c61c318286fc41
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ee40efcea4)
Error Prone reports the warning on several classes:
[NonOverridingEquals] equals method doesn't override Object.equals;
if this is a type-specific helper for a method that does override
Object.equals, either inline it into the callers or rename it to
avoid ambiguity.
See https://errorprone.info/bugpattern/NonOverridingEquals
Most of these are in the public API, so we can't rename or inline them
without breaking the API. FileSnapshot is not part of the public API,
but clients may be using it anyway, so we also shouldn't change that.
Suppress all the warnings instead. Having the check at severity ERROR
will at least make sure we don't introduce any new occurrences.
Change-Id: I92345c11256f06b4fa03ccc13337f72af5a43591
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
ParseableSimpleDateFormat is an enum, and enums must be immutable,
hence the member should be final. At the same time, make it private
since it does not need to be publicly visible.
Change-Id: I7e181f591038d556f1123b6e37adf8441059e99a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Reference comparison is intentional. The END and DELIM string
constants are used as sentinels and will always be the same
instances.
Suppress both ReferenceEquality and StringEquality warnings.
Change-Id: I4ce0495702c56b3911f42f26c2f81d28073cbe19
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Reference comparison with EMPTY and MISSING_FILE is intended; these
are static instances used as markers, and will always be the same
instances.
Change-Id: Ic27f5b797bdb9370cf8f6b3b7bb3f1523d4a454c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When cloning repository with --single-branch option, tag chains are not
packed and pack file is broken in some cases.
Typical test-case:
git tag -a test_tag <commit-id>
git tag -a test_prev_tag test_tag
git tag -d test_tag
git clone --single-branch <repository>
fatal: did not receive expected object <test_tag_id>
The reason for that is missing object for original test_tag reference,
which was deleted.
Problem description:
When pack-objects is given --include-tag, it peels each tag reference
down to a commit. If the commit is prepared to be packed, we we have to
include such tag too. The problem is when the tag points to through some
chain of other tag to commit. Then, the inner tags are not added leading
to broken pack.
Fix:
When going to commit, we have to check and add any of the tags on the
way (if they were not selected, which may happen with --single-branch
option).
Change-Id: I1682d4a2c52d674f90a1b021e0f6c3524c5ce5bc
Signed-off-by: Pavel Flaška <Pavel.Flaska@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Issues reported by downstream analyzers.
The "hash" method can be static.
It is a good practice to group overloaded methods. Move the write(URL)
method with the other writes.
Change-Id: Ia42c0d7081333edcb77e58d5e627929d29672490
Signed-off-by: Ivan Frade <ifrade@google.com>
Downstream analyzers reported empty fields in the javadoc. I corrected
few more details:
* Fill empty javadoc fields.
* Use <p> to separate description paragraphs.
* End description paragraphs with a period.
* Remove period at the end of field descriptions.
Change-Id: I749e4b821fc855999caddc442ac788fa514386ea
Signed-off-by: Ivan Frade <ifrade@google.com>
Reported by downstream analyzers. Suppress the warning on reference
equality for isMissing and fill an empty javadoc field.
Change-Id: I3494423daf2a53ca10e0a9c66553f00204c35396
Signed-off-by: Ivan Frade <ifrade@google.com>
Using the | and & operators in boolean conditions results in a warning
from Error Prone:
[ShortCircuitBoolean]
Prefer the short-circuiting boolean operators && and || to & and |.
see https://errorprone.info/bugpattern/ShortCircuitBoolean
Change-Id: I182f986263b8b9ac189907f4bd1662b4092a52d8
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Using the | and & operators in boolean conditions results in a warning
from Error Prone:
[ShortCircuitBoolean]
Prefer the short-circuiting boolean operators && and || to & and |.
see https://errorprone.info/bugpattern/ShortCircuitBoolean
Change-Id: I6cccca3fdd28bf93b302a9b8a66e68ac912cb60d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Using the | and & operators in boolean conditions results in a warning
from Error Prone:
[ShortCircuitBoolean]
Prefer the short-circuiting boolean operators && and || to & and |.
see https://errorprone.info/bugpattern/ShortCircuitBoolean
Change-Id: I4275c60306e43c74030c4465ba02cb853ad444e1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Fix all remaining instances of the OperatorPrededence warning, by adding
parentheses to make the precedence explicit.
Change-Id: Ib296dfed09f9be042d0ff0f7fad8214e4dd766b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* master:
Handle missing "ours" stage in WorkingTreeIterator.hasCrLfInIndex()
Config: Add helper method to check for empty value
ErrorProne: Increase severity of FutureReturnValueIgnored to ERROR
FS_Win32: Add missing parentheses on if-blocks
Upgrade spotbugs to 3.1.12
Abbreviated{Long}ObjectId: Make operator precedence explicit
GC: Update TODO comments
FS_POSIX: Fix reference comparison of Boolean.FALSE
Increase bazel timeout for long running tests
Use bazelisk to switch between used bazel version
Bump minimum Bazel version to 0.26.1
Bazel: Bump skylib library version to 0.8.0
Retry loading config when locked by another process
Make pull --rebase on an unborn branch do a checkout
Warn if configured cookie file is missing
Handle escaped CR-LF in git config files
DescribeCommand: use glob match instead of path match
Fix off-by-one error in RebaseTodoFile when reading a todo file
Consistently use "!isEmpty()" to detect non-empty list
TransportHttp: Check for non-empty list with "!isEmpty()" rather than
"size() > 0"
TransportHttp: Fix comparison of size with ">= 0"
NetscapeCookieFileTest: Split HttpCookiesMatcher to own class
Bazel: Add missing dependency on mockito for TransportHttpTest
Determine hard-linking and nlink support per FileStore
Support reading and writing cookies.
Repository: Add getIdentifier() method to avoid instanceof operator
Update to Orbit R20190602212107
PacketLineIn: Deprecate the END constant
PacketLineIn: Add an iterator over strings in the input stream
Replace most usages of PacketLineIn.END with PacketLineIn.end()
PacketLineIn: Deprecate DELIM constant
Replace trivial reference comparison of PacketLineIn.{DELIM,END}
PacketLineIn: Rename isDelim to isDelimiter
ProtocolV2ParserTest: Fix typo in comment
Upgrade Bouncy Castle to 1.61
Update to Orbit R20190531194818 and rollback update to Ant 1.10.6
cli: Add the --always option to describe
DescribeCommand: Support the "always" option
cli: Add the --tags option to describe
DescribeCommand: Consistenly omit the default value
Remove excess blank line in FileUtilsTest
PacketLineIn: Add helper methods to check for END and DELIM
UploadPackTest: Rename variable to avoid hiding class member
UploadPackTest: Add missing <> operator on instantiation of ArrayList
BitmapCalculator: javadoc fixes
RevWalkUtils: add progress callback to findBranchesReachableFrom
Upgrade maven-source-plugin to 3.1.0
Upgrade maven-jar-plugin to 3.1.2
Upgrade jacoco-maven-plugin to 0.8.4
BitmapCalculator and its test: add missing license header
RevWalk: new method createReachabilityChecker()
Change-Id: I4d76c7c0dbe6411c842f3468b709f7df51789c08
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In a delete-modify conflict with the deletion as "ours" there may be
no stage 2 in the index. Add appropriate null checks. Add a new test
for this case, and verify that the file gets added with a single LF
after conflict resolution with core.autocrlf=true. This matches the
behavior of canonical git for this case.
Bug: 547724
Change-Id: I1bafdb83d9b78bf85294c78325e818e72fae53bc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Rename MAGIC_EMPTY_VALUE to MISSING_ENTRY, make it private, and add
a helper method to check if a given string is that value.
This avoids that callers trigger the "reference equality" warning
from Error Prone.
Change-Id: Idc76f78c0cf1828aa48d02ee33911a4b5df50355
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The TODO comments say "in 5.0", but 5.0 was already released without
resolving them. Remove "in 5.0" on the assumption that the mentioned
improvements still need to be done at some point.
Change-Id: I3eb429803e2266de3fc490e1f3912991c08aa1ad
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When loading the config, a FileNotFoundException may occur if the file
exists but cannot be read (see [1]). This is the case on Windows with a
virus scanner checking the file. Therefore if the file exists and that
exception is thrown, retry multiple times, similar to how this was
already implemented for IOException.
[1] https://docs.oracle.com/javase/8/docs/api/java/io/FileNotFoundException.html
Bug: 529522
Change-Id: Ic5dc3b7b24bb0005d6256ed00513bc7c0b91e613
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A merging pull on an unborn branch was already supported. But a
rebasing pull failed. If the user has pull.rebase = true in his
user config, the pull would try to rebase. Rebasing needs a parent
commit, though. Native git handles this case:
git init
git remote add origin <URI>
git pull --rebase origin master
Check up front in PullCommand for the unborn head and just do a
checkout in this case. MergeCommand already has similar code.
Bug: 544965
Change-Id: I1277e1ac0b0364b4623fd791f3d6b07bd5f58fca
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
We logged a stack trace if the configured http.cookieFile was missing.
Instead only log a warning.
Bug: 548081
Change-Id: I42e39f5ad8ffce7b43162e5068f60af073b8a126
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Canonical git treats CR-LF in config files as LF.[1][2] JGit does so,
too, except when escaped as a line continuation. Correct this and
treat the sequence \-CR-LF as a line continuation.
[1] https://github.com/git/git/commit/db2c075d9
[2] https://github.com/git/git/blob/v2.21.0/config.c#L485
Bug: 545850
Change-Id: I51e7378a22c21b3baa3701163c423d04c900af5a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Otherwise tags may fail to match if their name contains slashes.
Canonical git also uses its wildcard matcher in glob mode.[1]
[1] https://github.com/git/git/blob/v2.21.0/builtin/describe.c#L182
Bug: 546703
Change-Id: I122c7959974fa1fc6a53dfc65837e4314a8badd4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Commit messages of length 1 were not read. 'lineEnd' is the offset
of the last character in the line before the terminating LF or CR-LF,
and 'nextSpace' is actually the offset of the character _after_ the
next space. With a one-character commit message, nextSpace == lineEnd.
The code also assumes the commit message to be optional, but actually
failed in that case because it read beyond the line ending. Fix that,
too.
Add a test case for reading a todo file.
Bug: 546245
Change-Id: I368d63615930ea2398a6230e756442fd88870654
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Replace "size() > 0" with "!isEmpty()" where appropriate.
In the Status implementation we can drop the check; the subsequent
loop will only execute when the list is non-empty anyway.
Change-Id: I355aff551a603373e702a9d44304f087b476263c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Error Prone reports:
[SizeGreaterThanOrEqualsZero] Comparison of a size >= 0 is always true,
did you intend to check for non-emptiness?
see https://errorprone.info/bugpattern/SizeGreaterThanOrEqualsZero
Change-Id: Ie964771cacca4b15569eb45f6e273ad2a7e2e49c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
It's quite possible that JGit can use the hard-linking mechanism
for atomic file creation on some volumes but not on others.
Ultimately it depends on the file systems on the mounted volumes.
Cache the information per FileStore instead of using a single
global flag. Also catch FileSystemException, it may be thrown
if the operating system reports a failure. The previously caught
AccessDeniedException is a sub-class of FileSystemException.
Bug: 547332
Change-Id: I1ef672b3468b0be79e71674344f16f28f9d11ba1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The git config entries "http.cookieFile" and
"http.saveCookies" are correctly evaluated.
Bug: 488572
Change-Id: Icfeeea95e1a5bac3fa4438849d4ac2306d7d5562
Signed-off-by: Konrad Windszus <konrad_w@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When Error Prone checks are enabled, the "ClassCanBeStatic" warning is
triggered:
Inner class is non-static but does not reference enclosing class
see https://errorprone.info/bugpattern/ClassCanBeStatic
Change-Id: I5a0e3bf0cf8c28176d9c98914c1c0dfab9c5736f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.3:
Prepare 5.3.3-SNAPSHOT builds
JGit v5.3.2.201906051522-r
Prepare 5.1.9-SNAPSHOT builds
JGit v5.1.8.201906050907-r
Test detecting modified packfiles
Enhance fsTick() to use filesystem timer resolution
Add debug trace to measure time needed to open pack index
Extend FileSnapshot for packfiles to also use checksum to detect changes
Wait opening new packfile until it can't be racy anymore
Avoid null PackConfig in GC
Add FileSnapshot test testing recognition of file size changes
Capture reason for result of FileSnapshot#isModified
Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows
Tune max heap size for tests
Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc
ObjectDirectory: fix closing of obsolete packs
Update API filters for methods added to fix bugs
Bazel: Increase severity of most error-prone checks to ERROR
Enable error-prone checks by default
Add bazel options to align with gerrit's
Include filekey file attribute when comparing FileSnapshots
Measure file timestamp resolution used in FileSnapshot
Fix FileSnapshot's consideration of file size
Fix API problem filters
Fix API problem filters
Change-Id: Ie1c7e4752661aec9e1754660934921224e2408eb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This change is needed to implement permission aware ref database in
Gerrit: [1], that is a pre-requisite to re-enable Git v2 protocol in
Gerrit: [2].
Background: Last year Git v2 protocol was enabled in Gerrit. The fact,
that JGit layer was not calling ref advertise filter for Git v2
protocol, introduced security vulnerability.
The lesson learned from this security incident: Gerrit should not rely
on ref advertise filter being called by JGit to implement crictical
security checks. Instead, the idea is to use the same approach as
currently used by Google's internal code on googlesource.com that
didn't suffer from this vulnerability: provide a custom repository to
JGit. The repository provides a RefDatabase that is permission-aware
and will only ever return refs that the user has access to.
However, due to hard coded instanceof operator usages in JGit code
base, some tests in Gerrit are failing with: [1] in place. This change
addresses this problem.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/212874
[2] https://gerrit-review.googlesource.com/c/gerrit/+/226754
Change-Id: I67c0f53ca33b149442e7ee3e51910d19e3f348d5
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.1.9-SNAPSHOT builds
JGit v5.1.8.201906050907-r
Test detecting modified packfiles
Enhance fsTick() to use filesystem timer resolution
Add debug trace to measure time needed to open pack index
Extend FileSnapshot for packfiles to also use checksum to detect changes
Wait opening new packfile until it can't be racy anymore
Avoid null PackConfig in GC
Add FileSnapshot test testing recognition of file size changes
Capture reason for result of FileSnapshot#isModified
Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows
Tune max heap size for tests
Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc
ObjectDirectory: fix closing of obsolete packs
Include filekey file attribute when comparing FileSnapshots
Measure file timestamp resolution used in FileSnapshot
Fix FileSnapshot's consideration of file size
Fix API problem filters
Fix API problem filters
Change-Id: Id79a3a19f9a31dff94d10a406c2b6e08a506931a
* stable-5.1:
Prepare 5.1.9-SNAPSHOT builds
JGit v5.1.8.201906050907-r
Test detecting modified packfiles
Enhance fsTick() to use filesystem timer resolution
Add debug trace to measure time needed to open pack index
Extend FileSnapshot for packfiles to also use checksum to detect changes
Wait opening new packfile until it can't be racy anymore
Avoid null PackConfig in GC
Add FileSnapshot test testing recognition of file size changes
Capture reason for result of FileSnapshot#isModified
Skip FileSnapshotTest#testSimulatePackfileReplacement on Windows
Tune max heap size for tests
Fix FileSnapshotTest.testNewFileNoWait() to match its javadoc
ObjectDirectory: fix closing of obsolete packs
Include filekey file attribute when comparing FileSnapshots
Measure file timestamp resolution used in FileSnapshot
Fix FileSnapshot's consideration of file size
Fix API problem filters
Change-Id: I3ac77bfa03f7436de12ab86e1bba29afee5ccd01
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If the attributes of FileSnapshot don't detect modification of a
packfile read the packfile's checksum and compare it against the
checksum cached in the loaded packfile.
Since reading the checksum needs less IO than reloading the complete
packfile this may help to reduce the overhead to detect modficiation
when a gc completes while ObjectDirectory scans for packfiles in another
thread.
Bug: 546891
Change-Id: I9811b497eb11b8a85ae689081dc5d949ca8c4be5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If
- pack.waitPreventRacyPack = true (default is false)
- packfile size > pack.minSizePreventRacyPack (default is 100 MB)
wait after a new packfile was written and before it is opened until it
cannot be racy anymore.
If a new packfile is accessed while it's still racy at least the pack's
index will be reread by ObjectDirectory.scanPacksImpl(). Hence it may
save resources to wait one tick of the file system timer to avoid this
reloading. On filesystems with a coarse timestamp resolution it may be
beneficial to skip this wait for small packfiles.
Bug: 546891
Change-Id: I0e8bf3d7677a025edd2e397dd2c9134ba59b1a18
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Deprecate the constant with the intention of making it private in
a future release.
All existing usage of the constant within JGit code has already been
replaced with the recommended alternatives in preceding commits.
Change-Id: I10eb95f3f92cb74f93a26bf1a6edd24615b75c6f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Allows callers to read all lines in the input stream until the
END marker is reached, without having to explicitly check for
the END marker.
Replace all remaining usage of the END marker with the new method.
Change-Id: I51f419c7f569ab7ed01e1aaaf6b40ed8cdc2116b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
PacketLineIn.END is only referenced in tests. Replace most of those
with a new package visible end() method.
Remaining usages of PacketLineIn.END are in the form:
while ((line = pckIn.readString()) != PacketLineIn.END) {
and are not trivial replacements, hence are not touched in this change.
Change-Id: Id77c5321ddcad127130b246bde8f08736e60e1ea
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Deprecate DELIM with the intention of making it private in a future
release.
Callers that want to test if a packet line string is the delimiter
should use the isDelimiter(String) method.
The only other references to DELIM in the JGit code are in tests. For
those, introduce a package visible delimiter() method.
Change-Id: I21e8bbac0ffb9ef710c9753e23435416b09a4891
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Replace reference comparisons of PacketLineIn's DELIM and END strings
with usage of the helper methods isDelimiter() and isEnd().
Change-Id: I52dcfc4ee9097f1bd6970601c716701847d9eebd
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Also now use JcaKeyBoxBuilder constructor in
BouncyCastleGpgKeyLocator.readKeyBoxFile(Path).
CQ: 19868
CQ: 19869
CQ: 19870
Change-Id: I45bd80e158aecd90448b0c7e59615db27aaef892
Signed-off-by: Brandon Weeks <bweeks@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Omit the default value of "false" for "useTags" like already done for
"longDesc".
Change-Id: I25aaacae028fc8cf27f4040ba45fe79609318aa1
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
These methods will allow clients to check for END and DELIM without
doing a reference comparison on the String objects, which raises
warnings from Error Prone.
Change-Id: I9e7e59843553ed4488ee8e864033198bbb60d67c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Initialize it using the repository's config already in the constructor.
Change-Id: I4ea620a7db72956e7109f739990f09644640206b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This allows to verify the expected behavior in
FileSnapshotTest#testSimulatePackfileReplacement and enables extending
FileSnapshot for packfiles to read the packfile's checksum as another
criterion to detect modifications without reading the full content.
Also add another field capturing the result of the last check if
lastModified was racily clean.
Remove unnecessary determination of raciness in the constructor. It was
determined twice in all relevant cases.
Change-Id: I100a2f49d7949693d7b72daa89437e166f1dc107
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This resolves a regression introduced in fef78212.
Change-Id: Ibb4521635a87012520566efc70870c59d11be874
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Fill out the description of when IOException is thrown.
Also fix a typo in the description for IncorrectObjectTypeException.
Change-Id: I9fafd19d68ddc4fe4e95e8516c2c38484b941a3a
Signed-off-by: Ivan Frade <ifrade@google.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Offer a version of findBranchesReachableFrom method with progress
monitor callback. This is required to allow UI clients to cancel long
running operations and show progress.
Bug: 547642
Change-Id: I31d1de54dbaa6ffb11e03da4c447963e8defa1d0
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
* stable-5.3:
SystemReader: Respect passed-in parent when no system config is present
Change-Id: I758ed0c2a3527ffcf6255158a49090aba090e942
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
The default implementation of openSystemConfig has special handling for
when the FS returns null from getGitSystemConfig: it should return a
"real" FileBasedConfig instance that never actually tries to load a
file. However, this codepath was not respecting the passed-in parent
config.
Change-Id: Id0bcdc93bb42f9ebe3f5ee4c6b4be8863e0133f9
Every caller would need to check if bitmaps are available in the repo to
instantiate a reachability checker.
Offer a method to create the reachability checker in the walk: the
caller has already a walk over the repo, and the walk has all the
information required.
This allows us to make the implementation classes package-private.
Change-Id: I355e47486fcd9d55baa7cb5700ec08dcc230eea5
Signed-off-by: Ivan Frade <ifrade@google.com>
The new ReachabilityChecker interface and its implementations are marked
as @since 5.5, but they will make it to the 5.4 release.
Change-Id: I88c31b3300ccf35d18c35faddb2517f0a57bdcfd
Signed-off-by: Ivan Frade <ifrade@google.com>
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Some filesystems expose unique file identifiers, e.g. inodes in Posix
filesystems which are named filekeys in Java's BasicFileAttributes. Use
them as another means to detect file modifications based on stat
information.
Running git gc on a repository yields a new packfile with the same id as
a packfile which existed before the gc if these packfiles contain the
same set of objects. The content of the old and the new packfile might
differ if a different PackConfig was used when writing the packfile.
Considering filekeys in FileSnapshot may help to detect such packfile
modifications.
Bug: 546891
Change-Id: I711a80328c55e1a31171d540880b8e80ec1fe095
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
FileSnapshot.notRacyClean() assumed a worst case filesystem timestamp
resolution of 2.5 sec (FAT has a resolution of 2 sec). Instead measure
timestamp resolution to avoid unnecessary IO caused by false positives
in detecting the racy git problem caused by finite filesystem timestamp
resolution [1].
Cache the measured resolution per FileStore since timestamp resolution
depends on the respective filesystem type. If timestamp resolution
cannot be measured or fails due to an exception fallback to the worst
case FAT timestamp resolution and avoid caching this value.
Add a 10% safety margin in FileSnapshot.notRacyClean(), though running
FsTest.testFsTimestampResolution() 1000 times which is not using a
safety margin didn't fail on Mac using APFS and Java 8, 11, 12.
Measured Java file timestamp resolution: [2]
[1] https://github.com/git/git/blob/master/Documentation/technical/racy-git.txt
[2] https://docs.google.com/spreadsheets/d/1imy0y6WmRqBf0kjCxzxj2X7M50eIVfa7oaUIzEOHmjo
Bug: 546891
Change-Id: I493f3b57b6b306285ffa7d392339d253e5966ab8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* fix equals() and hashCode() methods to consider size
* fix toString() to show size
Change-Id: I5485db55eda5110121efd65d86c7166b3b2e93d0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Check whether the value of the git config user.signingKey is a suffix
of the full fingerprint of the key. This was already used for finding
keys in secring.gpg, but not in pubring.kbx. This mechanism allows a
user to use any unique suffix to identify keys; to avoid needless
collisions it's recommended to use at least the last 16 characters of
the hex representation of the fingerprint, which is the key id.[1]
[1] https://tools.ietf.org/html/rfc4880#section-12.2
Bug: 545673
Change-Id: If6fb4879502b6ee4b8c26c21b2714aeac4e4670c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
In 7b96bd812e ("UploadPack: Use reachability checker to validate
non-advertised wants", 2019-05-16), a "walk.setRetainBody(false);"
statement was inadvertently deleted. (An earlier version of this commit
had this line in another part of the code and a review comment suggested
to move it back here; the line was then deleted from the other part of
the code but not readded.) Restore this line.
Change-Id: I96ff6106ba9e4eef429388c83e898b3363295f69
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
* changes:
UploadPack: Use reachability checker to validate non-advertised wants
BitmappedReachabilityChecker: Reachability check using bitmaps
BitmapCalculator: Get the reachability bitmap of a commit
ReachabilityChecker: Default implementation with a RevWalk
In "Reachable commit" request validators, we need to check that a "want"
in the request, that hasn't been advertised, is reachable from the refs
visible to the user.
Current code has intermixed the translation of ObjectIds to RevCommits
(and its error handling) with the actual walk, with the delegation to
bitmaps in restricted circunstances.
Refactor the code to make it "flatter" and more readable. Move ObjectIds
to RevCommits translation to its own functions. Use the reachability
checker instead of a newly defined walk.
Before the non-advertised wants were validated with bitmaps only if any
"want" refered to an non-commit. Now they will be validated with bitmaps
also if the "wants" refer all to commits.
Change-Id: Ib925a48cde89672b07a88bba4e24d0457546baff
Signed-off-by: Ivan Frade <ifrade@google.com>
The "basic" reachability check walks the graph starting from the tips
marking things as "uninteresting". If the target commit is marked as
"uninteresting" it was reached; it is reachable from those tips.
This requires a lot of walking and can be solved directly with bitmaps.
Most of the time the bitmaps are already calculated or a short walk
away.
This should improve the performance of reachability checks, for example
in Gitiles.
Change-Id: I83d33271f58d95d2dc9ed151967b3eda513c99f7
Signed-off-by: Ivan Frade <ifrade@google.com>
To make reachability checks with bitmaps, we need to get the
reachability bitmap of a commit, which is not always precalculated.
There is already a class returning such bitmap (BitmapWalker) but it
does too much unnecessary work: it calculates ALL reachable objects from
a commit (i.e. including trees and blobs), when for reachability the
commits are just enough.
Introduce BitmapCalculator to get the bitmap of a commit: either because
it is precalculated or generating it with a walk only over commits.
Change-Id: Ibb6c78affe9eeaf1fa362a06daf4fd2d91c1caea
Signed-off-by: Ivan Frade <ifrade@google.com>
It is common to check if a certain commit is reachable from some
starting points. For example gitiles does it to check if a commit
is visible to a user based on its permissions.
Offer this functionality in JGit.
Split the interface as the next commit will introduce an implementation
using bitmap indices.
Change-Id: I0933b305c8d734f7a64502910ff4d9ef4fc92ae1
Signed-off-by: Ivan Frade <ifrade@google.com>
- ObjectWalk.getTreeDepth() returns int hence there is no need to use
long depths in the lowestDepthVisited map.
- Also fix boxing warnings introduced in 0a15cb3a.
Change-Id: I6d73b6f41d5d20975d02f376c8588e411eaff0ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Replace:
in the "core section"
in the "diff section"
in the "merge section"
with:
in the "core" section
in the "diff" section
in the "merge" section
Change-Id: Ided7bf73e9c8aae4fc4e43d5d5b9f6d7e3066f0a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.3:
Prepare 5.3.2-SNAPSHOT builds
JGit v5.3.1.201904271842-r
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I8a8671f7767444a77b809bd66a27d776c8332736
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.2.3-SNAPSHOT builds
JGit v5.2.2.201904231744-r
Revert 4678f4b and provide another solution for bug 467631
Apache MINA sshd: make sendKexInit() work also for re-keying
Prepare 5.1.8-SNAPSHOT builds
JGit v5.1.7.201904200442-r
ObjectUploadListener: Add callback interface
Prepare 4.11.9-SNAPSHOT builds
JGit v4.11.8.201904181247-r
Prepare 4.9.11-SNAPSHOT builds
JGit v4.9.10.201904181027-r
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: Ie7e572ac7e346f21fe0c387d7448be168a9c127a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Making gitlinks and folders match in a tree walk was the wrong
approach to fix bug 467631. The problem is that in such a conflict
the tree walk may then not descend into the folder.
Revert the changes to Paths.java and PathsTest.java from commit
4678f4b. Instead test for the problem case from bug 467631 explicitly
in IndexDiff. Add Daniel's test case from bug 545162, and add yet
another test case for DiffEntry.scan() that covers the problem
originally reported in bug 545162.
Bug: 545162
Change-Id: Ie2214c5d5ee32ac6596b621f0f1c7b86d38fa9b7
Also-by: Daniel Veihelmann <daniel.veihelmann@gmail.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
* stable-4.8:
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: Idaa789e699f1ef568ea957184d0641355d9e3181
* stable-4.7:
Prepare 4.7.10-SNAPSHOT builds
JGit v4.7.9.201904161809-r
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I7984f68833f2d615399296e53cb9a64e5b4ca6ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.6:
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I6ea51dde6608a4163d681aa1ebf710f06da44b21
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.5:
Prepare 4.5.8-SNAPSHOT builds
JGit v4.5.7.201904151645-r
Remember the cause for invalidating a packfile
Fix API problem filters
Fix pack files scan when filesnapshot isn't modified
Change-Id: I0b4eaa521ebdea83ab18c05915d691c07a575a7d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a tree is visited during pack and filtered out with tree:<depth>, we
may need to include it if it is visited again at a lower depth.
Until now we revisit it no matter what the depth is. Now, avoid
visiting it if it has been visited at a lower or equal depth.
Change-Id: I68cc1d08f1999a8336684a05fe16e7ae51898866
Signed-off-by: Matthew DeVore <matvore@gmail.com>
If we are traversing a tree which is too deep, then there is no need to
traverse the children. Skipping children is much faster than traversing
the possibly thousands of objects which are directly or indirectly
referenced by the tree.
Change-Id: I6d68cc1d35da48e3288b9cc80356a281ab36863d
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This is used when fetching, and in particular to populate a partial
clone or a virtual file system cache as the user navigates. With this,
a client can pre-fetch a few directories deeper than only the current
directory.
depth:0 will omit all trees, and is useful if you only want to fetch
the commits of a repository, or fetch just a single tree or blob object.
depth:1 will fetch only the root tree of all commits fetched. depth:2
will fetch the root tree and all blobs and tree objects directly
referenced from it. depth:3 gets one more level, and so on. depth:#
will not filter a blob or tree that is explicitly marked wanted.
Bitmaps are disabled when this filter is used.
This implementation is quite slow because it iterates over all omitted
objects rather than skipping them. This will be addressed in follow-up
commits.
Change-Id: Ic312fee22d60e32cfcad59da56980e90ae2cae6a
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This reverts commit 39b0b51b12. Before
that change, SubmoduleWalk.forPath transferred ownership to the caller
on success. Afterward, it returns a closed SubmoduleWalk to the caller,
which does not appear to be intentional.
Change-Id: I9381daac5153706e24fd9117700089848b58c54e
Replace simple uses of Iterator with a corresponding for-loop.
Also add missing braces on loops as necessary.
Change-Id: I708d82acdf194787e3353699c07244c5ac3de189
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
DepthGenerator marks commits reinteresting for the ones that are
reachable from unshallow commits as it walks over the revisions. Those
unshallow commits won't necessarily be processed first. Because of this,
even if a commit is reachable from unshallow commits, if it's processed
before the uninteresting commits, it will not be processed as
reinteresting and processed as uninteresting. This causes unshallow
git-fetch to be failed.
This changes DepthGenerator to process unshallow commits first
independent to their depth. This makes uninteresting flag carry work
properly.
Change-Id: I94378271cf85fbe6302cefc19a167d8cf68e1a69
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
Keep track of the original cause for a packfile invalidation.
It is needed for the sysadmin to understand if there is a real
underlying filesystem problem and repository corruption or if it is
simply a consequence of a concurrency of Git operations (e.g. repack
or GC).
Change-Id: I06ddda9ec847844ec31616ab6d17f153a5a34e33
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since Java 7 the diamond operator can be used instead of explicit
type parameters.
Change-Id: I2dee5fce7afebb1d9088eeaec4484ee58b4fa492
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Use of String.equals("") can be replaced with with String.length() == 0
(for JDK5 and lower) or String.isEmpty() (for JDK6 and higher)
Change-Id: Id1462d22c5d249485d87993263a9239809e73c55
Signed-off-by: Carsten Hammer <carsten.hammer@t-online.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Do not reload packfiles when their associated filesnapshot is not
modified on disk compared to the one currently stored in memory.
Fix the regression introduced by fef78212 which, in conjunction with
core.trustfolderstats = false, caused any lookup of objects inside
the packlist to loop forever when the object was not found in the pack
list.
Bug: 546190
Change-Id: I38d752ebe47cefc3299740aeba319a2641f19391
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add resolveTipSha1, an inverse of exactRef(String ...), to RefDatabase
and provide a default implementation that runs in O(n) time where n is
the number of refs. For RefTable, provide an implementation that runs
in O(log(n)) time.
[ifrade@google.com: with tests in InMemoryRepositoryTest to exercise
the reftable code path, too]
Change-Id: I2811ccd0339cdc1c74b42cce2ea003f07a2ce9e1
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Ivan Frade <ifrade@google.com>
Use Files.delete() instead of File.delete(), and if there is
an exception thrown propagate it unless errors are to be ignored so
that the actual deletion failure cause is available to the caller
(and will be logged).
Change-Id: I5fdb5a4052942437ab365289ad4bb1b563c29456
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Add missing source bundles for
- org.eclipse.jgit.ant
- org.eclipse.jgit.archive
- org.eclipse.jgit.http.apache
- org.eclipse.jgit.http.server
- org.eclipse.jgit.junit
- org.eclipse.jgit.junit.http
- org.eclipse.jgit.junit.ssh
- org.eclipse.jgit.lfs
- org.eclipse.jgit.lfs.server
- org.eclipse.jgit.ui
Combine all source bundles into a single source feature
org.eclipse.jgit.source and delete the other source features.
Ensure all bundles are added to the jgit p2 repository.
Change-Id: I56785f49c940b79f41f763c26e63a4a820ed7cce
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The new references created in the peeling do not receive the update
index. In other words, the update index of a reference (if set) is lost
in the peeling.
Pass-through the update index to the newly created references.
Tested via InMemoryRepository, which uses DfsReftableDatabase.
Change-Id: I7ff7c737a9c3366fdec296a4d9b2e51d10227957
Signed-off-by: Ivan Frade <ifrade@google.com>
Code that creates references and wants to support ref dbs with and
without update indexes needs to set this value. This leds to a
proliferation of "-1" in the code base.
Make the "undefined" value a constant in the ref interface.
Change-Id: I2622a37536a84b4a4036dd55792e185486fa0628
Signed-off-by: Ivan Frade <ifrade@google.com>
- UploadPack.getFilterBlobLimit() didn't have a wrong @since tag but was
marked final shortly after 5.3 was released. This was probably caused by
using an outdated API baseline. Currently we should use 5.3.0 as the
baseline.
- remove unused filter on FS.fileAttributes()
Change-Id: I9adc1703e99a9ddb3ea2a1c12a83dccbc1f69a99
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Use .api_filters to suppress API baseline errors. The justification is
included in comments in that file.
Change-Id: I7b5c69d8d13733719b28ef7317fc9780ed2828d8
Signed-off-by: Matthew DeVore <matvore@gmail.com>
Inline newTreeVisit into enterTree and call the new method pushTree. Use
pushTree both for pushing children of the existing currVisit.
Change-Id: I75ea37f48b2befb738a3e88bed40ac08f1df9a03
Signed-off-by: Matthew DeVore <matvore@gmail.com>
Use the FilterSpec object so that less code has to know about the make-up of
FilterSpecs. When fields are added to FilterSpec, these pieces of code won't
need updating again.
Change-Id: I2b9e59a9926ff112faf62a3fa2d33c961a1779e5
Signed-off-by: Matthew DeVore <matvore@gmail.com>
This increases type-safety and is ground work for support of the
"tree:<depth>" filter.
Change-Id: Id19eacdcdaddb9132064c642f6d554b1060efe9f
Signed-off-by: Matthew DeVore <matvore@gmail.com>
When using @NonNull annotations in new code, if I write
public void setFrobber(@NonNull frobber) {
this.frobber = frobber;
}
then consumers of the JGit library that do not have nullness checking
enabled can easily pass in null by mistake. On the other hand, if I
write
public void setFrobber(@NonNull frobber) {
if (frobber == null) {
throw new NullPointerException();
}
this.frobber = frobber;
}
then Eclipse JDT complains:
Null comparison always yields false: The variable frobber is specified as @NonNull
Add a checkNotNull helper that offers the best of both worlds:
public void setFrobber(@NonNull frobber) {
this.frobber = checkNotNull(frobber);
}
Briefer code, null check is intact, and no warning.
Inspired by Guava's com.google.common.base.Preconditions.checkNotNull.
Change-Id: If59588d13a1119e899657ed2296931ea18ed0e2a
This code is not specific to V2 of the protocol, so it doesn't belong
there. Move it to its own class.
Change-Id: I8084764147d1d0e2bceaaead59b332a8ec1a0825
Signed-off-by: Matthew DeVore <matvore@gmail.com>
HashMap<String, Ref> has a memory overhead for refs. Use RefMap.
Change-Id: I3fb4616135dacf687cc3bc2b473effc66ccef5e6
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
The prune method did not delete empty fanout directories when loose
objects moved to a new pack file but only when loose unreferenced
objects were pruned.
Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Ensure the sentences to document fields consistenly end with a dot.
Change-Id: I084899cafe4dea6f2424132dce9f3adca2e20985
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
One occurrence in core JGit, several in jgit.pgm. One unused
occurrence in jgit.pgm; remove it.
Change-Id: I04c3dd9d9f542f1e1ac6df4cbf03bcefb6bfdf78
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
RefAdvertiser.send(Map<String, Ref>) is deprecated in favour of
RefAdvertiser.send(Collection<Ref>). Subclasses that need to override
the "send" method need to override also the deprecated version, because
it is still invoked by BaseReceivePack and UploadPack.
Remove the last usages of the deprecated method.
Change-Id: I7eba426970251f78801ddf96b87a65d1baaebdcf
Signed-off-by: Ivan Frade <ifrade@google.com>
* stable-5.3:
Prepare 5.3.1-SNAPSHOT builds
JGit v5.3.0.201903130848-r
Prepare 5.1.7-SNAPSHOT builds
JGit v5.1.6.201903130242-r
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ief9f6a9e89c4a1ef122daedbfd933676393fab06
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Prepare 5.1.7-SNAPSHOT builds
JGit v5.1.6.201903130242-r
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ia866587a910e620cb297bcb2a81fe0786f4e3db2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Prepare 5.1.7-SNAPSHOT builds
JGit v5.1.6.201903130242-r
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I8da2761ed27ff2817a4136f843b9b293e3c0f4e6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.0:
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I71771dcc086a9a0097bad74d65ec71ad97106a38
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.11:
Prepare 4.11.8-SNAPSHOT builds
JGit v4.11.7.201903122105-r
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I04c70f66c6f1e53bdc199fb5b24c9136c9a8e4f7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.10:
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ic66dc00455c05f7627919f682a7221ffeef1629d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.9:
Prepare 4.9.10-SNAPSHOT builds
JGit v4.9.9.201903122025-r
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: Ifcd8ea6a2b59f6f8e9d896d8407cfcaef0a9375d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.8:
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.7:
Prepare 4.7.9-SNAPSHOT builds
JGit v4.7.8.201903121755-r
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.6:
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-4.5:
Prepare 4.5.7-SNAPSHOT builds
JGit v4.5.6.201903121547-r
Check for packfile validity and fd before reading
Move throw of PackInvalidException outside the catch
Use FileSnapshot to get lastModified on PackFile
Include size when comparing FileSnapshot
Do not reuse packfiles when changed on filesystem
Silence API warnings for new API introduced for fixes
Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When reading from a packfile, make sure that is valid
and has a non-null file-descriptor.
Because of concurrency between a thread invalidating a packfile
and another trying to read it, the read() may result into a NPE
that won't be able to be automatically recovered.
Throwing a PackInvalidException would instead cause the packlist
to be refreshed and the read to eventually succeed.
Bug: 544199
Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
When a packfile is invalid, throw an exception explicitly
outside any catch scope, so that is not accidentally caught
by the generic catch-all cause, which would set the packfile
as valid again.
Flagging an invalid packfile as valid again would have
dangerous consequences such as the corruption of the in-memory
packlist.
Bug: 544199
Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Do not redundantly call File.lastModified() for extracting the
timestamp of the PackFile but rather use consistently the FileSnapshot
which reads all file attributes in a single bulk call.
Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Read and consider file size also, so that differing file size can help
to more accurately detect file changes without reading the file content.
Use bulk read to avoid multiple stat calls to retrieve file attributes.
Change-Id: I974288fff78ac78c52245d9218b5639603f67a46
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The pack reload mechanism from the filesystem works only by name
and does not check the actual last modified date of the packfile.
This lead to concurrency issues where multiple threads were loading
and removing from each other list of packfiles when one of those
was failing the checksum.
Rely on FileSnapshot rather than directly checking lastModified
timestamp so that more checks can be performed.
Bug: 544199
Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
* stable-5.3:
Reduce contention on PackFile.idx() function.
Use SystemReader in JSchConfigSessionFactoryTest
Avoid NPE in ObjectId.isId()
Change-Id: I1d13f6fb705258ae6d6e5fa5e733bfacd4f3d0e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.
Performance numbers are listed in the bug report.
Bug: 543739
Change-Id: Iff328d347fa65ae07ecce3267d44184161248978
Signed-off-by: Juergen Denner <j.denner@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
That method can easily be invoked with a null argument (e.g.
isId(repo.getFullBranch()), therefore it should handle null arguments.
Change was suggested in https://git.eclipse.org/r/#/c/137918/, which
tried to fix the same in egit only.
Bug:544982
Change-Id: I32d1df6e9b2946ab324eda7008721159019316b3
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
The commit body contains the commit message, which is not needed for
reachability checks.
Change-Id: Ie209c3b3f022579942f05b8b5d0625ce26400a5d
Signed-off-by: Jonathan Nieder <jrn@google.com>
The visibility change was merged after 5.3 release. The @since line
is wrong.
Fixing it in a new commit looks like the easiest way forward.
Change-Id: I6f72a9e684e99a4440cda1eb532c22b4b7bbdea5
Signed-off-by: Ivan Frade <ifrade@google.com>
Commit body contains the message that is not needed for reachability checks, and
takes up memory unnecessarily.
Change-Id: I0c7f6da249bf9c4fda9dc9e62e809322c68effce
Signed-off-by: Minh Thai <mthai@google.com>
This can help track down poor long tail performance that isn't accounted
for in the readIdxMicros or readBlockMicros metrics.
Change-Id: I701b9cfcc124f4ddb860d1766a11ea3557e604ce
Signed-off-by: Terry Parker <tparker@google.com>
* stable-4.11:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.10:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.
Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce59)
Externalize the message and log the pack file with absolute path.
Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit 9665d86ba1)
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.
Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
(cherry picked from commit 962babc4b2)
* stable-5.2:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
Change-Id: Ib972641105cef8089791fc65389c4f43e218620a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.1:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
Change-Id: I42b72a3bba3b4c9389d91a1a35a8004836567e7c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.
Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Externalize the message and log the pack file with absolute path.
Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Previously, we called destroy() to delete the temp file on failure only
when catching an IOException, not a RuntimeException. Use a slightly
different construction with a finally block to ensure it's always
deleted on error (assuming the JVM is still healthy enough).
Change-Id: Ie201f3cfc81099ee1cafed066632da76223cef1f
There are valid cases where a hook, invoked by ProtocolV2Hook
and probably implemented in a different package, is interested
in knowing the wanted refs in the request.
Increase visibility to public in the wanted-ref method.
Change-Id: I5da085ac7af4c396c1cb85e630f40a57fc70b33e
Signed-off-by: Ivan Frade <ifrade@google.com>
Similar to UploadPack.getDepth() to know the shallow clone depth, expose
the user-specified filter blob limit for partial clones.
Change-Id: I04bde06862a1cf8a9862d950c15023c49d16a2a6
Signed-off-by: Terry Parker <tparker@google.com>
* changes:
RebaseCommand: tighten check for --preserve-merges on --continue
RebaseCommand: use orig-head to abort
RebaseCommand: use orig-head in addition to head
Hard-coding ~/.gnupg for the GPG directory doesn't work on Windows,
where GnuPG uses %APPDATA%\gnupg by default. Make the determination
of the directory platform-dependent.
Bug: 544797
Change-Id: Id4bfd39a981ef7c5b39fbde46fce9a7524418709
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Instead of a new "unexpectedNlinkValue" message use the already
existing "failedAtomicFileCreation". Remove a stray double quote
from the latter.
Change-Id: I1ba5e9ea48d3f7615354b2ace2575883070b3206
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
With native git, .git/rebase-merge/rewritten exists actually in two
different cases:
* as a file in git rebase --merge recording OIDs for copying notes
* as a directory in git rebase --preserve-merges
Add a comment, and check for isDirectory() instead of exists().
Bug: 511487
Change-Id: I6a3317b4234d4f41c41b3004cdc7ea0abf2c6223
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
ONTO_NAME must be "onto_name", not "onto-name".
For native git, --preserve-merges is an interactive mode. Create the
INTERACTIVE marker file, otherwise a native git rebase --continue
will fall back into rebase --merge mode before git 2.19.0 since it
started looking for the REWRITTEN directory to make the distinction
only then.[1]
This allows a JGit interactive rebase to be continued via native git
rebase --continue.
[1] https://github.com/git/git/commit/6d98d0c0
Bug: 511487
Change-Id: I13850e0fd96ac77d03fbb581c8790d76648dbbc6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Aborting a rebase used ORIG_HEAD to reset. Strictly speaking this is
not correct, since other commands run during the rebase (for instance,
when the rebase stopped on a conflict) might have changed ORIG_HEAD.
Prefer the OID recorded in the orig-head file, falling back to the
older "head" file if "orig-head" doesn't exist, and use ORIG_HEAD only
if neither exists.
Bug: 511487
Change-Id: Ifa99221bb33e4e4754377f9b8f46e76c8936e072
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
With text=auto or core.autocrlf=true, git does not normalize upon
check-in if the file in the index contains already CR/LFs. The
documentation says: "When text is set to "auto", the path is
marked for automatic end-of-line conversion. If Git decides that
the content is text, its line endings are converted to LF on
checkin. When the file has been committed with CRLF, no conversion
is done."[1]
Implement the last bit as in canonical git: check the blob in the
index for CR/LFs. For very large files, we check only the first 8000
bytes, like RawText.isBinary() and AutoLFInputStream do.
In Auto(CR)LFInputStream, ensure that the buffer is filled as much as
possible for the isBinary() check.
Regarding these content checks, there are a number of inconsistencies:
* Canonical git considers files containing lone CRs as binary.
* RawText checks the first 8000 bytes.
* Auto(CR)LFInputStream checks the first 8096 (not 8192!) bytes.
None of these are changed with this commit. It appears that canonical
git will check the whole blob, not just the first 8k bytes. Also
note: the check for CR/LF text won't work with LFS (neither in JGit
nor in git) since the blob data is not run through the smudge filter.
C.f. [2].
Two tests in AddCommandTest actually tested that normalization was
done even if the file was already committed with CR/LF.These tests
had to be adapted. I find the git documentation unclear about the
case where core.autocrlf=input, but from [3] it looks as if this
non-normalization also applies in this case.
Add new tests in CommitCommandTest testing this for the case where
the index entry is for a merge conflict. In this case, canonical git
uses the "ours" version.[4] Do the same.
[1] https://git-scm.com/docs/gitattributes
[2] https://github.com/git/git/blob/3434569fc/convert.c#L225
[3] https://github.com/git/git/blob/3434569fc/convert.c#L529
[4] https://github.com/git/git/blob/f2b6aa98b/read-cache.c#L3281
Bug: 470643
Change-Id: Ie7310539fbe6c737d78b1dcc29e34735d4616b88
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Since 2011-02-10 (i.e., git 1.7.6)[1] native git uses "orig-head" for
REBASE_HEAD. JGit was still using "head". Currently native git has a
legacy fall-back for reading this, but for how long? Let's write to
both. Note that JGit never reads this file.
[1] https://github.com/git/git/commit/84df4560
Bug: 511487
Change-Id: Id3742bf9bbc0001d850e801b26cc8880e646abfc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The non-externalized warning message says there is a "possible SHA-1
collision" but then the Sha1CollisionException is always thrown.
Replace the message with the existing externalised string that does
not say "possible".
Change-Id: I9773ec76b416c356e234a658fb119f98d33eac83
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
DfsBlockCache.Ref might get cleared out if the JVM is running out of
memory. As a result, the index is not persisted for the request and
will be reloaded unnecessarily.
Change-Id: I3b57ad5e6673f77f2dc66177a649ac412a97fe20
Signed-off-by: Minh Thai <mthai@google.com>
Deprecate the method in favor of setEncoding(Charset).
Update the only caller in the code base that was still using
the deprecated variant.
Change-Id: I6357f2d0c727007013c72e9d5b7c72a3f5f3f2b1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.
Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Bug caused the pack to be 12 bytes short when cold cache. Also added
test for copyPackAsIs method.
Change-Id: Idf8fb0e50d1215245d4b032e2e00df4b218c115f
Signed-off-by: Minh Thai <mthai@google.com>
Android for instance forbids hard linking via a SELinux
policy. If we can't hard link, the NFS work-around for
atomic file creation cannot work at all. In this case,
fall back to not using the hard-linking mechanism.
Android throws an AccessDeniedException, so we catch that.
The javadoc on Files.createLink() indicates that another
possibility might be a SecurityException, so catch that,
too.
Bug: 543956
Change-Id: I551b7a45f7b2fbbd8cf94f0b7233dbd8a200520e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
This method tried to iterate spurious files which may exist in the
.git/refs folder, e.g. on Mac a .DS_Store may have been created there by
inspecting the folder using the finder application. This led to a
NotDirectoryException when deleteEmptyRefsFolders tried to create an
iterator for such a file entry. Skip files contained in the refs folder
to ensure the method only tries to iterate contained folders but not
files.
Change-Id: I5f31e733072a35db1e93908a9c69a8891ae5c206
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Single-branch-clone should be able to clone a single tag. Enhance
CloneCommand to accept also full refs of tags in setBranchesToClone().
Make sure we also include fetch ref specs for the fetch command for
tags. This mimics the behavior of native git's single-branch clone:
git clone --branch <tag> --single-branch <URI>
Bug: 542611
Change-Id: I285cf043751d9b0ba71258ee8214c0e5d1191428
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Several problems:
* The command didn't specify whether it expected short or full names.
* For the new name, it expected a short name, but then got confused
if tags or both local and remote branches with the same name existed.
* For the old name, it accepted either a short or a full name, but
again got confused if a short name was given and a tag with the
same name existed.
With such an interface, one cannot use Repository.findRef() to
reliably find the branch to rename. Use exactRef() for the new
name as by the time the Ref is needed its full name is known.
For determining the old Ref from the name, do the resolution
explicitly: first try exactRef (assuming the old name is a full
name); if that doesn't find anything, try "refs/heads/<old>" and
"refs/remotes/<old>" explicitly. Throw an exception if the name
is ambiguous, or if exactRef returned something that is not a
branch (refs/tags/... or also refs/notes/...).
Document in the javadoc what kind of names are valid, and add tests.
A user can still shoot himself in the foot if he chooses exceptionally
stupid branch names. For instance, it is still possible to rename a
branch to "refs/heads/foo" (full name "refs/heads/refs/heads/foo"),
but it cannot be renamed further using the new short name if a branch
with the full name "refs/heads/foo" exists. Similar edge cases exist
for other dumb branch names, like a branch with the short name
"refs/tags/foo". Renaming using the full name is always possible.
Bug: 542446
Change-Id: I34ac91c80c0a00c79a384d16ce1e727c550d54e9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The new API is intended for UIs to check if signing will be possible or
would fail
Bug: 543579
Change-Id: I6ce1fd4210e46d49dcdf420c99d08c93e022136c
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
BundleFetchConnection.readLine() must abort on EOF, otherwise
it gets stuck in an endless loop.
Bug: 543390
Change-Id: I4cb3428560277888af114b928950d620bb6564f9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
The import is only needed because of a reference to it in the Javadoc,
and can be avoided by explicitly specifying the package instead, which
is how it's referenced in other cases (Constants, FileHeader).
Change-Id: I0c6254a9adf1f52fb8f2c04a858b11696ad264f5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
This also includes a change to generating the jgit CLI jar. Shading is
no longer possible because it breaks the signature of BouncyCastle.
Instead, the Spring Boot Loader Maven plug-in is now used to generate an
executable jar.
Bug: 382212
Change-Id: I35ee3d4b06d9d479475ab2e51b29bed49661bbdc
Also-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This change introduces the concept of a GpgSigner which will sign
commits. The GpgSigner will be of a specific implementation (eg.,
Bouncycastle or OpenPgP executable). The actual implementation is not
part of this change.
Bug: 382212
Change-Id: Iea5da1e885c039e06bc8d679d46b124cbe504c8e
Also-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Medha Bhargav Prabhala <mprabhala@salesforce.com>
Signed-off-by: Gunnar Wagenknecht <gunnar@wagenknecht.org>
As explained in 'The "Double-Checked Locking is Broken"
Declaration'[*], Java's memory model does not support double-checked
locking:
class LazyReadableChannel {
private ReachableChannel rc = null;
public ReadableChannel get() {
if (rc == null) {
synchronized (this) {
if (rc == null) {
rc = new ReadableChannel();
}
}
}
return rc;
}
}
With JDK 5 and newer, there is a formal memory model that ensures this
works if "rc" is volatile, but it is still not thread-safe without
that.
Fortunately, this ReadableChannelSupplier is never passed between
threads, so it does not need to be thread-safe. Simplify by removing
the synchronization.
[*] https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html
Change-Id: I0698ee6618d734fc129dd4f63fc047c1c17c94a9
Signed-off-by: Jonathan Nieder <jrn@google.com>
This allows scanning through refs once instead of once per ref, which
should make the lookup less expensive for some RefDatabase
implementations.
Change-Id: I1434f834186cc9a6b4e52659e692b1000c926995
Signed-off-by: Jonathan Nieder <jrn@google.com>
To avoid opening the readable channel in case of DfsBlockCache
hits. Also cleaning up typos around DfsBlockCache.
Change-Id: I615e349cb4838387c1e6743cdc384d1b81b54369
Signed-off-by: Minh Thai <mthai@google.com>
Opening a readable channel can be expensive and the number of channels
can be limited in DFS. Ensure that caller of
BlockBasedFile.readOneBlock() is responsible for opening and closing
the file, and that the ReadableChannel is reused in the request. As a side
effect, this makes the code easier to read, with better use of
try-with-resources.
The downside is that this means a readable channel is always opened, even
when the entire pack is already available for copying from cache. This
should be an acceptable cost: it's rare enough not to overload the server
and from a client latency perspective, the latency cost is in the noise
relative to the data transfer cost involved in a clone. If this turns out
to be a problem in practice, we can reintroduce that optimization in a
followup change.
Change-Id: I340428ee4bacd2dce019d5616ef12339a0c85f0b
Signed-off-by: Minh Thai <mthai@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
We see the same index being loaded by multiple threads. Each is
hundreds of MB and takes several seconds to load, causing server to
run out of memory. This change introduces a lock to avoid these
duplicate works. It uses a new set of locks similar in implementation
to the loadLocks for getOrLoad of blocks. The locks are kept separate
to prevent long-running index loading from blocking out fast block
loading. The cache instance can be configured with a consumer to
monitor the wait time of the new locks.
Change-Id: I44962fe84093456962d5981545e3f7851ecb6e43
Signed-off-by: Minh Thai <mthai@google.com>
CheckoutCommand had a setForce() method. But this didn't correspond
to native git's 'git checkout -f' option. Deprecate the old setForce()
method and move its implementation to a new method setForceRefUpdate()
and use it to implement the -B option in the CLI class Checkout.
Add a setForced() method and use it to fix the associated '-f' option of
the CLI Checkout class to behave like native git's 'git checkout -f'
which overwrites dirty worktree files during checkout.
This is still not fully matching native git's behavior: updating
additionally dirty index entries is not done yet.
Bug: 530771
Change-Id: I776b78eb623b6ea0aca42f681788f2e4b1667f15
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
To avoid breaking ABI, take the opportunity to give these setters
(hopefully sometimes better) names and deprecate their old names.
Change-Id: Ib45011678c3d941f8ecc1a1e0fdf4c09cdc337e3
Signed-off-by: Mario Molina <mmolimar@gmail.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
Swallowing intermittent errors and trying to recover from them
makes JGit's behavior hard to predict and difficult to debug.
Propagate the errors instead. This doesn't violate JGit's usual
backward compatibility promise for clients because in these
contexts an IOException indicates either repository corruption or
a true I/O error. Let's consider these cases one at a time.
In the case of repository corruption, falling back e.g. to an empty
set of refs or a missing ref will not serve a caller well. The
fallback does not indicate the nature of the corruption, so they are
not in a good place to recover from the error. This is analogous to
Git, which attempts to provide sufficient support to recover from
corruption (by ensuring commands like "git branch -D" cope with
corruption) but little else.
In the case of an I/O error, the best we can do is to propagate the
error so that the user sees a dialog and has an opportunity to try
again. As in the corruption case, the fallback behavior does not
provide enough information for a caller to rely on the current error
handling, and callers such as EGit already need to be able to handle
runtime exceptions.
To be conservative, keep the existing behavior for the deprecated
Repository#peel method. In this example, the fallback behavior is to
return an unpeeled ref, which is distinguishable from the ref not
existing and should thus at least be possible to debug.
Change-Id: I0eb58eb8c77519df7f50d21d1742016b978e67a3
Signed-off-by: Jonathan Nieder <jrn@google.com>
Its implementation contains
} catch (IOException e) {
// Legacy API, assume error means "no"
return false;
}
Better to use ObjectDatabase#has, which throws IOException to report
errors.
Change-Id: I7de02f7ceb8f57b2a8ebdb16d2aa4376775ff933
Signed-off-by: Jonathan Nieder <jrn@google.com>
That constant is just a redirection to a java standard constant
meanwhile. It is not referenced anymore in jgit code (and egit is just
removing it). Clients can use the redirection target directly.
Change-Id: I058d013f61da8d7b771c499d8743aafb8faa5ea8
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
A checkout done directly after cloning (the "initial
checkout") has a different semantic as a default
checkout. That is defined in the documentation of
"git read-tree" [1]. JGit was detecting that it is
doing an initial checkout differently from native
git: jgit used to check that the index is empty
but native git required that the index file does
not exist [2]. Teach JGit to behave like native
git.
[1] https://github.com/git/git/blob/master/Documentation/git-read-tree.txt#L187
[2] https://marc.info/?t=154150811200001&r=1&w=2
Change-Id: I1dd0f1ede7cd7ea60d28607916d0165269a9f628
and allow package org.eclipse.jgit.http.server to use package
org.eclipse.jgit.internal.transport.parser.
Change-Id: Ief330c3e75a735853d0a5a265a9ff56fb5128b99
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: If861e28ce512f556e574352fa7d4a0df0984693f
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Another step toward merging BaseReceivePack into ReceivePack.
Change-Id: I43cf2e36e2d5b0cd85bf23c81469909c14757b63
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Another step toward eliminating BaseReceivePack as a separate API.
Change-Id: If7b7d5c65a043607a2424211adb479fa33a9077b
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This is a first step toward eliminating the BaseReceivePack API.
Inspired by a larger change by Dan Wang <dwwang@google.com>.
Change-Id: I5c876a67d8db24bf808823d9ea44d991b1ce5277
Signed-off-by: Jonathan Nieder <jrn@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
RefDatabase.getRef was declared final in
c1954f6c36
which only affects implementers.
Change-Id: I4c14232a119670d263d88db2b8d725dcdd36ab2a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This simplifies the BaseReceivePack class and decreases its API
surface, which should make merging with ReceivePack easier.
Inspired by 6aca8899a5 (Move first line
parsing for v0/v1 pack negotiation out of UploadPack, 2018-09-17).
Change-Id: I1fc175d15aa7cb5968c26fc83a95480403af617c
David Pursehouse