When reading from a packfile, make sure that is valid
and has a non-null file-descriptor.
Because of concurrency between a thread invalidating a packfile
and another trying to read it, the read() may result into a NPE
that won't be able to be automatically recovered.
Throwing a PackInvalidException would instead cause the packlist
to be refreshed and the read to eventually succeed.
Bug: 544199
Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
When a packfile is invalid, throw an exception explicitly
outside any catch scope, so that is not accidentally caught
by the generic catch-all cause, which would set the packfile
as valid again.
Flagging an invalid packfile as valid again would have
dangerous consequences such as the corruption of the in-memory
packlist.
Bug: 544199
Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Do not redundantly call File.lastModified() for extracting the
timestamp of the PackFile but rather use consistently the FileSnapshot
which reads all file attributes in a single bulk call.
Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Due to finite filesystem timestamp resolution the last modified
timestamp of files cannot detect file changes which happened in the
immediate past (less than one filesystem timer tick ago).
Read and consider file size also, so that differing file size can help
to more accurately detect file changes without reading the file content.
Use bulk read to avoid multiple stat calls to retrieve file attributes.
Change-Id: I974288fff78ac78c52245d9218b5639603f67a46
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The pack reload mechanism from the filesystem works only by name
and does not check the actual last modified date of the packfile.
This lead to concurrency issues where multiple threads were loading
and removing from each other list of packfiles when one of those
was failing the checksum.
Rely on FileSnapshot rather than directly checking lastModified
timestamp so that more checks can be performed.
Bug: 544199
Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
* stable-5.3:
Reduce contention on PackFile.idx() function.
Use SystemReader in JSchConfigSessionFactoryTest
Avoid NPE in ObjectId.isId()
Change-Id: I1d13f6fb705258ae6d6e5fa5e733bfacd4f3d0e3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In case of concurrent pack file access, threads may wait on the idx()
function even for already open files. This happens especially with a
slow file system.
Performance numbers are listed in the bug report.
Bug: 543739
Change-Id: Iff328d347fa65ae07ecce3267d44184161248978
Signed-off-by: Juergen Denner <j.denner@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This isolates the test from the concrete system it's running on.
SshSessionFactory reads the user also through SystemReader.
Change-Id: I1c796aa1c498fe3967456d8589e6be0a82ab8f44
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
That method can easily be invoked with a null argument (e.g.
isId(repo.getFullBranch()), therefore it should handle null arguments.
Change was suggested in https://git.eclipse.org/r/#/c/137918/, which
tried to fix the same in egit only.
Bug:544982
Change-Id: I32d1df6e9b2946ab324eda7008721159019316b3
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
The commit body contains the commit message, which is not needed for
reachability checks.
Change-Id: Ie209c3b3f022579942f05b8b5d0625ce26400a5d
Signed-off-by: Jonathan Nieder <jrn@google.com>
The visibility change was merged after 5.3 release. The @since line
is wrong.
Fixing it in a new commit looks like the easiest way forward.
Change-Id: I6f72a9e684e99a4440cda1eb532c22b4b7bbdea5
Signed-off-by: Ivan Frade <ifrade@google.com>
Commit body contains the message that is not needed for reachability checks, and
takes up memory unnecessarily.
Change-Id: I0c7f6da249bf9c4fda9dc9e62e809322c68effce
Signed-off-by: Minh Thai <mthai@google.com>
* stable-5.3:
Prepare 5.3.0-SNAPSHOT builds
JGit v5.3.0.201903061415-rc1
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I8c7dfd5141be0d27547d65f19054f9a1665b55fb
This can help track down poor long tail performance that isn't accounted
for in the readIdxMicros or readBlockMicros metrics.
Change-Id: I701b9cfcc124f4ddb860d1766a11ea3557e604ce
Signed-off-by: Terry Parker <tparker@google.com>
* master:
Update Orbit to R20190226160451
Upgrade maven-javadoc-plugin to 3.1.0
diff: add option metaVar for --src-prefix and --dst-prefix
ResolveMerger: Destroy TemporaryBuffer on unchecked exceptions
Expose the filter blob limit in UploadPack
Upgrade error_prone_core to 2.3.3
On Windows use %APPDATA%\gnupg as GPG directory if it exists
Remove duplicate externalized message
RebaseCommand: tighten check for --preserve-merges on --continue
RebaseCommand: fix ONTO_NAME, and --preserve-merges is interactive
RebaseCommand: use orig-head to abort
Fix core.autocrlf for non-normalized index
RebaseCommand: use orig-head in addition to head
SHA1: Use externalized message in log
JGitText: Remove unnecessary suffix from externalized message identifier
FS_POSIX: Externalize log message
Strongly reference indices in DfsPackFile
Update Orbit to S20190219190953
Update README
Update README
Add missing @since tag for new API method getUpdateIndex
Fix NPE in PlotCommitList
CommitBuilder: Deprecate setEncoding(String)
CommitBuilder: Add missing periods on methods' Javadoc
Upgrade wagon-ssh to 3.3.2
Fix bug in copyPackBypassCache's skip 'PACK' header logic
Upgrade spotbugs to 3.1.11
Atomic file creation: hard-linking may not be allowed
Update Orbit to S20190129210011
Fix GC.deleteEmptyRefsFolders
Enable cloning only specific tags
Delete jgit-4.5 target platform
Add 4.11-staging target platform and update Orbit to I20190123233226
Upgrade jacoco-maven-plugin to 0.8.3
Bazel: Format BUILD file with buildifier
RenameBranchCommand: more consistent handling of short ref names
SshdSessionFactory: generalize providing default keys
Allow to check for signing key
Handle premature EOF in BundleFetchConnection
pgm: Fix missing braces in Version.run()
pgm: Handle IOException in Version command
pgm: Fix missing braces in UploadPack.run()
pgm: Handle IOException in UploadPack command
pgm: Handle exceptions in Tag command
pgm: Fix missing braces in Status.run()
pgm: Handle exceptions in Status command
pgm: Fix missing braces in ShowRef.run()
pgm: Handle IOException in ShowRef command
pgm: Fix missing braces in Show.run()
pgm: Handle exceptions in Show command
pgm: Fix missing braces in Rm command
pgm: Handle GitAPIException in Rm command
pgm: Handle exceptions in RevParse command
pgm: Externalize error message
pgm: Fix missing braces in Reset.run()
pgm: Handle GitAPIException in Reset command
pgm: Handle GitAPIException in Repo command
pgm: Handle exceptions in Remote command
pgm: Handle exceptions in Reflog command
pgm: Handle IOException in ReceivePack command
SmartClientSmartServerTest: Open Repository in try-with-resource
SmartClientSmartServerTest: Open ObjectInserter.Formatter in try-with-resource
SmartClientSmartServerTest#addBrokenContext: Remove unused TestRepository
IO: Open TemporaryBuffer.Heap in try-with-resource
GitSmartHttpTools: Open SideBandOutputStream in try-with-resource
TemporaryBufferTest: Open TemporaryBuffer in try-with-resource
Scanner: Open RevWalk in try-with-resource
SubmoduleWalkTest: Open Repository in try-with-resource
SideBandOutputStreamTest: Open SideBandOutputStream in try-with-resource
UnionInputStreamTest: Open UnionInputStream in try-with-resource
Merge: Avoid non-localised literal string warning for "recursive"
pgm: Fix missing braces in Push.run()
pgm: Handle exceptions in Push command
pgm: Fix missing braces in MergeBase.run()
pgm: Handle IOException in MergeBase command
pgm: Fix missing braces in Merge command
pgm: Handle exceptions in Merge command
pgm: Fix missing braces in LsTree.run()
pgm: Handle exceptions in LsTree command
pgm: Fix missing braces in LsRemote.run()
pgm: Handle exceptions in LsRemote command
pgm: Handle exceptions in LsFiles command
pgm: Fix missing braces in Log.run()
pgm: Handle exceptions in Log command
pgm: Handle exceptions in Init command
pgm: Handle IOException in IndexPack command
pgm: Fix missing braces in Fetch.run()
pgm: Handle IOException in Fetch command
pgm: Handle GitAPIException in Gc command
pgm: Fix missing braces in DiffTree.run()
pgm: Handle exceptions in DiffTree command
pgm: Fix missing braces in Diff.run()
pgm: Handle exceptions in Diff command
RawParseUtils: Avoid import of java.nio.charset.StandardCharsets
Consistently import constants from StandardCharsets as static
LocalDiskRepositoryTestCase#createRepository: Default auto-close to false
UploadPack: Suppress false-positive resource leak warning
PushConnectionTest: Open TestRepository in try-with-resource
MergerTest: Open TestRepository in try-with-resource
MergeCommandTest: Open TestRepository in try-with-resource
PackWriterTest: Open TestRepository in try-with-resource
ReceivePackAdvertiseRefsHookTest: Open TestRepository in try-with-resource
SubmoduleStatusTest: Open TestRepository in try-with-resource
UploadPackTest: Open TestRepository in try-with-resource
PackParserTest: Open TestRepository in try-with-resource
SmartClientSmartServerTest: Open TestRepository in try-with-resource
Stop using deprecated methods of RemoteSetUrlCommand/RemoteRemoveCommand
SmartClientSmartServerTest: Stop using deprecated Repository#hasObject
DumbClientSmartServerTest: Open TestRepository in try-with-resource
DumbClientDumbServerTest: Open TestRepository in try-with-resource
DirCacheCheckoutTest: Open TestRepository in try-with-resource
HttpTestCase#fsck: Open TestRepository in try-with-resource
CheckoutCommandTest: Add comment to document intentionally empty catch block
pgm: Fix missing braces in Describe.run()
pgm: Handle exceptions in Describe command
pgm: Handle exceptions in Config command
pgm: Fix too wide lines in Commit.run()
pgm: Fix missing braces in Commit.run()
pgm: Handle exceptions in Commit command
pgm: Handle exceptions in Clean command
pgm: Handle GitAPIException in Fetch command
Fix missing braces in Branch.run()
pgm: Handle exceptions in Branch command
Fix missing braces in Blame.run()
pgm: Handle NoWorkTreeException and IOException in Blame command
Use try-with-resource for reader in Blame.run()
pgm: Handle GitAPIException in Add command
Change-Id: I6d546885beb9596120e201973995a7c6bfe115e0
* stable-5.2:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I81cf1cdc9d8dcd54536a79e522274fd244fa1d00
* stable-5.1:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: Id9f957ca8386f6035f4ab34d242cb6fb8c38e012
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.0:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I056f41f619a1bb22ca6be75ec21c32c1f6d4fd2f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.11:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: If75b149e693005dd3fe06b523e6e6784bedf44c1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.10:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I9052e318b5d920770f7c7121d36e3c58df9d5f5a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-4.9:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
ObjectDirectory: extra logging on packfile exceptions
Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
When the packfile checksum does not match the expected one
report the correct checksum error instead of reporting that
the number of objects is incorrect.
Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
(cherry picked from commit 436c99ce59)
Externalize the message and log the pack file with absolute path.
Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit 9665d86ba1)
The native.git_repository method doesn't work in the latest version
of bazel, and causes the build to fail with:
type 'struct' has no method git_repository()
Change-Id: Id6a57369b681c0afe811e9e3740b141fb7fb4653
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
(cherry picked from commit ec5fc57b79)
Display extra logging, including the exception with the associated
stacktrace, whenever a packFile can't be read and thus removed
from the packlist.
Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824
Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
(cherry picked from commit 962babc4b2)
* stable-5.2:
Fix error log message in ObjectDirectory.handlePackError()
Properly format pack checksums in PackFile.idx()
Cancel gc if thread was interrupted
PackFile: report correct message for checksum mismatch
ObjectDirectory: Clean up logging
Bazel: Stop using native.git_repository
Change-Id: Ib972641105cef8089791fc65389c4f43e218620a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Matthias Sohn