Commit Graph

5335 Commits

Author SHA1 Message Date
Saša Živkov 3d8649ddef Do not rely on ArrayIndexOutOfBoundsException to detect end of input
In the Config#StringReader we relied on ArrayIndexOutOfBoundsException
to detect the end of the input. Creation of exception with (deep) stack
trace can significantly degrade performance in case when we read
thousands of config files, like in the case when Gerrit reads all
external ids from the NoteDb.

Use the buf.length to detect the end of the input.

Change-Id: I12266f25751373a870ce3fa623cf2a95d882d521
2019-09-27 15:58:10 +02:00
Matthias Sohn e7a48bce3f Add missing braces in CherryPickCommand
Change-Id: I4b038ba7f86a45eb52422d3c2b4c4fc30ea16362
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-26 09:37:14 -04:00
Matthias Sohn cda3b2b492 Fire WorkingTreeModifiedEvent if cherry-pick failed with conflicts
Otherwise the paths modified by a cherry-pick with conflicts won't be
reported as modified via WorkingTreeModifiedEvents.

Change-Id: I875b67c0d2f68efdf90a9c32b80a2e074ed3570d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-09-26 15:10:05 +02:00
Han-Wen Nienhuys e5880d9531 reftable: pass OutputStream at construction time
This makes the intended use of the classes more clear. It also
simplifies generic functions that write reftables: they only need a
ReftableWriter as argument, as the stream is carried within the
ReftableWriter.

Change-Id: Idbb06f89ae33100f0c0b562cc38e5b3b026d5181
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-20 01:48:11 +02:00
Han-Wen Nienhuys d75a6b5d81 dfs: rename ReftableStack to DfsReftableStack
The functionality in ReftableStack is specific to DFS.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: If6003d104b1ecb0f3ca7e9c3815b233fa0abf077
2019-09-19 18:32:20 +02:00
Han-Wen Nienhuys 33dc08c162 reftable: add maxUpdateIndex to Reftable
This makes maxUpdateIndex() available in MergedReftable, so we can
know generically at which index to create the next reftable in a
stack.

Change-Id: Ia2314bc57c8b5dd7e69d5e61096fdce1d35abd11
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-09-19 18:32:20 +02:00
Matthias Sohn 7be05205dd Merge branch 'stable-5.5'
* stable-5.5:
  WorkingTreeIterator: handle different timestamp resolutions

Change-Id: I61df7392d8826ab526c39da2a8718723a1cc602a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:42:32 +02:00
Matthias Sohn 37c8b35344 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  WorkingTreeIterator: handle different timestamp resolutions

Change-Id: I83384336e0d766237d0579152673c9eefc9edd65
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:41:35 +02:00
Matthias Sohn 35339b455e Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  WorkingTreeIterator: handle different timestamp resolutions

Change-Id: I87afe29578b0270c79bc54c4e2cfda579c329237
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:40:50 +02:00
Matthias Sohn 84022ac9de Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  WorkingTreeIterator: handle different timestamp resolutions

Change-Id: Ia2a38949089a66abb5f4f1cd41717d2ae8d0eb5b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:39:54 +02:00
Matthias Sohn 831ba0a628 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  WorkingTreeIterator: handle different timestamp resolutions

Change-Id: Ic33ebec963af03a182dfe8f46486f5e8d00c0694
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:38:32 +02:00
Thomas Wolf e3f535cb15 WorkingTreeIterator: handle different timestamp resolutions
Older JGit stored only milliseconds timestamps in the index. Newer
JGit may get finer timestamps from the file system. This leads to
slow index diffs when a new JGit runs against an index produced
by older JGit because many timestamps will differ and JGit will
then do many content checks. See [1].

Handle this migration case by only comparing milliseconds if the
index entry has only millisecond precision.

The inverse may also occur; also compare only milliseconds if the
file timestamp has only millisecond precision.

Do the same also for microsecond resolution. On Windows, NTFS may
provide 100ns resolution and may be used by external programs writing
the index, but Java's WindowsFileAttributes may provide only
microseconds.

File timestamp precision in Java depends not only on the Java APIs
used by different JGit versions but may also change when running the
same Java code on different VMs. And of course the resolution may
vary among operating and file systems. Moreover, timestamp precision
in the index depends on the program that wrote the index. Canonical
git may use a different resolution, maybe even different between git
versions.

[1] https://www.eclipse.org/forums/index.php/t/1100344/

Change-Id: Idfd08606c883cb98787b2138f9baf0cc89a57b56
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-17 00:36:22 +02:00
Thomas Wolf 2e75fe6be1 Remove an old work-around for core.autocrlf = input
The removed code was trying to avoid mistakenly reporting differences
when core.autocrlf was set to "input" but a file had already been
committed with CR-LF. It did that by running the blob from the cache
through a CRLF-to-LF filter because older JGit would also run the file
from the working tree through such a filter.

The real fix for this case was done in commit 60cf85a. Since then files
are not normalized if they have already been committed with CR-LF and
this old fix attempt from bug 372834 is no longer needed.

Change-Id: Ib4facc153d81325cb48b4ee956a596b423f36241
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-09-12 12:02:11 +02:00
David Pursehouse 0bfba11186 Merge "Merge branch 'stable-5.5'" 2019-09-11 20:53:12 -04:00
Matthias Sohn 5c29fdd23c Merge branch 'stable-5.5'
* stable-5.5:
  Prepare 5.5.1-SNAPSHOT builds
  JGit v5.5.0.201909110433-r
  Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL

Change-Id: I113ce255739786623f7e101dc293b9a769abeff0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-11 22:19:16 +02:00
Han-Wen Nienhuys c78fbf23a5 dfs: use ReftableConfig() constructor in DfsReftableDatabase
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I756f086fccaba8e77db516160c14708f97519f93
2019-09-11 13:01:56 +02:00
Han-Wen Nienhuys 434e6c732a dfs: use RefDatabase API for exactRef in ReftableBatchRefUpdate
This removes one use of DFS specific code in this class.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I3ef6a4b98357cc6dc480892244ddc51d2fd751a2
2019-09-11 13:01:56 +02:00
Han-Wen Nienhuys 5c390cf9de reftable: add OutputStream argument to ReftableWriter constructor
This lets us write reftables generically with functions that take
just ReftableWriter argument

Change-Id: I7285951f62f9bd4c78e8f0de194c077d51fa4e51
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-09-11 13:01:56 +02:00
Han-Wen Nienhuys 90efbd216f reftable: read file footer in ReftableReader#allRefs
allRefs determined the end of the ref block without accounting for
index or log blocks. This could cause other blocks to be interpreted
as ref blocks, leading to "invalid block" error messages.

Change-Id: I7b9323e7d5e0e7d64535b3ec1efd576aed1e9870
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-09-11 13:01:56 +02:00
Han-Wen Nienhuys 726bcc4fdb reftable: add ReftableReflogReader
Change-Id: Iac2c4ac5e3aad0fa37157ed8866f1987d80268d6
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-09-11 12:59:55 +02:00
Matthias Sohn 7f262b8745 Prepare 5.5.1-SNAPSHOT builds
Change-Id: Iaf929168770dfef54ce2a7bfcbee9b87c450ca8a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-11 11:46:58 +02:00
Matthias Sohn de2df8adde JGit v5.5.0.201909110433-r
Change-Id: I5715730ac47d32462f235c2f50581bf1579d46b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-11 10:31:38 +02:00
Matthias Sohn 524fd0d8c6 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL

Change-Id: Ibdda7687da015b17cdca67427202766fe102fee1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-10 22:03:06 +02:00
Matthias Sohn 3c0230d844 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL

Change-Id: Ie8447fc28b03bfc918cee0cc2162a2aa3585f3ea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-10 22:00:40 +02:00
Matthias Sohn 8f96e1bebb Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL

Change-Id: I57093dfef181645088767471159c204c50961c3a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-10 21:26:40 +02:00
Matthias Sohn 718a8b11b3 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL

Change-Id: Ic4ee37fb36f61dc7707391f768bf78a33094bfd3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-10 21:26:04 +02:00
Matthias Sohn 84ac86ee61 Fix WorkingTreeIterator.compareMetadata() for CheckStat.MINIMAL
If CheckStat is MINIMAL or timestamps have no nanosecond part
WorkingTreeIterator.compareMetaData only checks the second part of
timestamps and ignores nanoseconds which may have ended up in the index
by using native git.

If

fileLastModified.getEpochSecond() == cacheLastModified.getEpochSecond()

we currently proceed comparing fileLastModified and cacheLastModified
with full precision which is wrong since we determined that we detected
reduced timestamp resolution.

Fix this and also handle smudged index entries for CheckStat.MINIMAL.

Change-Id: I6149885903ac63d79b42d234cc02aa4e19578f3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-10 16:08:45 +02:00
Matthias Sohn 830e214805 Merge branch 'stable-5.5'
* stable-5.5:
  bazel: fix running http tests
  Set parameter name in parameterized http tests
  Format BUILD files with buildifier
  Format BUILD files with buildifier
  [error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
  Bazel: Format BUILD files with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Fix warning about deprecated lib.bzl
  Format lib/BUILD with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Bump minimum supported version to 0.29.0
  Bazel: Bump skylib library version to 0.8.0
  Use bazelisk to switch between used bazel version
  Bazel: Require minimum bazel version 0.17.1
  Fix wrong placeholder index in error message packInaccessible
  JGitText: Remove unused externalized strings
  RepoText: Remove unused externalized string
  CLI: Remove unused externalized strings

Change-Id: Idb0848dd33a76328b24908dc86db335cca742a1c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-08 17:36:59 +02:00
Matthias Sohn b31d5360f0 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Format BUILD files with buildifier
  Format BUILD files with buildifier
  [error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
  Bazel: Format BUILD files with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Fix warning about deprecated lib.bzl
  Format lib/BUILD with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Bump minimum supported version to 0.29.0
  Bazel: Bump skylib library version to 0.8.0
  Use bazelisk to switch between used bazel version
  Bazel: Require minimum bazel version 0.17.1
  Fix wrong placeholder index in error message packInaccessible
  JGitText: Remove unused externalized strings
  RepoText: Remove unused externalized string
  CLI: Remove unused externalized strings

Change-Id: Idf3abd80ad3b00188f655e638d9908228770911f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-08 17:35:14 +02:00
Matthias Sohn 80fd4cb075 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Format BUILD files with buildifier
  [error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
  Bazel: Format BUILD files with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Fix warning about deprecated lib.bzl
  Format lib/BUILD with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Bump minimum supported version to 0.29.0
  Bazel: Bump skylib library version to 0.8.0
  Use bazelisk to switch between used bazel version
  Bazel: Require minimum bazel version 0.17.1
  Fix wrong placeholder index in error message packInaccessible
  JGitText: Remove unused externalized strings
  RepoText: Remove unused externalized string
  CLI: Remove unused externalized strings

Change-Id: Iaea77a89fcd821df5dbb24f5e3e39e6d1dfc0bd4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-08 15:05:19 +02:00
Matthias Sohn 82badd0a87 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  [error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
  Bazel: Format BUILD files with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Fix warning about deprecated lib.bzl
  Format lib/BUILD with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Bump minimum supported version to 0.29.0
  Bazel: Bump skylib library version to 0.8.0
  Use bazelisk to switch between used bazel version
  Bazel: Require minimum bazel version 0.17.1
  Fix wrong placeholder index in error message packInaccessible
  JGitText: Remove unused externalized strings
  RepoText: Remove unused externalized string
  CLI: Remove unused externalized strings

Change-Id: I0d8b7f79177a20dc00c89e2cf0005eb3d3039532
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-08 11:17:32 +02:00
Matthias Sohn d4752e2900 [error prone] Suppress NonAtomicVolatileUpdate in SimpleLruCache
We don't need to update time atomically since it's only used to order
cache entries in LRU order.

Change-Id: I756fa6d90b180c519bf52925f134763744f2c1f1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-08 10:37:54 +02:00
Matthias Sohn 38198972d4 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Bazel: Fix warning about deprecated lib.bzl
  Format lib/BUILD with buildifier
  Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
  Bazel: Bump minimum supported version to 0.29.0
  Bazel: Bump skylib library version to 0.8.0
  Use bazelisk to switch between used bazel version
  Bazel: Require minimum bazel version 0.17.1
  Fix wrong placeholder index in error message packInaccessible
  JGitText: Remove unused externalized strings
  RepoText: Remove unused externalized string
  CLI: Remove unused externalized strings

Change-Id: Iea37a8e39e9d4872dc607c9222dcf191ce4e4757
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-07 12:16:42 +02:00
David Pursehouse 85cea8c583 Bazel: Add fixes for --incompatible_load_java_rules_from_bzl
Change-Id: I24746335db132bf20f400cb3db400737596d4542
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-09-07 15:36:11 +09:00
Matthias Sohn 185dc84623 Fix wrong placeholder index in error message packInaccessible
See: https://www.eclipse.org/lists/jgit-dev/msg03850.html
Change-Id: I0e121a2b7ac399f4a663fa49ab330d27363e9a55
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-07 01:19:07 +02:00
David Pursehouse 8792743c51 JGitText: Remove unused externalized strings
Change-Id: I995d7a1b6ab2866221eee9f5cb828b97192daf4a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-09-07 01:19:07 +02:00
David Pursehouse 689474248c RepoText: Remove unused externalized string
Change-Id: Ida47637f54afdb76513be9b04aae32107567d4e3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-09-07 01:19:06 +02:00
Yunjie Li ea9231b39d ReceivePack: Prevent pointing a branch to a non-commit object
Since commit c3b0dec509fe136c5417422f31898b5a4e2d5e02, Git has
disallowed writing a non-commit to refs/heads/* refs.  JGit still
allows that, which can put users in a bad situation.  For example,

	git push origin v1.0:master

pushes the tag object v1.0 to refs/heads/master, instead of the
intended commit object v1.0^{commit}.

Prevent that by validating that the target of the ref points to a
commit object when pushing to refs/heads/*.

Git performs the same check at a lower level (in the RefDatabase).  We
could do the same here, but for now let's start conservatively by
handling it in pushes first.

[jn: fleshed out commit message]

Change-Id: I8f98ae6d8acbcd5ef7553ec732bc096cb6eb7c4e
Signed-off-by: Yunjie Li <yunjieli@google.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-09-06 12:18:17 -07:00
Matthias Sohn fdd63b183c Prepare 5.5.0-SNAPSHOT builds
Change-Id: I96bbefd698c74e450d05d21572d4769c6c5aaea6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 18:00:19 +02:00
Matthias Sohn 685f2f8ec4 JGit v5.5.0.201909041048-rc1
Change-Id: I77a8c73cfd0a27b1242eddf32da513ce0148260e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 16:47:57 +02:00
Matthias Sohn 4d78215673 Merge branch 'stable-5.5'
* stable-5.5:
  Prepare 5.4.4-SNAPSHOT builds
  JGit v5.4.3.201909031940-r
  Prepare 5.3.6-SNAPSHOT builds
  JGit v5.3.5.201909031855-r
  Prepare 5.1.12-SNAPSHOT builds
  JGit v5.1.11.201909031202-r
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings
  sshd: fix proxy connections with the DefaultProxyDataFactory
  sshd: support the HashKnownHosts configuration
  sshd: configurable server key verification
  sshd: allow setting a null ssh config
  sshd: simplify OpenSshServerKeyVerifier
  sshd: simplify ServerKeyLookup interface
  Use https in update site URLs

Change-Id: Icd21a8fcccffd56bfedbd037e48028308db6d13b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 02:43:03 +02:00
Matthias Sohn 8f742b9d30 Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Prepare 5.4.4-SNAPSHOT builds
  JGit v5.4.3.201909031940-r
  Prepare 5.3.6-SNAPSHOT builds
  JGit v5.3.5.201909031855-r
  Prepare 5.1.12-SNAPSHOT builds
  JGit v5.1.11.201909031202-r
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: Icd6630db6458971f840c3ab4553e00f6c775ede0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 02:24:09 +02:00
Matthias Sohn 32116da0cd Prepare 5.4.4-SNAPSHOT builds
Change-Id: I64c03436c285bd8eb5e87abe560417402b60e695
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 01:48:25 +02:00
Matthias Sohn fb08804e91 JGit v5.4.3.201909031940-r
Change-Id: I48ff1d51122ff73e0561f1fcf52a532221810e5d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 01:40:09 +02:00
Matthias Sohn 7cdb01939e Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.6-SNAPSHOT builds
  JGit v5.3.5.201909031855-r
  Prepare 5.1.12-SNAPSHOT builds
  JGit v5.1.11.201909031202-r
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: Icfd5a612055feeff04733c0d2c44837ab554a2c4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 01:28:15 +02:00
Matthias Sohn 927ae2f55b Prepare 5.3.6-SNAPSHOT builds
Change-Id: I4d57f8ea0ab1f8fbd73fd75861f2f751c4aba0a6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 01:03:41 +02:00
Matthias Sohn 2a3c8a86aa JGit v5.3.5.201909031855-r
Change-Id: I7f5dcebc2e5816c845fa63046c67bc2301926e2a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 00:54:34 +02:00
Matthias Sohn 4a3cd7f1b7 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.12-SNAPSHOT builds
  JGit v5.1.11.201909031202-r
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: I7e88c7c7d202f1e3fb8e143277650aa5fefff439
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 00:49:42 +02:00
Matthias Sohn aa62f191d0 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.12-SNAPSHOT builds
  JGit v5.1.11.201909031202-r
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: Iac5dc9683cea97db04d20f27c10f2e103d3ae7b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-04 00:08:21 +02:00
Matthias Sohn c8e42ca3ba Prepare 5.1.12-SNAPSHOT builds
Change-Id: I353ac3f23024063722abc7340bc9a6df3c615741
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 22:56:39 +02:00
Matthias Sohn 002f1628c6 JGit v5.1.11.201909031202-r
Change-Id: I288acf670160b9bcd2d04f572a902b5838b4d4f8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 18:01:55 +02:00
Matthias Sohn 7aec793266 Merge branch 'stable-5.0' into stable-5.1
* stable-5.0:
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: I172136a031ff0730e575327cafb3527c9650a71d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 17:40:19 +02:00
Matthias Sohn c31879ac4e Merge branch 'stable-4.11' into stable-5.0
* stable-4.11:
  Prepare 4.11.10-SNAPSHOT builds
  JGit v4.11.9.201909030838-r
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: Ifb6a4dbea2f48fd2ffa66eb737d61920aefedfbd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 15:59:39 +02:00
Matthias Sohn 261881055d Prepare 4.11.10-SNAPSHOT builds
Change-Id: Ic98bd87d96ce627d38e1ecc2c3cb28b517ab489e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 14:49:20 +02:00
Matthias Sohn 9a2deb2049 JGit v4.11.9.201909030838-r
Change-Id: Ie60468569074197d49e19cbd0ffaedfe88f3d74e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 14:37:49 +02:00
Matthias Sohn b0fd436c62 Merge branch 'stable-4.10' into stable-4.11
* stable-4.10:
  Bazel: Update bazlets to the latest master revision
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: If672b4f0c350f4e8ff7e1e706485cffd8137236d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 14:28:49 +02:00
Matthias Sohn b525036e58 Merge branch 'stable-4.9' into stable-4.10
* stable-4.9:
  BatchRefUpdate: repro racy atomic update, and fix it
  Delete unused FileTreeIteratorWithTimeControl
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Silence API warnings

Change-Id: Id5bf44645655fca40ad22bb1f1ad20a7c2e8f6db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 14:20:37 +02:00
Han-Wen Nienhuys 0e3d4a273f BatchRefUpdate: repro racy atomic update, and fix it
PackedBatchRefUpdate was creating a new packed-refs list that was
potentially unsorted. This would be papered over when the list was
read back from disk in parsePackedRef, which detects unsorted ref
lists on reading, and sorts them. However, the BatchRefUpdate also
installed the new (unsorted) list in-memory in
RefDirectory#packedRefs.

With the timestamp granularity code committed to stable-5.1, we can
more often accurately decide that the packed-refs file is clean, and
will return the erroneous unsorted data more often. Unluckily timed
delays also cause the file to be clean, hence this problem was
exacerbated under load.

The symptom is that refs added by a BatchRefUpdate would stop being
visible directly after they were added. In particular, the Gerrit
integration tests uses BatchRefUpdate in its setup for creating the
Admin group, and then tries to read it out directly afterward.

The tests recreates one failure case. A better approach would be to
revise RefList.Builder, so it detects out-of-order lists and
automatically sorts them.

Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=548716 and
https://bugs.chromium.org/p/gerrit/issues/detail?id=11373.

Bug: 548716
Change-Id: I613c8059964513ce2370543620725b540b3cb6d1
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 10:37:30 +02:00
Matthias Sohn 7e0cd90cf7 Silence API warnings
Change-Id: I27fd62de51ca0eedcc7e2e256487bda1e18bce8a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03 02:55:37 +02:00
Thomas Wolf 2d34d0bd9c sshd: support the HashKnownHosts configuration
Add the constant, and implement hashing of known host names in
OpenSshServerKeyDatabase. Add a test verifying that the hashing
works.

Bug: 548492
Change-Id: Iabe82b666da627bd7f4d82519a366d166aa9ddd4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-09-02 21:30:27 +02:00
David Pursehouse 2f751c34e1 Merge changes I1217f5f2,Iba037e0e,I61e6c93c,I6304d1cb
* changes:
  dfs: Take size as long instead of int
  dfs: Read at the aligned position
  dfs: Add a position argument
  dfs: Move the deeply nested code to its own method
2019-08-30 07:09:51 -04:00
Jonathan Nieder 14ff64b587 Merge branch 'stable-5.5'
* stable-5.5:
  Return a new instance from openSystemConfig and openUserConfig

Change-Id: I3ba9fa8ceced6e9f883fc23afb0e2c812391beac
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-08-29 18:17:10 -07:00
Jonathan Nieder 66ac51781a Merge branch 'stable-5.4' into stable-5.5
* stable-5.4:
  Return a new instance from openSystemConfig and openUserConfig

Change-Id: If576513ebbd80c7f01b83be5ab2edd436f5739e3
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-08-29 18:16:52 -07:00
Jonathan Nieder 013f2621d0 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Return a new instance from openSystemConfig and openUserConfig

Change-Id: I5f2f957ff27b7941ab29b2dc1c885fbf0093705a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-08-29 18:16:39 -07:00
Jonathan Nieder 47a95d1aed Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Return a new instance from openSystemConfig and openUserConfig

Change-Id: I6491549ab13aad2a4c3f8444a090a94a378eccdb
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-08-29 18:16:29 -07:00
Jonathan Nieder 60283a58a7 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Return a new instance from openSystemConfig and openUserConfig

Change-Id: I93afa16b14e3e2d127705e3378fb703d6c329694
Signed-off-by: Jonathan Nieder <jrn@google.com>
2019-08-29 18:15:23 -07:00
Matthias Sohn af0296b42c Return a new instance from openSystemConfig and openUserConfig
Move the handling of cached user and system config to getSystemConfig
and getUserConfig methods and revert the implementation of
openSystemConfig and openUserConfig to the old stateless
implementation.

This ensures the open methods respect the passed-in parent config, which
may be different on each invocation. Additionally, returning a new
instance matches the behavior of the previous implementation of the
default system reader, which downstream callers may be depending on.

Move the implementation of the new caching methods getSystemConfig and
getUserConfig up to SystemReader. This avoids that we break the ABI for
subclasses of SystemReader.

Also see [1] which fixed a similar problem with Gerrit's custom
SystemReader.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/225458

Change-Id: If54a2491932d8fc914d4649cb73c9e837c5b8ad0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-30 02:34:13 +02:00
Matthias Sohn 5e24753ab0 Prepare 5.6.0-SNAPSHOT builds
Change-Id: I9eceb7b1272b27b979144041c75eb09ab4eeca6b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-29 10:08:52 +02:00
Matthias Sohn 1cda44bb8f Merge branch 'stable-5.5'
* stable-5.5:
  Update Orbit to S20190827152740 for 2019-09 RC1
  Silence warnings for non-externalized strings
  Prepare 5.5.0-SNAPSHOT builds
  JGit v5.5.0.201908280940-m3

Change-Id: I8298387fac1899df3f2fdc808cef33a9a7c7b2ec
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-29 10:08:15 +02:00
Matthias Sohn cd9422f239 Silence warnings for non-externalized strings
These warnings were missed to address in a0048208 which introduced them.

Change-Id: Ia2d15fdce72c10378d020682b80fe7fc548c0d4c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-29 09:24:40 +02:00
Matthias Sohn d5a0d76ed0 Prepare 5.5.0-SNAPSHOT builds
Change-Id: Ib498303c8787a5fbee89377eaed332a5a6b4f3b0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-28 21:14:09 +02:00
Jonathan Tan f1f7b42882 DfsCachedPack: Add accessor for packfile object
In order to support implementations of CachedPackUriProvider (which need
to supply, among other things, the checksum of the packfile
corresponding to a CachedPack), in a004820858 ("UploadPack: support
custom packfile-to-URI mapping", 2019-08-20), the method getCheckSum()
was added to PackIndex. However, there is no way to access the PackIndex
from a DfsCachedPack.

Therefore, add an accessor for the DfsPackFile stored in the
DfsCachedPack. Now, a user who has a DfsCachedPack can use
DfsCachedPack#getPackFile then DfsPackFile#getPackIndex then
PackIndex#getCheckSum to obtain the checksum of a pack.

Change-Id: Ia010c016f6cac0f058ee20eff4c10f57338bfefc
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-08-28 10:28:42 -07:00
Matthias Sohn e0ea41b500 JGit v5.5.0.201908280940-m3
Change-Id: I0920767979d7927bc18f3e395963aeebae5ea540
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-28 15:40:54 +02:00
Jonathan Tan f78446b56f Merge "RevWalk: Traverse all parents of UNINTERESTING commits" 2019-08-26 19:05:41 -04:00
Alex Spradlin a80df5380f RevWalk: Traverse all parents of UNINTERESTING commits
When firstParent is set, RevWalk traverses only the first parent of a
commit, even though that commit is UNINTERESTING. Since we want the
maximal UNINTERESTING set, we shouldn't prune any parents here. This
issue is apparent only when some of the commits being traversed are
unparsed, since walker.carryFlagsImpl() propagates the UNINTERESTING
flag to all parsed ancestors, masking the issue.

Therefore teach RevWalk to traverse all parents when a commit is
UNINTERESTING and not only the first parent. Since this issue is
masked by commit parsing, also test situations when the commits
involved are unparsed.

Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
Change-Id: I95e2ad9ae8f1f50fbecae674367ee7e0855519b1
2019-08-26 15:53:42 -07:00
Matthias Sohn c1873b0604 Merge "Fix error occurring when SecurityManager is enabled" 2019-08-23 16:59:03 -04:00
Matthias Sohn ad5339a6b0 Merge branch 'stable-5.4'
* stable-5.4:
  Prepare 5.4.3-SNAPSHOT builds
  JGit v5.4.2.201908231537-r
  Prepare 5.3.5-SNAPSHOT builds
  JGit v5.3.4.201908231101-r
  Prepare 5.1.11-SNAPSHOT builds
  JGit v5.1.10.201908230655-r
  Use AtomicReferences to cache user and system level configs
  Fix copy-paste typo in CloneCommand#cleanup
  SystemReader: Use correct constructor of FileBasedConfig

Change-Id: I241dd1314f5535147ad641576a25f49cae6fe62d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 22:31:46 +02:00
Matthias Sohn 85a1d8bcf8 Prepare 5.4.3-SNAPSHOT builds
Change-Id: Iaf5140446dcf86a50bdb1a30860dfd624d3ec337
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 22:10:41 +02:00
Matthias Sohn 2c0f02244c JGit v5.4.2.201908231537-r
Change-Id: I1d808c63db81435173987c399c90bda7bc5c8d9d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 21:36:24 +02:00
Nail Samatov b9d2926df4 Fix error occurring when SecurityManager is enabled
It's expected that jgit should work without native git installation.
In such case Security Manager can be configured to deny access to the
files outside of git repository. JGit tries to find cygwin
installation. If Security manager restricts access to some folders
in PATH, it should be considered that those folders are absent
for jgit.
Also JGit tries to detect if symbolic links are supported by OS. If
security manager forbids creation of symlinks, it should be assumed
that symlinks aren't supported.

Bug: 550115
Change-Id: Ic4b243cada604bc1090db6cc1cfd74f0fa324b98
Signed-off-by: Nail Samatov <sanail@yandex.ru>
2019-08-23 20:38:26 +03:00
Matthias Sohn e6619980ed Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.5-SNAPSHOT builds
  JGit v5.3.4.201908231101-r
  Prepare 5.1.11-SNAPSHOT builds
  JGit v5.1.10.201908230655-r
  Use AtomicReferences to cache user and system level configs
  Fix copy-paste typo in CloneCommand#cleanup
  SystemReader: Use correct constructor of FileBasedConfig

Change-Id: I4422632766fc5554a20f75346c5480b7fb611484
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 17:59:49 +02:00
Matthias Sohn f30382b191 Prepare 5.3.5-SNAPSHOT builds
Change-Id: Ic0b9dfcac814ac7584e5eb19105f46344a10465a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 17:44:04 +02:00
Matthias Sohn 06309fdc43 JGit v5.3.4.201908231101-r
Change-Id: Ia98f47d9a10d6b8cccca6b697230dad4f350d720
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 17:01:29 +02:00
Matthias Sohn 6dfdfe1f67 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Prepare 5.1.11-SNAPSHOT builds
  JGit v5.1.10.201908230655-r
  Use AtomicReferences to cache user and system level configs
  Fix copy-paste typo in CloneCommand#cleanup

Change-Id: I3ababc0f9b540cf1785925395d32a85d30aed40d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 17:00:11 +02:00
Matthias Sohn d999d12ad0 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Prepare 5.1.11-SNAPSHOT builds
  JGit v5.1.10.201908230655-r
  Use AtomicReferences to cache user and system level configs
  Fix copy-paste typo in CloneCommand#cleanup

Change-Id: Ie383633de334e65901ed4c6deb2d59cfcce98e16
2019-08-23 16:00:38 +02:00
Matthias Sohn 418722df34 Prepare 5.1.11-SNAPSHOT builds
Change-Id: I8608580424d0e523dcace63f83ae530ba78dbb15
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 15:55:45 +02:00
Matthias Sohn 1c44bf8caf JGit v5.1.10.201908230655-r
Change-Id: I68d15ebd6054bdeaf62e14cddb87724828db7943
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 13:38:56 +02:00
Matthias Sohn d67fe38915 Use AtomicReferences to cache user and system level configs
This ensures that only one instance of user and one instance of system
config is set.

Change-Id: Idd00150f91d2d40af79499dd7bf8ad5940f87c4e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-23 05:07:05 -04:00
Adrien Bustany 28553c189c Fix copy-paste typo in CloneCommand#cleanup
deleteChildren was called on directory instead of gitDir, leading to a
potential null pointer exception if the git directory existed initially.

Bug: 550340
Change-Id: Iafc3b2961253a99862a59e81c7371f7bc564b412
Signed-off-by: Adrien Bustany <adrien-xx-eclipse@bustany.org>
2019-08-23 08:45:55 +09:00
David Pursehouse c2a5b7fdcf SystemReader: Use correct constructor of FileBasedConfig
The merge done in change If0c5010a2 resolved a conflict incorrectly
and reverted the fix that was done in change Id0bcdc93b.

Change-Id: I0f5fde33d1f366817f2b966eb42535f7bd3b063e
Reported-by: Thomas Wolf <thomas.wolf@paranor.ch>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-08-22 19:23:55 +09:00
Matthias Sohn 1553c94223 Merge branch 'stable-5.4'
* stable-5.4:
  Prepare 5.4.2-SNAPSHOT builds
  JGit v5.4.1.201908211225-r
  Prepare 5.3.4-SNAPSHOT builds
  JGit v5.3.3.201908210735-r
  Add missing @since tag on FileTreeIterator#getLastModifiedInstant
  Prepare 5.1.10-SNAPSHOT builds
  JGit v5.1.9.201908210455-r
  Avoid sign extension when comparing mtime with Instant#getEpochSecond
  Fix deprecation in DirCache caused by Instant based DirCacheEntry

Change-Id: I63156d6b73044d5d8820869a397a0aa94e607b50
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 22:39:47 +02:00
Matthias Sohn 2eb83f4b4a Prepare 5.4.2-SNAPSHOT builds
Change-Id: I5b2fde2d69ef1cc11b085874ab783ba48e070470
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 18:26:36 +02:00
Matthias Sohn 0af59e1212 JGit v5.4.1.201908211225-r
Change-Id: Ifa99b930bd20a9bc10696c414c73ebde0cd4b0c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 18:17:59 +02:00
Matthias Sohn 5c93bbd344 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Prepare 5.3.4-SNAPSHOT builds
  JGit v5.3.3.201908210735-r
  Add missing @since tag on FileTreeIterator#getLastModifiedInstant
  Prepare 5.1.10-SNAPSHOT builds
  JGit v5.1.9.201908210455-r
  Avoid sign extension when comparing mtime with Instant#getEpochSecond
  Fix deprecation in DirCache caused by Instant based DirCacheEntry

Change-Id: Ic3c8d3adf522b73f87c713538a14697eb690f508
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 18:16:23 +02:00
Matthias Sohn bbf3da1aac Prepare 5.3.4-SNAPSHOT builds
Change-Id: I1ccfeb28e87d13b25a8892947e62f1151130b6a6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 17:45:59 +02:00
Matthias Sohn e9fcf1932e JGit v5.3.3.201908210735-r
Change-Id: I799859e47616af0ac5ee3a982cb9350d141681d7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 13:30:43 +02:00
Matthias Sohn 48498fd3e6 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Add missing @since tag on FileTreeIterator#getLastModifiedInstant
  Prepare 5.1.10-SNAPSHOT builds
  JGit v5.1.9.201908210455-r
  Avoid sign extension when comparing mtime with Instant#getEpochSecond
  Fix deprecation in DirCache caused by Instant based DirCacheEntry

Change-Id: If6d5f4dfd9fc8e8c09e29aa11b1004057eafeb9f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 12:13:27 +02:00
Matthias Sohn 7ac3a63e7f Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Add missing @since tag on FileTreeIterator#getLastModifiedInstant
  Prepare 5.1.10-SNAPSHOT builds
  JGit v5.1.9.201908210455-r
  Avoid sign extension when comparing mtime with Instant#getEpochSecond
  Fix deprecation in DirCache caused by Instant based DirCacheEntry

Change-Id: Id824c0b8b14dad5947ae9da1f90c3471e07b400f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 12:02:33 +02:00
Matthias Sohn e80a62e226 Add missing @since tag on FileTreeIterator#getLastModifiedInstant
Change-Id: I809399e6a71e0079d2f0007b0d3f00b531d451bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 11:52:20 +02:00
Matthias Sohn edc67125b7 Prepare 5.1.10-SNAPSHOT builds
Change-Id: If10afc8e663299a15db8c5fd0574fb51bf7e7ae9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 11:51:41 +02:00
Matthias Sohn 38adb548c3 JGit v5.1.9.201908210455-r
Change-Id: Iaade207292acb8b27e01aca7e1af97ad7db1e854
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 10:52:10 +02:00
Matthias Sohn c130e5e708 Avoid sign extension when comparing mtime with Instant#getEpochSecond
Ensure we use the same type when comparing seconds since the epoch.

This does not prevent that in 2038 timestamps in seconds since the epoch
stored in a 32 bit integer will overflow. Integer.MAX_VALUE translates
to 2038-01-19T03:14:07Z. After this date we'll have an issue since we
store seconds since the epoch in a 32 bit integer in some places.

Bug: 319142
Change-Id: If0c03003d40b480f044686e2f7a2f62c9f4e2fe1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 00:29:45 +02:00
Michael Keppler 6cb21049d7 Fix deprecation in DirCache caused by Instant based DirCacheEntry
Replace the two int variables smudge_s and smudge_ns by an Instant and
use the new method DirCacheEntry.mightBeRacilyClean(Instant).

Change-Id: Id70adbb0856a64909617acf65da1bae8e2ae934a
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-21 00:07:49 +02:00
Jonathan Tan 2d0f1c1dc1 Merge "UploadPack: support custom packfile-to-URI mapping" 2019-08-20 14:47:35 -04:00
Jonathan Tan a004820858 UploadPack: support custom packfile-to-URI mapping
Teach UploadPack to take a provider of URIs corresponding to cached
packs. When fetching, if the client supports the packfile-uri feature,
and if such a cached pack were to be streamed, instead send the
corresponding URI.

This packfile-uri feature is implemented in the jt/fetch-cdn-offload
branch of Git. There is interest in this feature [1], but it is not yet
merged.

[1] https://public-inbox.org/git/cover.1552073690.git.jonathantanmy@google.com/

Change-Id: I9a32dae131c9c56ad2ff4a8a9638ae3b5e44dc15
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-08-20 10:54:20 -07:00
Matthias Sohn 53160ee12b Externalize error message in ReftableWriter#throwIllegalEntry
Also use MessageFormat as done elsewhere.

Change-Id: I36a4df9002310fa28321de9d325ee213f3cbd433
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-20 00:35:51 +02:00
Matthias Sohn 3e2c88199f Merge branch 'stable-5.4'
* stable-5.4:
  Fix NPE in RebaseTodoFile#parseComments
  Fix NPE in ObjectIdOwnerMap#get
  Fix NPE in CommitOnlyTest#getHead
  FileUtils#lastModifiedInstant should not log error if path doesn't exist
  Cache user global and system-wide git configurations
  Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
  Add missing dependencies for running FS_POSIXTest in Eclipse
  Fix javadoc for SystemReader#getInstance
  Upgrade tycho-extras to 1.4.0
  Improve retry handling when saving FileStoreAttributes fails
  Ensure FSTest uses MockSystemReader
  Make supportsAtomicCreateNewFile return true as default
  Update orbit to R20190602212107-2019-06 to enable backports from master
  Handle InvalidPathException in FS_POSIX#createNewFileAtomic
  Ensure root cause of lock creation failures is logged
  Implement toString in MockSystemReader and MockConfig
  LocalDiskRefTreeDatabaseTest shall use MockSystemReader
  Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
  Ensure we use MockSystemReader in tests
  Override FileBasedConfig's save method in MockConfig
  Remove FileBasedConfig.load(boolean) introduced in d45219ba
  Disable debug log for FS in org.eclipse.jgit.test
  Bazel: enable logging for tests in org.eclipse.jgit.test
  LockFile: log exception if creation of lock file failed
  Stop using deprecated Constants.CHARACTER_ENCODING

Change-Id: I709de5edb626536529a99220aae7751b127c9bff
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-20 00:16:11 +02:00
Matthias Sohn 35d99477b8 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix NPE in RebaseTodoFile#parseComments
  Fix NPE in ObjectIdOwnerMap#get
  Fix NPE in CommitOnlyTest#getHead
  FileUtils#lastModifiedInstant should not log error if path doesn't exist
  Cache user global and system-wide git configurations
  Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
  Add missing dependencies for running FS_POSIXTest in Eclipse
  Fix javadoc for SystemReader#getInstance
  Improve retry handling when saving FileStoreAttributes fails
  Ensure FSTest uses MockSystemReader
  Make supportsAtomicCreateNewFile return true as default
  Update orbit to R20190602212107-2019-06 to enable backports from master
  Handle InvalidPathException in FS_POSIX#createNewFileAtomic
  Ensure root cause of lock creation failures is logged
  Implement toString in MockSystemReader and MockConfig
  LocalDiskRefTreeDatabaseTest shall use MockSystemReader
  Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
  Ensure we use MockSystemReader in tests
  Override FileBasedConfig's save method in MockConfig
  Remove FileBasedConfig.load(boolean) introduced in d45219ba
  Disable debug log for FS in org.eclipse.jgit.test
  Bazel: enable logging for tests in org.eclipse.jgit.test
  LockFile: log exception if creation of lock file failed
  Stop using deprecated Constants.CHARACTER_ENCODING

Change-Id: I43c2ab8b44c3e87d48e4072907ad169c81e3ffe0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-20 00:10:15 +02:00
Matthias Sohn 18fb58ae05 Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix NPE in RebaseTodoFile#parseComments
  Fix NPE in ObjectIdOwnerMap#get
  Fix NPE in CommitOnlyTest#getHead
  FileUtils#lastModifiedInstant should not log error if path doesn't exist
  Cache user global and system-wide git configurations
  Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
  Add missing dependencies for running FS_POSIXTest in Eclipse
  Fix javadoc for SystemReader#getInstance
  Improve retry handling when saving FileStoreAttributes fails
  Ensure FSTest uses MockSystemReader
  Make supportsAtomicCreateNewFile return true as default
  Update orbit to R20190602212107-2019-06 to enable backports from master
  Handle InvalidPathException in FS_POSIX#createNewFileAtomic
  Ensure root cause of lock creation failures is logged
  Implement toString in MockSystemReader and MockConfig
  LocalDiskRefTreeDatabaseTest shall use MockSystemReader
  Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
  Ensure we use MockSystemReader in tests
  Override FileBasedConfig's save method in MockConfig
  Remove FileBasedConfig.load(boolean) introduced in d45219ba
  Disable debug log for FS in org.eclipse.jgit.test
  Bazel: enable logging for tests in org.eclipse.jgit.test
  LockFile: log exception if creation of lock file failed
  Stop using deprecated Constants.CHARACTER_ENCODING

Change-Id: If0c5010a2cf151ebebb2f2088fac3ee02c5007b9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-19 23:41:17 +02:00
Matthias Sohn 0eea7368e2 Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix NPE in RebaseTodoFile#parseComments
  Fix NPE in ObjectIdOwnerMap#get
  Fix NPE in CommitOnlyTest#getHead
  FileUtils#lastModifiedInstant should not log error if path doesn't exist
  Cache user global and system-wide git configurations
  Avoid setup and saving FileStoreAttributes compete for ~/.gitconfig lock
  Add missing dependencies for running FS_POSIXTest in Eclipse
  Fix javadoc for SystemReader#getInstance
  Improve retry handling when saving FileStoreAttributes fails
  Ensure FSTest uses MockSystemReader
  Make supportsAtomicCreateNewFile return true as default
  Update orbit to R20190602212107-2019-06 to enable backports from master
  Handle InvalidPathException in FS_POSIX#createNewFileAtomic
  Ensure root cause of lock creation failures is logged
  Implement toString in MockSystemReader and MockConfig
  LocalDiskRefTreeDatabaseTest shall use MockSystemReader
  Ensure LocalDiskRepositoryTestCase#setup fully uses MockSystemReader
  Ensure we use MockSystemReader in tests
  Override FileBasedConfig's save method in MockConfig
  Remove FileBasedConfig.load(boolean) introduced in d45219ba
  Disable debug log for FS in org.eclipse.jgit.test
  Bazel: enable logging for tests in org.eclipse.jgit.test
  LockFile: log exception if creation of lock file failed
  Stop using deprecated Constants.CHARACTER_ENCODING

Change-Id: I48c585f3c9287be7d6ddb6b01a1955444e13fa31
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-19 18:30:35 +02:00
Han-Wen Nienhuys d7ee5aafee reftable: increment pointer *after* reading uint16
This fixes timezone reads out of the reflog.

Change-Id: I126d8742e5e904a074c544514180720466164f7c
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-08-19 11:40:28 +02:00
Han-Wen Nienhuys 9f9163cbca reftable: enforce ordering for ref and log writes
Previously, the API did not enforce ordering of writes.  Misuse of
this API would lead to data effectively being lost.

Guard against that with IllegalArgumentException, and add a test.

Change-Id: I04f55c481d60532fc64d35fa32c47037a03988ae
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-08-19 11:40:25 +02:00
Han-Wen Nienhuys ca3b4b6083 reftable: add some more comments
Change-Id: I95208f7a05919b5019895cf41ae378c97da6e680
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
2019-08-19 11:25:58 +02:00
Han-Wen Nienhuys ca9107d166 reftable: fix seeking to refs in reflog implementation
Small reftables omit the log index. Currently,
ReftableWriter#shouldHaveIndex does this if there is a single-block
log, but other writers could decide on different criteria.

In the case that the log index is missing, we have to linearly search
for the right block. It is never appropriate to use binary search on
blocks for log data, as the blocks are compressed and therefore
irregularly sized.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Id59874edf6bf45c7dec502d9465888e077ffe198
2019-08-19 11:10:20 +02:00
Matthias Sohn 86a567f615 Fix NPE in RebaseTodoFile#parseComments
Change-Id: I5487f3c2609eaf2a0ddf71ebb2f6c9701fb7600c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-19 10:16:23 +02:00
Anton Khodos 5e44bfa3ad Fix ApplyCommand which doesn't work if patch adds empty file
Bug: 548219
Change-Id: Ibb32132a38e54508a24489322da58ddfd80a1d9a
Signed-off-by: Anton Khodos <khodosanton@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-19 02:37:04 +02:00
Matthias Sohn a4216e5382 Fix NPE in ObjectIdOwnerMap#get
Change-Id: I3812961a27ac410d610ef50c73a28f21bb05ae79
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-18 11:59:54 +02:00
Matthias Sohn 31356f5d18 FileUtils#lastModifiedInstant should not log error if path doesn't exist
Change-Id: Id8447735beb24becb41612d3d29d5351f8273d22
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-18 11:59:54 +02:00
Matthias Sohn f383206ace Cache user global and system-wide git configurations
So far the git configuration and the system wide git configuration were
always reloaded when jgit accessed these global configuration files to
access global configuration options which are not in the context of a
single git repository. Cache these configurations in SystemReader and
only reload them if their file metadata observed using FileSnapshot
indicates a modification.

Change-Id: I092fe11a5d95f1c5799273cacfc7a415d0b7786c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-08-18 11:47:26 +02:00
Matthias Sohn b51bb4f7bd Fix javadoc for SystemReader#getInstance
The existing javadoc was copied from another method and not adapted.

Change-Id: I39a7e5d719b2c379de9bd1a4710a55a73700c6f0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-16 16:51:45 +02:00
Thomas Wolf 8c7f1a699a TransportHttp: resolve redirect URIs from the "Location" header
The "Location" header in a redirect response may contain a relative
URI. Resolve it against the URI the request was made.

Bug: 550033
Change-Id: I29de07dfbbbc794090821b7c190cb2cf662c5a60
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-08-16 08:31:59 +02:00
Thomas Wolf 39e83a6583 GPG: also consider pubring.gpg when looking for keys
The algorithm for finding keys was already improved in commit db0eb9f8,
but that wasn't quite correct yet.

If there is no pubring.kbx but a private-keys-v1.d directory and a
pubring.gpg, GPG also uses pubring.gpg in combination with the
private-keys-v1.d directory. GPG has three ways to locate public and
private key pairs:

* pubring.kbx and private-keys-v1.d (GPG >= 2.1)
* pubring.gpg and private-keys-v1.d (GPG >= 2.1)
* pubring.gpg and secring.gpg (GPG < 2.1)

See [1] and [2]. pubring.kbx may not exist if the user migrated from
an older GPG installation and didn't run the agent. Since we don't
know which GPG version the user has we must try secring.gpg also if
we found the public key in pubring.gpg, but didn't find the secret
key in the private key directory. Note that GPG < 2.1 also may have
a private key directory, used by the agent. But it may also _not_ have
that directory.

[1] https://lists.gnupg.org/pipermail/gnupg-users/2015-December/054881.html
[2] https://www.gnupg.org/faq/whats-new-in-2.1.html#nosecring

Bug: 549439
Change-Id: I6088014b16c585b6a3408bb31dba3c116e6b583d
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-08-15 12:45:29 +02:00
Matthias Sohn 2d84bb4341 Improve retry handling when saving FileStoreAttributes fails
- fix handling of interrupts in FileStoreAttributes#saveToConfig
- increase retry wait time to 100ms
- don't wait after last retry
- dont retry if failure is caused by another exception than
LockFailedException

Change-Id: I108c012717d2bcce71f2c6cb9cf0879de704ebc2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-13 08:33:32 +02:00
Masaya Suzuki c0cfdcd2f1 dfs: Take size as long instead of int
Practically we wouldn't have 2GB+ objects in the DfsBlockCache, but by
making it long, we can clean up some long-to-integer conversions.

Change-Id: I1217f5f273a1420d80e2307ac9ff4a52460237a2
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-08-12 10:34:24 -07:00
Masaya Suzuki 46678b303c dfs: Read at the aligned position
The position is anyway aligned in BlockBasedFile, so this is no-op.

Change-Id: Iba037e0ecff339393dd2c03fc5ae4fe858031e4f
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-08-12 10:34:24 -07:00
Masaya Suzuki e837bdd0fa dfs: Add a position argument
This makes DfsBlockCache methods more unified. Also this reduces a magic
number embedded in DfsBlockCache.

Change-Id: I61e6c93ca283c0395738103bd2d94091edbccd4e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-08-12 10:34:05 -07:00
Masaya Suzuki 566384fa45 dfs: Move the deeply nested code to its own method
This is a pure code move.

Change-Id: I6304d1cb2b22cfce969b7b5eaaec911ea28579c0
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-08-12 10:19:55 -07:00
Vishal Devgire 9b4a55c0b6 Make supportsAtomicCreateNewFile return true as default
The method org.eclipse.jgit.util.FS.supportsAtomicCreateNewFile()
should default to true as mentioned in docs [1]

org.eclipse.jgit.util.FS_POSIX.supportsAtomicCreateNewFile() method
will set the value to false if the git config
core.supportsatomiccreatenewfile is not set.

It should default to true if the configuration is undefined.

[1]
4169a95a65/org.eclipse.jgit/src/org/eclipse/jgit/util/FS_POSIX.java (L372)

Bug: 544164
Change-Id: I16ccf989a89da2cf4975c200b3228b25ba4c0d55
Signed-off-by: Vishal Devgire <vishaldevgire@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-12 14:35:06 +02:00
Matthias Sohn 6ef225c519 Handle InvalidPathException in FS_POSIX#createNewFileAtomic
Bug: 547400
Change-Id: Ic3541e360a2968ba3532a3d3fa4828b0d0463c02
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-12 12:18:16 +02:00
Matthias Sohn cc29da8e0f Ensure root cause of lock creation failures is logged
Change-Id: I91cdf1e085a29c0aabd6d22c6ebe848b2d75f42c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-12 12:17:11 +02:00
Matthias Sohn a7338dd1e5 Remove FileBasedConfig.load(boolean) introduced in d45219ba
We can't add this method to the super class StoredConfig since that
abstracts from filesystem storage. MockSystemReader.MockConfig is a
StoredConfig and is also used by tests for dfs based storage. Hence
remove this leaky abstraction.

This implies we always use the fallback FileStoreAttributes which means
a config file modification is considered racy within the first 2
seconds. This should not be an issue since typically configs change
rarely and re-reading a config within the racy period is relatively
cheap since configs are small.

Change-Id: Ia2615addc24a7cadf3c566ee842c6f4f07e159a5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-10 23:57:21 +02:00
Matthias Sohn 42000a71e3 LockFile: log exception if creation of lock file failed
Change-Id: I4de75d12ec9e61193494916307289378cdb6220e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-10 00:57:04 +02:00
Thomas Wolf db0eb9f8ae GPG: check secring.gpg for keys if pubring.kbx is empty
If no OpenPGP key is found in pubring.kbx, try the legacy secring.gpg.
This appears to be consistent with GPG[1].

[1] https://lists.gnupg.org/pipermail/gnupg-users/2015-December/054881.html

Bug: 549439
Change-Id: I1557fd9b1f555a9b521fcd57cd3caccbdbacbeda
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-08-09 16:53:31 -04:00
Matthias Sohn 3cea3676c7 Merge branch 'stable-5.4'
* stable-5.4: (82 commits)
  Export all packages of o.e.j.ant and o.e.j.archive bundles
  Do not require test bundles to export all packages
  Fix API problem filters
  Increase severity of AmbiguousMethodReference to ERROR
  [error prone] suppress AmbiguousMethodReference in AnyLongObjectId
  [error prone] fix ReferenceEquality warning in CommitBuilder
  [error prone] suppress NonAtomicVolatileUpdate warning in SimpleLruCache
  [error prone] fix ReferenceEquality warning in CommitGraphPane#authorFor
  [error prone] fix ReferenceEquality warning in RevWalk#isMergedInto
  [error prone] fix ReferenceEquality warning in RefUpdate#updateImpl
  [error prone] fix ReferenceEquality warning in static equals methods
  [error prone] suppress AmbiguousMethodReference in AnyObjectId
  [error prone] fix "FutureReturnValueIgnored" error in FS
  Fix formatting and add missing braces in Repository#stripWorkDir
  Repository: fix reference comparison of Files
  MergeAlgorithm: Suppress Error Prone warning about reference equality
  Fix NarrowingCompoundAssignment warnings from Error Prone
  FS_POSIX: handle Files.getFileStore() failures
  Fix OpenSshConfigTest#config
  FileSnapshot: fix bug with timestamp thresholding
  In LockFile#waitForStatChange wait in units of file time resolution
  Cache FileStoreAttributeCache per directory
  Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
  Persist minimal racy threshold and allow manual configuration
  Measure minimum racy interval to auto-configure FileSnapshot
  Reuse FileUtils to recursively delete files created by tests
  Fix FileAttributeCache.toString()
  Add test for racy git detection in FileSnapshot
  Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
  Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
  Add missing javadoc in org.eclipse.jgit.junit
  Enhance RepeatRule to report number of failures at the end
  Fix FileSnapshotTests for filesystem with high timestamp resolution
  Retry deleting test files in FileBasedConfigTest
  Measure filesystem timestamp resolution already in test setup
  Refactor FileSnapshotTest to use NIO APIs
  Measure stored timestamp resolution instead of time to touch file
  Handle CancellationException in FileStoreAttributeCache
  Fix FileSnapshot#saveNoConfig
  Use Instant for smudge time in DirCache and DirCacheEntry
  Use Instant instead of milliseconds for filesystem timestamp handling
  Workaround SecurityException in FS#getFsTimestampResolution
  Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
  FS: ignore AccessDeniedException when measuring timestamp resolution
  Add debug trace for FileSnapshot
  Use FileChannel.open to touch file and set mtime to now
  Persist filesystem timestamp resolution and allow manual configuration
  Increase bazel timeout for long running tests
  Bazel: Fix lint warning flagged by buildifier
  Update bazlets to latest version
  Bazel: Add missing dependencies for ArchiveCommandTest
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  Add support for nanoseconds and microseconds for Config#getTimeUnit
  Optionally measure filesystem timestamp resolution asynchronously
  Delete unused FileTreeIteratorWithTimeControl
  FileSnapshot#equals: consider UNKNOWN_SIZE
  Timeout measuring file timestamp resolution after 2 seconds
  Fix RacyGitTests#testRacyGitDetection
  GlobalBundleCache: Fix ClassNewInstance warning from Error Prone
  IncorrectObjectTypeException: Fix typos in constructors' Javadoc
  Change RacyGitTests to create a racy git situation in a stable way
  Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
  Fix non-deterministic hash of archives created by ArchiveCommand
  Update Maven plugins ecj, plexus, error-prone
  Update Maven plugins and cleanup Maven warnings
  Make inner classes static where possible
  Error Prone: Increase severity of NonOverridingEquals to ERROR
  Error Prone: Increase severity of ImmutableEnumChecker to ERROR
  GitDateParser#ParseableSimpleDateFormat: Make formatStr private final
  BatchRefUpdateTest: Suppress ImmutableEnumChecker warning
  PacketLineIn: Suppress comparison warnings for END and DELIM
  FileSnapshot#toString: Suppress ReferenceEquality warnings
  Blame: Suppress ReferenceEquality warning for RevCommit instances
  Fix API problem filters
  pgm: add missing optional dependency to org.tukaani:xz
  NetscapeCookieFile: Make hash static and group overloaded write
  NetscapeCookieFile: Javadoc fixes
  Config: Handle reference-equality warning (and empty javadoc)
  Error Prone: Increase severity of ShortCircuitBoolean to ERROR
  ObjectWalk: Prefer boolean operators over logical operators in comparisons
  BasePackFetchConnection: Prefer boolean operators over logical operators in comparisons
  PackWriter: Prefer boolean operators over logical operators in comparisons

Change-Id: I825fd55bcb5345fb7afe066bf54ca50325f40acb
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 14:32:02 +02:00
Matthias Sohn 83502bb201 Fix API problem filters
- add missing filters for methods introduced in 5.4.1 to fix error prone
warnings
- remove no longer needed filters

Change-Id: Ia74d4e9876eae8acd403c8dea1fdf1227d68037e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 13:12:59 +02:00
David Pursehouse ad02a7cbad Stop using deprecated Constants.CHARACTER_ENCODING
Change-Id: I105b8a05bc64f249879a0795a059958553cc60c6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-08-09 12:11:41 +09:00
Matthias Sohn 4b9ed3f678 [error prone] fix ReferenceEquality warning in CommitBuilder
Comparing with UTF_8 constant in StandardCharsets doesn't require to use
equals.

Change-Id: I6c73a929367f32c9e76ce99f6c0af268480d9230
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 01:50:01 +02:00
Matthias Sohn b4cb06b294 [error prone] suppress NonAtomicVolatileUpdate warning in SimpleLruCache
It's not important to update time field, scalability is more important
than perfect LRU ordering of cache entries.

Change-Id: I22466c580cd3613b81e1989130b2724af9d6c466
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 01:49:59 +02:00
Matthias Sohn 70258a9cb2 [error prone] fix ReferenceEquality warning in RevWalk#isMergedInto
Change-Id: Ibef75e2bc76e90f6e29c4cb3ba1c1f6e67009b10
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 01:49:59 +02:00
Matthias Sohn 722deaa671 [error prone] fix ReferenceEquality warning in RefUpdate#updateImpl
Change-Id: I6687e1eec6b6ecfe319a598ef8aec1976cee354b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 01:49:59 +02:00
Matthias Sohn bac0e8fd8d [error prone] fix ReferenceEquality warning in static equals methods
Implement a helper method suppressing the ReferenceEquality error prone
warning and use it to fix this warning in static equals methods where
this comparison is used to implement fast path of static equals
implementation.

See https://errorprone.info/bugpattern/ReferenceEquality

Change-Id: I33538a3406007d24efec3a504e031ca1069572ed
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-09 01:49:56 +02:00
Matthias Sohn 8f7e851346 [error prone] suppress AmbiguousMethodReference in AnyObjectId
Move the implementation of the static equals() method to a new method
and suppress the error. Deprecate the old method to signal that we
intend to remove it in the next major release.

See https://errorprone.info/bugpattern/AmbiguousMethodReference

Change-Id: I5e29c97f4db3e11770be589a6ccd785e2c9ac7f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 23:39:23 +02:00
Matthias Sohn 2f8911181b [error prone] fix "FutureReturnValueIgnored" error in FS
Change-Id: I53731091b3e34ac2e93a18f0ad6dd04dc56f8177
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 14:42:25 +02:00
Matthias Sohn a52b331d71 Fix formatting and add missing braces in Repository#stripWorkDir
Change-Id: I601d917f3741e0207a8ee7a365d9c2dea6422401
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 14:42:14 +02:00
Matthias Sohn 400bfd6665 Repository: fix reference comparison of Files
Change-Id: Ib46ea2c0d5039c88b4fc59723135c503a8c950ce
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 14:41:16 +02:00
David Pursehouse 6370098e54 MergeAlgorithm: Suppress Error Prone warning about reference equality
Error Prone reports:

  [ReferenceEquality] Comparison using reference equality instead of
  value equality

The END_EDIT instance is used as a marker, and thus it's OK to use
a reference equality comparison. Factor the comparison to a method
and add a suppression.

Change-Id: I7d9dc1fa21f46c984787056b0b5d163e313026a6
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-08-08 14:34:24 +02:00
David Pursehouse 72ae089206 Fix NarrowingCompoundAssignment warnings from Error Prone
Error Prone reports:

  [NarrowingCompoundAssignment] Compound assignments from long to int
  hide lossy casts

and

  [NarrowingCompoundAssignment] Compound assignments from int to byte
  hide lossy casts

  See https://errorprone.info/bugpattern/NarrowingCompoundAssignment

Fix the warnings by adding explicit casts or changing types as
necessary.

Now that all occurrences of the warning are fixed, increase its
severity to ERROR.

Change-Id: Idb3670e6047b146ae37daee07212ff9455512623
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-08-08 14:28:27 +02:00
Thomas Wolf aefb11298c FS_POSIX: handle Files.getFileStore() failures
Android unconditionally throws a SecurityException;[1] getFileStore()
is not supported. Catch the exception and don't attempt the hard-
linking atomic file mechanism.

[1] https://android.googlesource.com/platform/libcore/+/21e6175e25

Bug: 548947
Change-Id: Idfba2d9dbcbc80ea15ab2ae7889e5142444c1581
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-08-08 14:14:38 +02:00
Matthias Sohn f54db4a857 Merge branch 'stable-5.3' into stable-5.4
* stable-5.3:
  Fix OpenSshConfigTest#config
  FileSnapshot: fix bug with timestamp thresholding
  In LockFile#waitForStatChange wait in units of file time resolution
  Cache FileStoreAttributeCache per directory
  Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
  Persist minimal racy threshold and allow manual configuration
  Measure minimum racy interval to auto-configure FileSnapshot
  Reuse FileUtils to recursively delete files created by tests
  Fix FileAttributeCache.toString()
  Add test for racy git detection in FileSnapshot
  Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
  Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
  Add missing javadoc in org.eclipse.jgit.junit
  Enhance RepeatRule to report number of failures at the end
  Fix FileSnapshotTests for filesystem with high timestamp resolution
  Retry deleting test files in FileBasedConfigTest
  Measure filesystem timestamp resolution already in test setup
  Refactor FileSnapshotTest to use NIO APIs
  Measure stored timestamp resolution instead of time to touch file
  Handle CancellationException in FileStoreAttributeCache
  Fix FileSnapshot#saveNoConfig
  Use Instant for smudge time in DirCache and DirCacheEntry
  Use Instant instead of milliseconds for filesystem timestamp handling
  Workaround SecurityException in FS#getFsTimestampResolution
  Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
  FS: ignore AccessDeniedException when measuring timestamp resolution
  Add debug trace for FileSnapshot
  Use FileChannel.open to touch file and set mtime to now
  Persist filesystem timestamp resolution and allow manual configuration
  Increase bazel timeout for long running tests
  Bazel: Fix lint warning flagged by buildifier
  Update bazlets to latest version
  Bazel: Add missing dependencies for ArchiveCommandTest
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  Add support for nanoseconds and microseconds for Config#getTimeUnit
  Optionally measure filesystem timestamp resolution asynchronously
  Delete unused FileTreeIteratorWithTimeControl
  FileSnapshot#equals: consider UNKNOWN_SIZE
  Timeout measuring file timestamp resolution after 2 seconds
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
  Fix non-deterministic hash of archives created by ArchiveCommand
  Update Maven plugins ecj, plexus, error-prone
  Update Maven plugins and cleanup Maven warnings
  Make inner classes static where possible
  Fix API problem filters

Change-Id: Iec3ad6ccc194582cb844310dc172c3103dae4457
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 13:24:14 +02:00
Matthias Sohn 718555518d Merge branch 'stable-5.2' into stable-5.3
* stable-5.2:
  Fix OpenSshConfigTest#config
  FileSnapshot: fix bug with timestamp thresholding
  In LockFile#waitForStatChange wait in units of file time resolution
  Cache FileStoreAttributeCache per directory
  Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
  Persist minimal racy threshold and allow manual configuration
  Measure minimum racy interval to auto-configure FileSnapshot
  Reuse FileUtils to recursively delete files created by tests
  Fix FileAttributeCache.toString()
  Add test for racy git detection in FileSnapshot
  Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
  Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
  Add missing javadoc in org.eclipse.jgit.junit
  Enhance RepeatRule to report number of failures at the end
  Fix FileSnapshotTests for filesystem with high timestamp resolution
  Retry deleting test files in FileBasedConfigTest
  Measure filesystem timestamp resolution already in test setup
  Refactor FileSnapshotTest to use NIO APIs
  Measure stored timestamp resolution instead of time to touch file
  Handle CancellationException in FileStoreAttributeCache
  Fix FileSnapshot#saveNoConfig
  Use Instant for smudge time in DirCache and DirCacheEntry
  Use Instant instead of milliseconds for filesystem timestamp handling
  Workaround SecurityException in FS#getFsTimestampResolution
  Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
  FS: ignore AccessDeniedException when measuring timestamp resolution
  Add debug trace for FileSnapshot
  Use FileChannel.open to touch file and set mtime to now
  Persist filesystem timestamp resolution and allow manual configuration
  Increase bazel timeout for long running tests
  Bazel: Fix lint warning flagged by buildifier
  Update bazlets to latest version
  Bazel: Add missing dependencies for ArchiveCommandTest
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  Add support for nanoseconds and microseconds for Config#getTimeUnit
  Optionally measure filesystem timestamp resolution asynchronously
  Delete unused FileTreeIteratorWithTimeControl
  FileSnapshot#equals: consider UNKNOWN_SIZE
  Timeout measuring file timestamp resolution after 2 seconds
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Deprecate Constants.CHARACTER_ENCODING in favor of
StandardCharsets.UTF_8
  Fix non-deterministic hash of archives created by ArchiveCommand
  Update Maven plugins ecj, plexus, error-prone
  Update Maven plugins and cleanup Maven warnings
  Make inner classes static where possible
  Fix API problem filters

Change-Id: I238adfd3080a5fed9d64c3c757297da6ea893918
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 13:04:38 +02:00
Matthias Sohn 0046b2a8fe Merge branch 'stable-5.1' into stable-5.2
* stable-5.1:
  Fix OpenSshConfigTest#config
  FileSnapshot: fix bug with timestamp thresholding
  In LockFile#waitForStatChange wait in units of file time resolution
  Cache FileStoreAttributeCache per directory
  Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
  Persist minimal racy threshold and allow manual configuration
  Measure minimum racy interval to auto-configure FileSnapshot
  Reuse FileUtils to recursively delete files created by tests
  Fix FileAttributeCache.toString()
  Add test for racy git detection in FileSnapshot
  Repeat RefDirectoryTest.testGetRef_DiscoversModifiedLoose 100 times
  Fix org.eclipse.jdt.core.prefs of org.eclipse.jgit.junit
  Add missing javadoc in org.eclipse.jgit.junit
  Enhance RepeatRule to report number of failures at the end
  Fix FileSnapshotTests for filesystem with high timestamp resolution
  Retry deleting test files in FileBasedConfigTest
  Measure filesystem timestamp resolution already in test setup
  Refactor FileSnapshotTest to use NIO APIs
  Measure stored timestamp resolution instead of time to touch file
  Handle CancellationException in FileStoreAttributeCache
  Fix FileSnapshot#saveNoConfig
  Use Instant for smudge time in DirCache and DirCacheEntry
  Use Instant instead of milliseconds for filesystem timestamp handling
  Workaround SecurityException in FS#getFsTimestampResolution
  Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
  FS: ignore AccessDeniedException when measuring timestamp resolution
  Add debug trace for FileSnapshot
  Use FileChannel.open to touch file and set mtime to now
  Persist filesystem timestamp resolution and allow manual configuration
  Increase bazel timeout for long running tests
  Bazel: Fix lint warning flagged by buildifier
  Update bazlets to latest version
  Bazel: Add missing dependencies for ArchiveCommandTest
  Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file
  Add support for nanoseconds and microseconds for Config#getTimeUnit
  Optionally measure filesystem timestamp resolution asynchronously
  Delete unused FileTreeIteratorWithTimeControl
  FileSnapshot#equals: consider UNKNOWN_SIZE
  Timeout measuring file timestamp resolution after 2 seconds
  Fix RacyGitTests#testRacyGitDetection
  Change RacyGitTests to create a racy git situation in a stable way
  Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
  Fix non-deterministic hash of archives created by ArchiveCommand
  Update Maven plugins ecj, plexus, error-prone
  Update Maven plugins and cleanup Maven warnings
  Make inner classes static where possible
  Fix API problem filters

Change-Id: Ia57385b2a60f48a5317c8d723721c235d7043a84
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-08 11:54:56 +02:00
Han-Wen Nienhuys e60b9e1879 FileSnapshot: fix bug with timestamp thresholding
Increase the safety factor to 2.5x for extra safety if max of measured
timestamp resolution and measured minimal racy threshold is < 100ms, use
1.25 otherwise since for large filesystem resolution values the
influence of finite resolution of the system clock should be negligible.

Before, not yet using the newly introduced minRacyThreshold measurement,
the threshold was 1.1x FS resolution, and we could issue the
following sequence of events,

  start
  create-file
  read-file (currentTime)
  end

which had the following timestamps:

  create-file 1564589081998
  start 1564589082002
  read 1564589082003
  end 1564589082004

In this case, the difference between create-file and read is 5ms,
which exceeded the 4ms FS resolution, even though the events together
took just 2ms of runtime.

Reproduce with:
  bazel test --runs_per_test=100 \
    //org.eclipse.jgit.test:org_eclipse_jgit_internal_storage_file_FileSnapshotTest

The file system timestamp resolution is 4ms in this case.

This code assumes that the kernel and the JVM use the same clock that
is synchronized with the file system clock. This seems plausible,
given the resolution of System.currentTimeMillis() and the latency for
a gettimeofday system call (typically ~1us), but it would be good to
justify this with specifications.

Also cover a source of flakiness: if the test runs under extreme load,
then we could have

  start
  create-file
  <long delay>
  read
  end

which would register as an unmodified file. Avoid this by skipping the
test if end-start is too big.

[msohn]:
- downported from master to stable-5.1
- skip test if resolution is below 10ms
- adjust safety factor to 1.25 for resolutions above 100ms

Change-Id: I87d2cf035e01c44b7ba8364c410a860aa8e312ef
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-07 13:56:08 +02:00
Matthias Sohn 3b368d5578 In LockFile#waitForStatChange wait in units of file time resolution
Since we now measure file time resolution we can use it to replace the
hard coded wait time of 25ms. FileSnapshot#equals will return true until
the mtime of the old (o) and the new FileSnapshot (n) differ by at least
one  file time resolution.

Change-Id: Icb713a80ce9eb929242ed083406bfb6650c72223
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-07 13:21:39 +02:00
Matthias Sohn 6857138e19 Cache FileStoreAttributeCache per directory
Cache FileStoreAttributeCache entries since looking up FileStore for a
file may be expensive on some platforms.

Implement a simple LRU cache based on ConcurrentHashMap using a simple
long counter to order access to cache entries.

Change-Id: I4881fa938ad2f17712c05da857838073a2fc4ddb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
2019-08-06 14:54:39 +02:00
Matthias Sohn 275f3da783 Fix FileSnapshot#save(long) and FileSnapshot#save(Instant)
Use the fallback timestamp resolution as already described in the
javadoc of these methods. Using zero file timestamp resolution doesn't
make sense.

Change-Id: Iaad2a0f99c3be3678e94980a0a368181b6aed38c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-06 14:54:39 +02:00
Matthias Sohn d45219baac Persist minimal racy threshold and allow manual configuration
To enable persisting the minimal racy threshold per FileStore add a
new config option to the user global git configuration:

- Config section is "filesystem"
- Config subsection is concatenation of
  - Java vendor (system property "java.vendor")
  - Java version (system property "java.version")
  - FileStore's name, on Windows we use the attribute volume:vsn instead
    since  the name is not necessarily unique.
  - separated by '|'
  e.g.
    "AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
  The same prefix is used as for filesystem timestamp resolution, so
  both values are stored in the same config section
- The config key for minmal racy threshold is "minRacyThreshold" as a
  time value, supported time units are those supported by
  DefaultTypedConfigGetter#getTimeUnit
- measure for 3 seconds to limit runtime which depends on hardware, OS
  and Java version being used

If the minimal racy threshold is configured for a given FileStore the
configured value is used instead of measuring it.

When the minimal racy threshold was measured it is persisted in the user
global git configuration.

Rename FileStoreAttributeCache to FileStoreAttributes since this class
is now declared public in order to enable exposing all attributes in one
object.

Example:

[filesystem "AdoptOpenJDK|11.0.3|/dev/disk1s1"]
	timestampResolution = 7000 nanoseconds
	minRacyThreshold = 3440 microseconds

Change-Id: I22195e488453aae8d011b0a8e3276fe3d99deaea
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Also-By: Marc Strapetz <marc.strapetz@syntevo.com>
2019-08-06 14:54:39 +02:00
Matthias Sohn 5911521ba6 Measure minimum racy interval to auto-configure FileSnapshot
By running FileSnapshotTest#detectFileModified we found that the sum of
measured filesystem timestamp resolution and measured clock resolution
may yield a too small interval after a file has been modified which we
need to consider racily clean. In our tests we didn't find this behavior
on all systems we tested on, e.g. on MacOS using APFS and Java 8 and 11
this effect was not observed.

On Linux (SLES 15, kernel 4.12.14-150.22-default) we collected the
following test results using Java 8 and 11:

In 23-98% of 10000 test runs (depending on filesystem type and Java
version) the test failed, which means the effective interval which needs
to be considered racily clean after a file was modified is larger than
the measured file timestamp resolution.

"delta" is the observed interval after a file has been modified but
FileSnapshot did not yet detect the modification:

"resolution" is the measured sum of file timestamp resolution and clock
resolution seen in Java.

Java version    filesystem  failures resolution  min delta   max delta
1.8.0_212-b04   btrfs	    98.6%       1 ms        3.6 ms      6.6 ms
1.8.0_212-b04   ext4        82.6%       3 ms        1.1 ms      4.1 ms
1.8.0_212-b04   xfs         23.8%       4 ms        3.7 ms      3.9 ms
1.8.0_212-b04   zfs         23.1%       3 ms        4.8 ms      5.0 ms
11.0.3+7        btrfs       98.1%       3 us        0.7 ms      4.7 ms
11.0.3+7        ext4        98.1%       6 us        0.7 ms      4.7 ms
11.0.3+7        xfs         98.5%       7 us        0.1 ms      8.0 ms
11.0.3+7        zfs         98.4%       7 us        0.7 ms      5.2 ms

Mac OS
1.8.0_212       APFS        0%          1 s
11.0.3+7        APFS        0%          6 us

The observed delta is not distributed according to a normal gaussian
distribution but rather random in the observed range between "min delta"
and "max delta".

Run this test after measuring file timestamp resolution in
FS.FileAttributeCache to auto-configure JGit since it's unclear what
mechanism is causing this effect.

In FileSnapshot#isRacyClean use the maximum of the measured timestamp
resolution and the measured "delta" as explained above to decide if a
given FileSnapshot is to be considered racily clean. Add a 30% safety
margin to ensure we are on the safe side.

Change-Id: I1c8bb59f6486f174b7bbdc63072777ddbe06694d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-08-06 14:54:35 +02:00
Jonathan Nieder 863c2687df Merge "Constants for objects and info/{http-,}alternates" 2019-08-02 12:28:54 -04:00
David Turner 59640fb4f0 Constants for objects and info/{http-,}alternates
These are useful to avoid typos, and also for tab completion.

Change-Id: I0f2d267e46b36bc40297c9657c447f3fd8b9f831
Signed-off-by: David Turner <dturner@twosigma.com>
2019-08-02 12:16:31 -04:00
Jonathan Tan 9110037e3e UploadPack: move writing of "packfile" header
In a subsequent patch, in some cases, PackWriter#writePack will be
responsible for both the "packfile-uris" and "packfile" sections,
meaning that (in these cases) it must write the "packfile" section
header itself.

In preparation for that patch, move the writing of the "packfile"
section header closer to the invocation of PackWriter#writePack when the
entire fetch response is configured to use the sideband. This means that
"packfile" is written *after* objects are counted (and progress messages
sent to the client in sideband 2) when the "sideband-all" feature is
used (whether "packfile-uris" is used or not), and written *before*
objects are counted otherwise.

Having code to write "packfile" in two places is unfortunate but
necessary. When "sideband-all" is not used, object counting has to
happen after "packfile" is written, because "packfile" activates the
sideband that allows counting progress to be transmitted. When
"packfile-uris" is used, object counting has to happen before "packfile"
is written, because object counting determines whether to send
"packfile-uris" or "packfile". When "sideband-all" is used but
"packfile-uris" is not used, either way works; this commit uses
"packfile-uris" behavior in this case.

Also make the naming of the sideband-activating methods in PacketLineOut
more consistent.

Change-Id: Ifbfd26cc26af10c41b77758168833702d6983df1
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-07-30 23:37:49 -07:00
Matthias Sohn d17efe880d Fix FileAttributeCache.toString()
We should not list the complete cache but only show the cache entry at
hand.

Change-Id: I22be2a4dcbf0145155e23f2389bfcf5662cf23a6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-26 15:53:29 +02:00
Matthias Sohn 130aa31262 Add test for racy git detection in FileSnapshot
Repeat the test 10000 times to get statistics if measured
fsTimestampResolution is working in practice to detect racy git
situations.

Add a class to compute statistics for this test. Log delta between
lastModified and time when FileSnapshot failed to detect modification.
This happens if the racy git limit determined by measuring filesystem
timestamp resolution and clock resolution is too small. If it would be
correct FileSnapshot would always detect modification or mark it
modified if time since modification is smaller than the racy git limit.

Change-Id: Iabe7af1a7211ca58480f8902d4fa4e366932fc77
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-26 15:49:38 +02:00
Ivan Frade e9dd131d55 PreUploadHookChain: Use list instead of array internally
The newly introduced ProtocolV2HookChain is implemented using lists
instead of arrays.

Update PostUploadHookChain to keep the hook chains implementation
consistent.

Change-Id: I5ae0c923f117ac48558a989464f5d5d868d81f76
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-07-24 14:09:48 -07:00
Ivan Frade d1f7efcf2b PostUploadHookChain: Use a list instead of array internally
The newly introduced ProtocolV2HookChain is implemented using lists
instead of arrays.

Update PostUploadHookChain to keep hook chain implementations
consistent.

Change-Id: Ic5694feab943e8949896b93103dbf427716c9bd7
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-07-24 14:09:48 -07:00
Jonathan Tan b734412724 Merge "UploadPack: add getProtocolV2Hook() method" 2019-07-24 13:37:45 -04:00
Jonathan Tan 690fc2267b Merge "ProtocolV2HookChain: Allow to create a chain of protocol V2 hooks" 2019-07-24 13:37:07 -04:00
Masaya Suzuki 6272694185 JschConfigSessionFactory: [findbugs] Set synchronized
Set synchronized to make the config access consistent.

> Inconsistent synchronization of
org.eclipse.jgit.transport.JschConfigSessionFactory.config; locked 80%
of time

In order to make JschConfigSessionFactory threadsafe, synchronize this
method as well.

Change-Id: I32d1bfc2e98363d254992144e795ce72fe1e8846
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-07-22 16:01:39 -07:00
Matthias Sohn 99d351d0cb Measure stored timestamp resolution instead of time to touch file
Measure granularity of timestamps stored in the filesystem by setting
and then getting lastModified timestamp until the read value changed.
Increase increment exponentially to limit number of iterations starting
with 1 microsecond since Java's FileTime (up to Java 12) truncates
timestamps to 1 microsecond resolution. The chosen algorithm yields 2000
steps between 1 ms and 2.5 s.

Also measure clock resolution and add that for the total timestamp
resolution. This avoids systematic measurement errors introduced by
doing IO to touch a file.

Change-Id: I9b37138619422452373e298d9d8c7cb2c384db3f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-19 14:45:55 +02:00
Matthias Sohn 37f7679fc9 Handle CancellationException in FileStoreAttributeCache
Change-Id: If5985fbf04f630b1d72a1bafd508e0e15e1436be
2019-07-19 14:45:55 +02:00
Matthias Sohn 72d6e304b8 Fix FileSnapshot#saveNoConfig
We should not use configuration when creating FileSnapshot when
accessing FileBasedConfig.

Change-Id: Ic521632870f18bb004751642b9d30648dd94049a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-19 14:45:55 +02:00
Ivan Frade 04cecae3da UploadPack: add getProtocolV2Hook() method
A caller cannot install a second hook in the UploadPack without
overwriting whatever is already there.

Offer a method to get the current protocol v2 hook, so it can be chained
with new hooks.

Change-Id: Icb06f94ec52b8c8714f509b5b8622d6db42960fb
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-07-18 10:39:25 -07:00
Ivan Frade 9b7a4a30ca ProtocolV2HookChain: Allow to create a chain of protocol V2 hooks
UploadPack only supports one protocol-v2 hook. There are already cases
where more than one is needed.

Offer a Chain class to compose ProtocolV2Hooks, as other hooks do. It
looks like a single hook but it calls all its members.

Change-Id: Idd173ca7df6672079ac0de03c67f77abac376538
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-07-18 10:33:51 -07:00
Matthias Sohn d8d9427277 Use Instant for smudge time in DirCache and DirCacheEntry
Change-Id: I98050a51baf4726c5717ef62ce7f026173666bdf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-18 03:36:18 +02:00
Matthias Sohn 95e8264cc8 Use Instant instead of milliseconds for filesystem timestamp handling
This enables higher file timestamp resolution on filesystems like ext4,
Mac APFS (1ns) or NTFS (100ns) providing high timestamp resolution on
filesystem level.

Note:
- on some OSes Java 8,9 truncate milliseconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes truncates timestamp resolution to microseconds when
converting the internal representation to FileTime exposed in the API,
see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution

Change-Id: I25ffff31a3c6f725fc345d4ddc2f26da3b88f6f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-18 03:27:52 +02:00
Matthias Sohn 4db39f5074 Workaround SecurityException in FS#getFsTimestampResolution
On Android FS#getFsTimestampResolution always throws a
SecurityException, handle this by falling back to the fallback timestamp
resolution.

Bug: 548947
Change-Id: I0ee6cb3c20e189bdc8d488434a930427ad6f2df2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:15 +02:00
Matthias Sohn 79ede0c322 Fix NPE in FS$FileStoreAttributeCache.getFsTimestampResolution
Bug: 548682
Change-Id: I48840d3a68cf1db92c056d218a0d5ed0b9ea4c45
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:14 +02:00
Matthias Sohn be66222d1a FS: ignore AccessDeniedException when measuring timestamp resolution
It seems on cygwin creating a file under a writable directory can fail
with AccessDeniedException. Log a warning in this case and fallback to
worst case timestamp resolution of 2 seconds.

Bug: 548648
Change-Id: Ic50c31ce9dc9ccadd4db5247df929418ac62d45c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:14 +02:00
Matthias Sohn 09fec1a102 Add debug trace for FileSnapshot
Checking lastModified is time critical hence debug trace is the only way
to analyze issues since debugging is impractical.

Also add configuration for buffering of log4j output to reduce runtime
impact when debug trace is on. Limit buffer to 1MiB and comment this
configuration out since we may not always want to use buffering.

Change-Id: Ib1a0537b67c8dc3fac994a77b42badd974ce6c97
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:14 +02:00
Matthias Sohn 8a0ed8a004 Use FileChannel.open to touch file and set mtime to now
Use options
- StandardOpenOption.CREATE to create touched file if not existing
- StandardOpenOption.SYNC to enforce synch of data and meta data changes
- StandardOpenOption.WRITE

Also set mtime explicitly in FileUtils#touch to the current system time.
This should fix that the previous implementation didn't work on
- locally cached Windows network share (CSC-CACHE filesystem) mapped as
a drive
- nfsv4 mounts on Linux
and that it didn't create unborn file like Linux command "touch".
Apache common's and Guava's touch() use the same approach.

Immediately after creating the probe file used to measure timestamp
resolution touch it. This ensures we always use the local system clock
when measuring filesystem timestamp resolution. This should prevent that
clock skew could influence the measured timestamp resolution in case of
a mounted network filesystem.

Bug: 548598
Change-Id: Iaeaf5967963f582395a195aa637b8188bfadac60
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:14 +02:00
Matthias Sohn 16760c3e9a Persist filesystem timestamp resolution and allow manual configuration
To enable persisting filesystem timestamp resolution per FileStore add a
new config section to the user global git configuration:

- Config section is "filesystem"
- Config subsection is concatenation of
  - Java vendor (system property "java.vm.vendor")
  - runtime version (system property "java.vm.version")
  - FileStore's name
  - separated by '|'
  e.g.
    "AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"
  The prefix is needed since some Java versions do not expose the full
  timestamp resolution of the underlying filesystem. This may also
  depend on the underlying operating system hence concrete key values
  may not be portable.
- Config key for timestamp resolution is "timestampResolution" as a time
  value, supported time units are those supported by
  DefaultTypedConfigGetter#getTimeUnit

If timestamp resolution is already configured for a given FileStore
the configured value is used instead of measuring the resolution.

When timestamp resolution was measured it is persisted in the user
global git configuration.

Example:

[filesystem "AdoptOpenJDK|1.8.0_212-b03|/dev/disk1s1"]
	timestampResolution = 1 seconds

If locking the git config file fails retry saving the resolution up to 5
times in order to workaround races with another thread.

In order to avoid stack overflow use the fallback filesystem timestamp
resolution when loading FileBasedConfig which creates itself a
FileSnapshot to help checking if the config changed.

Note:
- on some OSes Java 8,9 truncate to milliseconds or seconds, see
https://bugs.openjdk.java.net/browse/JDK-8177809, fixed in Java 10
- UnixFileAttributes up to Java 12 truncates timestamp resolution to
microseconds when converting the internal representation to FileTime
exposed in the API, see https://bugs.openjdk.java.net/browse/JDK-8181493
- WindowsFileAttributes also provides only microsecond resolution up to
Java 12

Hence do not attempt to manually configure a higher timestamp resolution
than supported by the Java version being used at runtime.

Bug: 546891
Bug: 548188
Change-Id: Iff91b8f9e6e5e2295e1463f87c8e95edf4abbcf8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-17 09:42:14 +02:00
Jonathan Tan 79d776429e Support "sideband-all" in protocol v2 fetch
Allow the client to specify "sideband-all" in a fetch v2 request,
indicating that the whole response is to be multiplexed (with a sideband
indicator on every non-flush and non-delim pkt) instead of only the
packfile being multiplexed. This allows, for example, progress messages
to be sent at any point in the response.

This implements the "sideband-all" feature documented in
Documentation/technical/protocol-v2.txt in Git.

Change-Id: I3e7f21c88ff0982b1b7ebb09c9ad6c742c4483c8
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-07-16 11:13:30 -07:00
Jonathan Tan c1ed69de4a UploadPack: remove pckOut instance field
It is difficult to track what's happening with the pckOut instance
field, so replace it with a local variable in #upload instead.

Change-Id: Ibd9225b28334b7133eccdc6d82b26fc96cbde299
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-07-16 10:05:18 -07:00
Ivan Frade 848d9f0d71 TreeRevFilter: Refresh code to latest coding standards
Use brackets in all "if" statements and remove the "final" from local
variables and method arguments.

https://wiki.eclipse.org/EGit/Contributor_Guide#Coding_standards

Change-Id: I185f3112848fc1218cd7adb9828488f03fa4ddfc
Signed-off-by: Ivan Frade <ifrade@google.com>
2019-07-12 16:19:10 -07:00
Matthias Sohn 121c957405 Add support for nanoseconds and microseconds for Config#getTimeUnit
Change-Id: I0a5828438810dd23790cba52d7ae2e055c6a3fc9
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-11 10:12:04 +02:00
Matthias Sohn a950eac23b Optionally measure filesystem timestamp resolution asynchronously
In order to avoid blocking on the main thread during measurement
interactive applications like EGit may want to measure the filesystem
timestamp resolution asynchronously.

In order to enable measurement in the background call
FileStoreAttributeCache.setAsyncfileStoreAttrCache(true)
before the first access to cached FileStore attributes.

Bug: 548188
Change-Id: I8c9a2dbfc3f1d33441edea18b90e36b1dc0156c7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-11 10:12:04 +02:00
Han-Wen Nienhuys 84e6c24e58 FileSnapshot#equals: consider UNKNOWN_SIZE
Add a unittest.

In commit I5485db55 ("Fix FileSnapshot's consideration of file size"),
the special casing of UNKNOWN_SIZE was forgotten.

This change, together with I493f3b57b ("Measure file timestamp
resolution used in FileSnapshot") introduced a regression that would
occasionally surface in Gerrit integration tests marked UseLocalDisk,
with the symptom that creating the Admin user in NoteDb failed with a
LOCK_FAILURE.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I7ffd972581f815c144f810481103c7985af5feb0
2019-07-09 15:33:23 +02:00
Michael Keppler 3f41bd9d2f Fix typos in Javadoc
Change-Id: I225e04aefd02e56a62e16f3db53fdda77d5e7253
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-07-08 20:04:55 +02:00
Thomas Wolf 758124fa9c Correct @since in RevWalk for the --first-parent methods
Fixes PDE API checks complaining: the methods were added
in JGit 5.5.0.

Change-Id: I9ff860c3408c6bb3891fa0da7547394d0fe9d0b6
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2019-07-06 21:45:19 +02:00
Matthias Sohn 850b9d7540 Timeout measuring file timestamp resolution after 2 seconds
It was reported that measuring file timestamp resolution may hang
indefinitely on nfs. Hence timeout this measurement at the known worst
filesystem timestamp resolution (FAT) of 2 seconds.

Bug: 548188
Change-Id: I17004b0aa49d5b0e76360a008af3adb911b289c0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-07-03 21:34:29 +02:00
Masaya Suzuki 3091dc4348 Move ServiceMayNotContinueException handling code from sendPack
All other exceptions are handled in a wrapped sendPack method.
Consolidate the error handling code.

Change-Id: Ieac0ce64960534d009d1e6b025130b021b744794
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-06-27 19:38:54 -04:00
Masaya Suzuki f8267c9edb Move exception handling code to the caller
By doing this, exceptions thrown by sendPack are also covered by the
same code.

Change-Id: I3509f2d832af1410f307e931577e4d07e32b014e
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-06-27 19:38:45 -04:00
Masaya Suzuki ca360ea2b5 Do not ignore secondary failure
When another exception is thrown while handling another exception, that
exception can be attached to the original exception since Java 7
(Throwable#getSuppressed). Attach the secondary exception to the
original exception instead of throwing it away.

Change-Id: Ia093b8207714f2638e0343bc45a83d4342947505
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
2019-06-27 19:38:36 -04:00
Dave Borowitz 4973f05252 RevWalk: Add a setFirstParent that mimics C git's --first-parent
RevWalk does not currently provide a --first-parent equivalent and the
feature has been requested.

Add a field to the RevWalk class to specify whether walks should
traverse first parents only. Modify Generator implementations to support
the feature.

Change-Id: I4a9a0d5767f82141dcf6d08659d7cb77c585fae4
Signed-off-by: Dave Borowitz <dborowitz@google.com>
Signed-off-by: Alex Spradlin <alexaspradlin@google.com>
2019-06-27 16:27:33 -07:00
Jonathan Tan cbccfed4b3 UploadPack: support lazy fetches if no bitmap
If a client clones with "--filter=blob:none", the checkout that "git
clone" automatically does causes the client to fetch all blobs at HEAD.
When fetching from a non-bitmapped repository, this will fail if an
object walk is ever needed, because JGit currently rejects such requests
- see the commit message of d3021788d2 ("Use bitmaps for non-commit
reachability checks", 2017-11-10) for more information.

Rejecting such requests in the absence of bitmaps is probably
overzealous: it is true that the server would prefer to have bitmaps in
this case, but there might be a small proportion of repos (for example,
very small repos or newly created ones) that do not have bitmaps, yet
the server would still like to have partial clones for them.

So, allow such requests, performing the object walk reachability check
if necessary. Limit this to servers with "uploadpack.allowFilter"
configured, so that servers wanting to support partial clone have this
functionality, and servers that do not support partial clone do not have
to pay the object walk reachability check cost.

Change-Id: I51964bafec68696a799625d627615b4f45ddbbbf
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
2019-06-25 09:22:47 -07:00
David Pursehouse dad9e1ff95 GlobalBundleCache: Fix ClassNewInstance warning from Error Prone
Error Prone reports:

  [ClassNewInstance] Class.newInstance() bypasses exception checking;
  prefer getDeclaredConstructor().newInstance()

  See https://errorprone.info/bugpattern/ClassNewInstance

This was the only occurrence of the warning in the code base; now it's
fixed, increase the severity to ERROR to prevent future occurrences.

Change-Id: Ic04d1c5d2bd458bbb4bb399d6ce9d147bd48d0b1
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-06-22 18:44:23 +09:00
Michael Keppler 4b0c192f2d Fix bundle localization of Apache SSH bundle
The placeholders in manifest and plugin.properties did not match. To
avoid similar issues, all placeholders have been changed to
Bundle-Vendor and Bundle-Name now.

Bug:548503
Change-Id: Ibd4b9bc237b323e614506b97e5fbc99416365040
Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
2019-06-21 17:54:06 +02:00
David Pursehouse 99a5fa2283 IncorrectObjectTypeException: Fix typos in constructors' Javadoc
Change-Id: Ib63310a603ba432e65d0c46e4b6b8d440ca6a115
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-06-20 08:41:34 +09:00
René Scheibe 8f9697b4c1 Fix "reset -hard" bug that folders could not be deleted
The deleted code is not required as removed files are deleted correctly in
doCheckout() anyway.

The deleted code failed in case a non-empty directory had to be deleted.
file.delete() returned false, triggering an exception.

Bug: 479266
Change-Id: I011bb3882ff0c35b238aa3eccad7889041210277
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
2019-06-19 11:52:02 +02:00
René Scheibe b4edf9ec14 Fix "reset -hard" bug that folders could not be created
Creating a folder failed in case a file with the same name already
existed.

Bug: 479266
Change-Id: Ia987660ec0968ad4081dbd5a60e80660539497e3
Signed-off-by: René Scheibe <rene.scheibe@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-06-19 11:51:24 +02:00
David Pursehouse f18b5010fc Deprecate Constants.CHARACTER_ENCODING in favor of StandardCharsets.UTF_8
Change-Id: I621ba174235a6fb56236e54d24bce704bb5afb28
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-06-19 10:43:01 +02:00