Commit Graph

5349 Commits

Author SHA1 Message Date
Shawn Pearce 53dd9a9e4b Rename extensions.refsStorage to refStorage
This matches the proposal that has been discussed at length on
git-core mailing list and seems to be the accepted convention.

Change-Id: I9f6ab15144826893d1e2a4b48a2d657d6dd445ec
2017-08-11 18:20:50 -07:00
Thomas Wolf a489a8ae9a Ensure EOL stream type is DIRECT when -text attribute is present
Otherwise fancy combinations of attributes (binary or -text in
combination with crlf or eol) may result in the corruption of binary
data.

Bug: 520910
Change-Id: I3ffc666c13d1b9d2ed987b69a67bfc7f42ccdbfc
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-11 22:56:50 +02:00
Thomas Wolf 4bc539a814 Use relative paths for attribute rule matching
Attribute rules must match against the entry path relative to the
attribute node containing the rule. The global entry path is to be
used only for the init and the global node (and of course the root
node).

Bug: 520677
Change-Id: I80389a2dc272a72312729ccd5358d7c75e1ea20a
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-08-11 21:59:49 +02:00
Shawn Pearce ed29dec1ea Expose LongMap in util package
This is a useful primitive collection type like IntList.

Change-Id: I04b9b2ba25247df056eb3a1725602f1be6d3b440
2017-08-09 10:42:09 -07:00
Shawn Pearce 40c9c59e07 NB: encode and decode 24-bit ints
Change-Id: Ie036dc46e5a88a4e87dc52e880505bbe34601ca7
2017-08-09 10:42:09 -07:00
Shawn Pearce 22201e8cca Update thread-safety warning about Repository
Change-Id: I1026a77cc688467d5a89a41121146f1bd3d56fa5
2017-08-08 06:44:35 -07:00
Dave Borowitz 8bbe34f27c ReflogWriter: Minor cleanup
Remove unnecessary finals, use consistent punctuation in Javadoc, reflow
some lines, etc.

Change-Id: Ic64db41c86917725ac649022290621406156bcc4
2017-08-02 16:52:34 -04:00
Dave Borowitz cf9662cdfe Eliminate SectionParser construction boilerplate
Happily, most anonymous SectionParser implementations can be replaced
with FooConfig::new, as long as the constructor takes a single Config
arg. Many of these, the non-public ones, can in turn be inlined. A few
remaining SectionParsers can be lambdas.

Change-Id: I3f563e752dfd2007dd3a48d6d313d20e2685943a
2017-08-02 16:50:57 -04:00
David Pursehouse 9e54cbb693 Merge changes Ia6c3935c,I46a29eae
* changes:
  Remove unused import introduced in a551b64
  Silence API errors caused by adding enum constants in dbb137e
2017-08-02 03:12:16 -04:00
Matthias Sohn 4d1334ce46 Merge "git ignore .DS_Store" 2017-08-01 17:30:00 -04:00
Matthias Sohn fd303105d5 Merge "git ignore all bazel folders" 2017-08-01 17:28:48 -04:00
Matthias Sohn ba85764223 Remove unused import introduced in a551b64
Change-Id: Ia6c3935cf061590e7305d0a80a1051e9aebcbb43
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-01 23:26:44 +02:00
Matthias Sohn 3eaa8d8e2a Silence API errors caused by adding enum constants in dbb137e
Change-Id: I46a29eae7b617f3f43f270c40072a1c103ef77f2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-08-01 23:26:42 +02:00
David Pursehouse 4085646f6d Merge changes I424295df,Ib003f7c8
* changes:
  Treat RawText of binary data as file with one single line.
  Trim boilerplate in RawParseUtils_LineMapTest.
2017-08-01 10:18:48 -04:00
Han-Wen Nienhuys a551b64694 Treat RawText of binary data as file with one single line.
This avoids executing mergeAlgorithm.merge on binary data, which is
unlikely to be useful.

Arguably, binary data should not make it to
ResolveMerger#contentMerge, but this approach has the following
advantages:

* binary detection is exact, since it doesn't only look at the start
  of the blob.

* it is cheap, as we have to iterate over the bytes anyway to find
  '\n'.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I424295df1dc60a719859d9d7c599067891b15792
2017-08-01 16:00:46 +02:00
Terry Parker 8c6a9a286e Merge "Use w1 for hashCode of AbbreviatedObjectId" 2017-07-28 19:24:11 -04:00
David Pursehouse 8391cc233b Merge "IntList: support contains(int)" 2017-07-28 14:18:21 -04:00
David Pursehouse 9f462a9914 Merge "Replace findbugs by spotbugs" 2017-07-28 13:47:21 -04:00
Shawn Pearce 4a00f18e8e Use w1 for hashCode of AbbreviatedObjectId
Very short abbreviations that are under 8 hex digits do not
have values in w2. Use w1 as the Java hashCode() instead, so
that the prefix of the abbreviation is always included in the
hashing function used by any java.util.Collection type.

Change-Id: Idaf69f86b62630ba4a022d31b4c293c6d138f557
2017-07-28 10:20:45 -07:00
Shawn Pearce 652a6b0334 IntList: support contains(int)
LongList supports contains(long).
IntList should also support contains(int).

Change-Id: Ic7a81c3c25b0f10d92087b56e9f200b676060f63
2017-07-28 10:18:27 -07:00
Matthias Sohn de7698476b Replace findbugs by spotbugs
SpotBugs [1] is the spiritual successor of FindBugs, carrying on from
the point where it left off with support of its community.

[1] http://spotbugs.readthedocs.io/

Change-Id: I127f2c54b04265b6565e780116617ffa8a4d7eaf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-07-28 16:15:54 +01:00
Dave Borowitz 45da0fc6f7 RefDirectory: Add in-process fair lock for atomic updates
In a server scenario such as Gerrit Code Review, there may be many
atomic BatchRefUpdates contending for locks on both the packed-refs file
and some subset of loose refs. We already retry lock acquisition to
improve this situation slightly, but we can do better by using an
in-process lock. This way, instead of retrying and potentially exceeding
their timeout, different threads sharing the same Repository instance
can wait on a fair lock without having to touch the disk lock. Since a
server is probably already using RepositoryCache anyway, there is a high
likelihood of reusing the Repository instance.

Change-Id: If5dd1dc58f0ce62f26131fd5965a0e21a80e8bd3
2017-07-28 11:03:32 -04:00
Dave Borowitz 6f23210781 RefDirectory: Retry acquiring ref locks with backoff
If a repo frequently uses PackedBatchRefUpdates, there is likely to be
contention on the packed-refs file, so it's not appropriate to fail
immediately the first time we fail to acquire a lock. Add some logic to
RefDirectory to support general retrying of lock acquisition.

Currently, there is a hard-coded wait starting at 100ms and backing off
exponentially to 1600ms, for about 3s of total wait. This is no worse
than the hard-coded backoff that JGit does elsewhere, e.g. in
FileUtils#delete. One can imagine a scheme that uses per-repository
configuration of backoff, and the current interface would support this
without changing any callers.

Change-Id: I4764e11270d9336882483eb698f67a78a401c251
2017-07-28 07:53:25 -04:00
David Pursehouse 2ff6ac0fae git ignore .DS_Store
Change-Id: I04887b91d07ee93f51d27c9a3597e258b9affc0b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-07-28 11:09:47 +01:00
David Pursehouse 6eef858cf6 git ignore all bazel folders
Change-Id: I38ae02d07020d8ccb8dc1ac4c0fa08ce93512f94
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-07-28 11:09:47 +01:00
David Pursehouse 5188c23104 Merge "Fix committing empty commits" 2017-07-28 06:08:33 -04:00
David Pursehouse 94aebcb949 Merge "Support overriding a batch's reflog on a per-ReceiveCommand basis" 2017-07-28 06:07:08 -04:00
Christian Halstrick da0770fdec Fix committing empty commits
Allow to explicitly create an empty commit even if committing only
certain files.

Bug: 510685 
Change-Id: If9bf664d7cd824f8e5bd6765fa6cc739af3d7721
2017-07-28 10:46:42 +01:00
David Pursehouse 7e4946626e Merge changes from topic 'batch-ref-update-reflog'
* changes:
  BatchRefUpdate: Expand javadocs and add @Nullable
  PackedBatchRefUpdate: Write reflogs
  Extract constants for reflog entry message prefixes
2017-07-28 05:40:45 -04:00
Han-Wen Nienhuys ab0eedcead Trim boilerplate in RawParseUtils_LineMapTest.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: Ib003f7c8f2816dd57e941799a665e70ecd6645a2
2017-07-27 13:49:39 +02:00
Zhen Chen b0695e5b7b Add commit check for head references
Make sure all refs/heads/* point to a commit object.

Change-Id: I9c7cf347aaf63d5ef604d520c2383c6cf3043890
Signed-off-by: Zhen Chen <czhen@google.com>
2017-07-26 10:12:37 -07:00
Zhen Chen 673acfc6bd Add connectivity check from references
Make sure all objects referenced by references are reachable. Stop at
the first missing object.

Change-Id: Ifcd7392c4321b17d9290bd87f038bc62bc10dabb
Signed-off-by: Zhen Chen <czhen@google.com>
2017-07-26 10:12:37 -07:00
Zhen Chen 2c2999643f Add dfs fsck implementation
JGit already had some fsck-like classes like ObjectChecker which can
check for an individual object.

The read-only FsckPackParser which will parse all objects within a pack
file and check it with ObjectChecker. It will also check the pack index
file against the object information from the pack parser.

Change-Id: Ifd8e0d28eb68ff0b8edd2b51b2fa3a50a544c855
Signed-off-by: Zhen Chen <czhen@google.com>
2017-07-26 10:12:29 -07:00
Dave Borowitz 104107bf43 Support overriding a batch's reflog on a per-ReceiveCommand basis
Change-Id: I86a4b8f6b4f85b2bae64c1b121e4ee527d46de83
2017-07-26 11:40:15 -04:00
Dave Borowitz a1e11461cc BatchRefUpdate: Expand javadocs and add @Nullable
Change-Id: I22d739a9677e24f36323dceadf7d375ac2f446e8
2017-07-26 11:39:39 -04:00
Dave Borowitz 22e9106224 PackedBatchRefUpdate: Write reflogs
On-disk reflogs are not stored in the packed-refs file, so we cannot
ensure atomic updates. We choose the lesser evil of dropping failed
reflog updates on the floor, rather than throwing an exception even
though the underlying ref updates succeeded.

Add tests for reflogs to BatchRefUpdateTest.

Change-Id: Ia456ba9e36af8e01fde81b19af46a72378e614cd
2017-07-26 11:39:33 -04:00
Dave Borowitz 82f68500c0 Improve BatchRefUpdateTest readability
* Factor out helpers for setting up and executing updates.
* Use common assert methods, with a special enum type that papers over
  the fact that there is no ReceiveCommand.Result for transaction
  aborted.
* Static import ReceiveCommand.Type constants.
* Add blank lines to separate repo setup, update execution, and asserts.

Change-Id: Ic3717f94331abfc7ae3e92065f3fe32026bf7cea
2017-07-25 13:14:50 -04:00
Dave Borowitz dbb137e0f3 Extract constants for reflog entry message prefixes
Document explicitly that these are untranslated to (mostly) match C git.

Change-Id: I3abcffb4fd611d053bf4373e5d6a14a66f7b9b6b
2017-07-25 13:14:50 -04:00
Dave Borowitz 65b64768b3 Move BatchRefUpdate tests to a new file
Run with @Parameterized, so we don't have to duplicate test setup for
each atomic/non-atomic test. We still have to have two different sets of
asserts for the cases where the behavior is different. In fact, this is
a readability win: it emphasizes that performing the exact same setup
except for the atomic setting will have different behavior.

Change-Id: I78a8214075e204732a423341f14c09de273a7854
2017-07-25 13:14:50 -04:00
Dave Borowitz 26962861d4 Implement atomic BatchRefUpdates for RefDirectory
The existing packed-refs file provides a mechanism for implementing
atomic multi-ref updates without any changes to the on-disk format or
lockfile protocol. We just need to make sure that there are no loose
refs involved in the transaction, which we can achieve by packing the
refs while holding locks on all loose refs. Full details of the
algorithm are in the PackedBatchRefUpdate javadoc.

This change does not implement reflog support, which will come in a
later change.

Change-Id: I09829544a0d4e8dbb141d28c748c3b96ef66fee1
2017-07-25 13:14:50 -04:00
Dave Borowitz cf9e3fad52 Separate RefUpdate.Result.REJECTED_{MISSING_OBJECT,OTHER_REASON}
ReceiveCommand.Result has a slightly richer set of possibilities, so it
makes sense for RefUpdate.Result to have more values in order to match.
In particular, this allows us to return REJECTED_MISSING_OBJECT from
RefUpdate when an object is missing.

The comment in RefUpdate#safeParse about expecting some old objects to be
missing is only applicable to the old ID, not the new ID. A missing new
ID is a bug or programmer error, and we should not update a ref to point
to one.

Fix various tests that started failing because they depended for no good
reason on setting refs to point to nonexistent objects; it's always easy
to create a real object when necessary.

It is possible that some downstream users of RefUpdate.Result might
choose to handle one of the new statuses differently, for example by
providing a more user-readable error message; that is not done in this
change.

Change-Id: I734b1c32d5404752447d9e20329471436ffe05fc
2017-07-25 13:12:34 -04:00
David Pursehouse 4940ea14b7 Add missing newlines at ends of Java files
Change-Id: Iead36f53d57ead0eb3edd3f9efb63b6630c9c20c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-07-25 10:37:21 +01:00
Dave Borowitz e93b0026ce Temporarily @Ignore flaky CommitCommandTest methods
Change-Id: Ia2c42d014323bd29b85bf76f1a20c83f612406d7
2017-07-25 05:17:12 -04:00
Joan Goyeau 826e22e7cc Fix JGit set core.fileMode to false by default instead of true for non Windows OS.
Bug: 519887
Change-Id: I4ae0d6783a9dc62f78ead54ddd1ab2b5b66a811c
Signed-off-by: Joan Goyeau <joan@goyeau.com>
2017-07-24 13:57:21 +01:00
Dmitry Pavlenko 843e444561 Fix matching ignores and attributes pattern of form a/b/**.
Fix patch matching for patterns of form a/b/** : this should not match
paths like a/b but still match a/b/ and a/b/c.

Change-Id: Iacbf496a43f01312e7d9052f29c3f9c33807c85d
Signed-off-by: Dmitry Pavlenko <pavlenko@tmatesoft.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-07-24 09:16:33 +01:00
David Pursehouse ba91e8a086 Merge changes from topic 'packed-batch-ref-update'
* changes:
  Add tests for updating single refs to missing objects
  Fix deleting symrefs
  RefDirectory: Throw exception if CAS of packed ref list fails
  ReceiveCommand: Explicitly check constructor preconditions
  BatchRefUpdate: Document when getPushOptions is null
2017-07-24 03:38:42 -04:00
Shawn Pearce ad269ae426 Merge "Make 'inCoreLimit' of LocalFile used in ResolveMerger configurable" 2017-07-22 18:45:56 -04:00
Changcheng Xiao 1baf86d4d2 Make 'inCoreLimit' of LocalFile used in ResolveMerger configurable
This change makes it possible to configure the 'inCoreLimit' of LocalFile
used in ResolveMerger#insertMergeResult. Since LocalFile itself has some
risks, e.g. it may be left behind as garbage in case of failure. It should
be good to be able to control the size limit for using LocalFile.

Change-Id: I3dc545ade370b2bbdb7c610ed45d5dd4d39b9e8e
Signed-off-by: Changcheng Xiao <xchangcheng@google.com>
2017-07-22 21:51:12 +02:00
Shawn Pearce d4cfa95ba3 dfs: optionally store blockSize in DfsPackDescription
Allow a DFS implementation to report blockSize to DfsPackFile,
bypassing alignment errors and corrections in the DfsBlockCache when
the blockSize of a specific file differs from the cache's configured
blockSize.

Change-Id: Ic376314d4a86a0bd528c033e169d93eef035b233
2017-07-21 08:33:17 -07:00
Shawn Pearce a1e4825c4e dfs: silence resource warnings in DfsBlockCacheTest
Change-Id: Ia934d8578592dc20837944d50acfb8920e260893
2017-07-19 14:59:50 -07:00