Commit Graph

3201 Commits

Author SHA1 Message Date
Dave Borowitz 2ccea7f05a ChangeIdUtil: Don't throw IOException
This could have only happened during the getBytes call. Instead, use
Constants.encode, which is a non-throwing implementation.

This change is binary compatible with existing code compiled against
older versions of JGit, although it might break compilation of
previously compiling code due to dead catch blocks.

Change-Id: I191fec5cac718657407230de141440e86d0151fb
2016-01-19 17:59:19 -05:00
Shawn Pearce a693d7b33c Merge "Transport: Implement AutoCloseable" 2016-01-19 17:36:39 -05:00
Shawn Pearce 6b662af76e Transport: Implement AutoCloseable
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.

Change-Id: I0c6ff3e39ebecdd7a028dbcae1856a818937b186
2016-01-19 14:27:11 -08:00
Shawn Pearce 68a6d0c0ab Merge changes from topic 'deadtree'
* changes:
  Ignore API errors about Tree, TreeEntry, FileTreeEntry and friends being removed
  Revert "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends""
2016-01-19 17:08:54 -05:00
Shawn Pearce 9d3b56a905 Merge "ReceiveCommand.abort(): Utility to mark batch of commands as failed" 2016-01-19 15:54:07 -05:00
Shawn Pearce 462017e02b Ignore API errors about Tree, TreeEntry, FileTreeEntry and friends being
removed

Bug: 486105
Change-Id: I04adcdb68bee7d5f608bb7ab959fe36a890f9ecd
2016-01-19 18:49:11 +01:00
Andrey Loskutov 7f31a9c9ad Revert "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and
friends""

This reverts commit 2cc80187d3.

Bug: 486105
Change-Id: Id4f9987c33d66cbed9de6e4d4d6784afdd01a3cf
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-19 18:48:48 +01:00
Andrey Loskutov 2006e90abc Annotated to be removed Tree API with @noreference and @noextend
See https://wiki.eclipse.org/Eclipse/API_Central/API_Removal_Process.

Bug: 486105
Change-Id: I460e43da0d487279608729a2081c614e7065f56f
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-19 14:27:41 +01:00
Shawn Pearce eadfcd3ec1 ReceiveCommand.abort(): Utility to mark batch of commands as failed
If one or more commands is failing the entire group usually has to
also fail with "transaction aborted". Pull this loop into a helper
so the idiom can be easily reused in several places throughout JGit.

Change-Id: I3b9399b7e26ce2b0dc5f7baa85d585a433b4eaed
2016-01-18 11:39:09 -08:00
Shawn Pearce f52581c6a5 Merge "Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends"" 2016-01-15 10:45:43 -05:00
Shawn Pearce 2cc80187d3 Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends"
This reverts commit 0f8743d4d7.

JGit is unable to iterate its API.

Change-Id: Ie3d6a28e622a5c0cf54768a2299f1c44c0114c19
2016-01-15 10:45:36 -05:00
Shawn Pearce 95bdc589b8 Merge "PackWriter: Declare preparePack object sets as @NonNull" 2016-01-12 19:45:40 -05:00
Shawn Pearce 4e650c0d76 PackWriter: Declare preparePack object sets as @NonNull
Require callers to pass in valid sets for both want and have
collections. Offer PackWriter.NONE as a handy constant for an
empty collection for the have part of preparePack instead of null.

Change-Id: Ifda4450f5e488cbfefd728382b7d30797e229217
2016-01-12 19:30:32 -05:00
Shawn Pearce 17580101f5 Merge "GC: Pack RefTrees in their own pack" 2016-01-12 18:49:02 -05:00
Shawn Pearce 40051505d7 GC: Pack RefTrees in their own pack
The RefTree graph needs to be quickly accessed to read references.
It is also distinct graph disconnected from the rest of the
repository. Store the commit and tree objects in their own pack.

Change-Id: Icbb735be8fa91ccbf0708ca3a219b364e11a6b83
2016-01-12 12:15:53 -08:00
Shawn Pearce 9e3d391953 Change to extensions.refsStorage
git-core just rerolled the extensible backends series with refsStorage
as the configuration key. Update JGit to match git-core.

Change-Id: If345a2403a996e358b29cfa2a2298f6e8d59d96b
2016-01-11 20:44:10 -08:00
Shawn Pearce 398d8e877f RefTree: Change peel suffix to " ^" (space carrot)
Using ^{} as the peel suffix has caused problems when projects used
tags like v2.1 and then v2.1.1, v2.2.2, etc.  The peeled value for
v2.1 was stored very far away in the tree relative to v2.1 itself as
^ sorts in the ASCII/UTF-8 encoding after all other common tag
characters like digits and dots.

Use " ^" instead as space is not valid in a reference name, sorts
before all other valid reference characters (thus forcing next entry
locality) and this looks like a peeled marker for the prior tag.

Change-Id: I26d2247a0428dfe26a9c319c02159502b3a67455
2016-01-11 15:27:14 -08:00
Shawn Pearce 088c2fc6e3 FileRepository: Support extensions.refsBackendType = RefTree
This experimental code can be enabled in $GIT_DIR/config:

  [core]
    repositoryformatversion = 1

  [extensions]
    refsBackendType = RefTree

When these are set the repository will read references from the
RefTree rooted by the $GIT_DIR/refs/txn/committed reference.

Update debug-rebuild-ref-tree to rebuild refs/txn/committed only from
the bootstrap layer.  This avoids misuse by rebuilding using packed-refs
and $GIT_DIR/refs tree.

Change-Id: Icf600e4a36b2f7867822a7ab1f1617d73c710a4b
2016-01-11 15:27:07 -08:00
Shawn Pearce 48e245fc60 RefTreeDatabase: Ref database using refs/txn/committed
Instead of storing references in the local filesystem rely on the
RefTree rooted at refs/txn/committed.  This avoids needing to store
references in the packed-refs file by keeping all data rooted under
a single refs/txn/committed ref.

Performance to scan all references from a well packed RefTree is very
close to reading the packed-refs file from local disk.

Storing a packed RefTree is smaller due to pack file compression,
about 49.39 bytes/ref (on average) compared to packed-refs using
~65.49 bytes/ref.

Change-Id: I75caa631162dc127a780095066195cbacc746d49
2016-01-11 15:26:42 -08:00
Shawn Pearce 8f31aa7c28 Merge "RevCommit: Better support invalid encoding headers" 2016-01-11 15:48:16 -05:00
Shawn Pearce 31d92ace5b RevCommit: Better support invalid encoding headers
With this support we no longer need the 'utf-8' alias. UTF-8 will be
automatically tried when the encoding header is not recognized and used
if the character sequence cleanly decodes as UTF-8.

Modernize some of the references to use StandardCharsets.

Change-Id: I4c0c88750475560e1f2263180c4a98eb8febeca0
2016-01-11 12:30:35 -08:00
Shawn Pearce 0f8743d4d7 Remove deprecated Tree, TreeEntry, FileTreeEntry and friends
These types were deprecated in 0.9.1 (aka 384a19eee0).
If anyone is still using them, its time to stop.

Change-Id: I3f73347ba78c639e0c6a504812bc1a0702f829b1
2016-01-09 09:45:33 -08:00
Shawn Pearce 1243e25aad Paths.pathCompare: Utility to sort paths from byte[]
Consolidate copies of this function into one location.

Add some unit tests to prevent bugs that were accidentally
introduced while trying to make this refactoring.

Change-Id: I82f64bbb8601ca2d8316ca57ae8119df32bb5c08
2016-01-09 09:44:57 -08:00
Shawn Pearce bace383507 Add Paths utility class
Simple container for some path related utility functions.

Change-Id: Ice2bec6ad12b1e2cea15988c01aa9dd4e016a849
2016-01-08 16:58:23 -08:00
Shawn Pearce 73eb32be82 RefTree: Store references in a Git tree
A group of updates can be applied by updating the tree in one step,
writing out a new root tree, and storing its SHA-1. If references
are stored in RefTrees, comparing two repositories is a matter of
checking if two SHA-1s are identical. Without RefTrees comparing two
repositories requires listing all references and comparing the sets.

Track the "refs/" directory as a root tree by storing references
that point directly at an object as a GITLINK entry in the tree.
For example "refs/heads/master" is written as "heads/master".

Annotated tags also store their peeled value with ^{} suffix, using
"tags/v1.0" and "tags/v1.0^{}" GITLINK entries.

Symbolic references are written as SYMLINK entries with the blob of
the symlink carrying the name of the symbolic reference target.

HEAD is outside of "refs/" namespace so it is stored as a special
"..HEAD" entry. This name is chosen because ".." is not valid in
a reference name and it almost looks like "../HEAD" which names
HEAD if the reader was inside of the "refs/" directory.

A new Command type is required to handle symbolic references and
peeled references.

Change-Id: Id47e5d4d32149a9e500854147edd7d93c1041a39
2016-01-07 15:40:26 -08:00
Shawn Pearce 5001038f52 Merge "InMemoryRepository: Abort BatchRefUpdate if a command previously failed" 2016-01-06 15:18:04 -05:00
Shawn Pearce 4c9eda17be InMemoryRepository: Abort BatchRefUpdate if a command previously failed
If any command has already been marked as failing, fail the entire batch.

Change-Id: I1692240841aa4f4cb252bdccbc6f11d9246929c1
2016-01-06 12:05:46 -08:00
Shawn Pearce f0d634eed7 DFS: Allow other RefDatabase implementations
Permit a DfsRepository implementation to use a different RefDatabase
than DfsRefDatabase.

Change-Id: Ia263285f547bde1943993cc994d0222185021a16
2016-01-06 11:54:45 -08:00
Matthias Sohn 91a3f17474 Merge changes Idaed9310,I62bd7c0b
* changes:
  ObjectChecker: honor some git-core fsck.* options
  ObjectChecker: allow some objects to skip errors
2016-01-03 10:27:37 -05:00
Shawn Pearce 163be57d0f Merge changes from topic 'add-df'
* changes:
  DirCache: Do not create duplicate tree entries
  DirCacheEditor: Replace file-with-tree and tree-with-file
  AddCommand: Use NameConflictTreeWalk to identify file-dir changes
2016-01-01 12:58:35 -05:00
Shawn Pearce 09500165a8 Fix "remote: Counting objects: ..." formatting
Trailing whitespace is usually removed in properties files so
JGitText did not supply a space between : and the remote message.
Ensure the space exists at runtime by reading the localized string
and appending a space if it is missing.

Messages should be dynamically fetched and not held in a static
class variable, as they can be changed using thread locals.

Change-Id: If6a3707d64094253b1a5304fbfafcf195db7497a
2015-12-31 17:37:16 -08:00
David Ostrovsky 13502fef8f Implement Buck driven build
Today there are plenty of modern build tool systems available in the
wild (in no particular order):

* http://bazel.io
* https://pantsbuild.github.io
* http://shakebuild.com
* https://ninja-build.org
* https://buckbuild.com

The attributes, that all these build tools have in common, are:

* reliable
* correct
* very fast
* reproducible

It must not always be the other build tool, this project is currently
using. Or, quoting Gerrit Code Review maintainer here:

  "Friends, don't let friends use <the other build tool system>!"

This change is non-complete implementation of JGit build in Buck,
needed by Gerrit Code Review to replace its dependency with standlone
JGit cell. This is very useful when a developer is working on both
projects and is trying to integrate changes made in JGit in Gerrit.

The supported workflow is:

  $ cd jgit
  $ emacs <hack>
  $ cd ../gerrit
  $ buck build --config repositories.jgit=../jgit gerrit

With --config repositories.jgit=../jgit jgit cell is routed through
JGit development tree.

To build jgit, issue:

  $ buck build //:jgit
  [-] PROCESSING BUCK FILES...FINISHED 0,0s

Yes, you can't measure no-op build time, given that Buck daemon is
used.

Change-Id: I301a71b19fba35a5093d8cc64d4ba970c2877a44
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2015-12-31 10:08:55 -08:00
Shawn Pearce 2fbd98dbe8 Merge "Fix encoding problem from curl repostory on github" 2015-12-30 23:47:28 -05:00
Shawn Pearce 80edcac06f Merge "Fix hanging fetch via SSH" 2015-12-30 23:27:09 -05:00
Eryk Szymanski 53c3bbe42f Fix encoding problem from curl repostory on github
Pushing curl repository to gerrit fails with a message:
remote: error: internal error while processing changes
java.nio.charset.IllegalCharsetNameException: 'utf8'

curl repository url: https://github.com/bagder/curl.git

To avoid this problem encodingAliases in RawParseUtils have
been extended to contain "'utf8'" (single quoted utf8) string.

Change-Id: I40f613cfdcabf0dc9455bee45116ab8d8c7dd6ee
Signed-off-by: Eryk Szymanski <eryksz@gmail.com>
2015-12-30 23:24:12 -05:00
Shawn Pearce ac41920a43 ObjectChecker: honor some git-core fsck.* options
Accept some of the same section keys that fsck does in git-core,
allowing repositories to skip over specific kinds of acceptable
broken objects, e.g.:

  [fsck]
    duplicateEntries = ignore
    zeroPaddedFilemode = ignore

The zeroPaddedFilemode = ignore is a synonym for the JGit specific
allowLeadingZeroFileMode = true. Only accept the JGit key if git-core
key was not specified.

Change-Id: Idaed9310e2a5ce5511670ead1aaea2b30aac903c
2015-12-30 15:19:09 -08:00
Shawn Pearce fa7ce0e0f3 ObjectChecker: allow some objects to skip errors
Some ancient objects may be broken, but in a relatively harmless way.
Allow the ObjectChecker caller to whitelist specific objects that are
going to fail checks, but that have been reviewed by a human and decided
the objects are OK enough to permit continued use of.

This avoids needing to rewrite history to scrub the broken objects out.

Honor the git-core fsck.skipList configuration setting when receiving a
push or fetching from a remote repository.

Change-Id: I62bd7c0b0848981f73dd7c752860fd02794233a6
2015-12-30 15:18:48 -08:00
Matthias Sohn 3fc93f8a56 Rename files using NIO2 atomic rename
Bug: 319233
Change-Id: I5137212f5cd3195a52f90ed5e4ce3cf194a13efd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2015-12-30 17:22:33 -05:00
Shawn Pearce e3acf01748 ObjectChecker: use java.text.Normalizer directly
Base Java version for JGit is now Java 7. The java.text.Normalizer
class was available in Java 6. Reflection is no longer required to
normalize strings for Mac OS X.

Change-Id: I98e14b72629a7a729a2d40a3aa275932841274e8
2015-12-30 15:45:13 -05:00
Shawn Pearce 7177b506b4 Merge changes I3db332bf,I614e7110,I37a4ccd6,I86b81e7f,Ic8b362e0, ...
* changes:
  Sort "eager" path-like options to the end of the help
  reset command: provide convenient and meaningful options help
  commit command: allow to specify path(s) argument(s)
  status command: consume more then one argument after --
  repo command: properly name the required 'path' argument
  Un-ignored existing CLI tests which run just fine on Java 7+
  Don't treat command termination due '-h' option as a fatal error
2015-12-30 15:44:30 -05:00
Shawn Pearce 018a91b715 Unify fetch and receive ObjectChecker setup
This avoids duplication of code between receive-pack and fetch-pack paths.
Separate methods are still required to check use of receive.fsckobjects vs.
fetch.fsckobjects, both of which default to transfer.fsckobjects.

Change-Id: I41193e093e981a79fc2f63914e273aaa44b82162
2015-12-29 16:55:41 -08:00
Shawn Pearce 29aa444760 PackWriter: use lib.ObjectIdSet to avoid wrapper
Hoist ObjectIdSet up to lib as part of the public API and add
the interface to some common types like PackIndex and JGit custom
ObjectId map types.  This cleans up wrapper code in a number of
places by allowing direct use of the types as an ObjectIdSet.

Future commits can now rely on ObjectIdSet as a simple read-only
type to check a set of objects from a number of storage options.

Change-Id: Ib62b062421d475bd52abd6c84a73916ef36e084b
2015-12-29 15:32:24 -08:00
Shawn Pearce 683c41af92 DirCache: Do not create duplicate tree entries
If a file (e.g.  "A") and a subtree file (e.g. "A/foo.c") both appear
in the DirCache this cache should not be written out as a tree object.
The "A" file and "A" subtree conflict with each other in the same tree
and will fail fsck.

Detect this condition during DirCacheBuilder and DirCacheEditor
finish() so the application can be halted early before it updates a
DirCache that might later write an invalid tree structure.

Change-Id: I95660787e88df336297949b383f4c5fda52e75f5
2015-12-29 11:33:39 -08:00
Shawn Pearce b71ba69410 DirCacheEditor: Replace file-with-tree and tree-with-file
If a PathEdit tries to store a file where a subtree was, or a subtree
where a file was, replace the entry in the DirCache with the new
name(s).  This supports switching between file and tree entry types
using a DirCacheEditor.

Add new unit tests to cover the conditions where these can happen.

Change-Id: Ie843d9388825f9e3d918a5666aa04e47cd6306e7
2015-12-29 11:33:39 -08:00
Shawn Pearce 3776b14ab4 AddCommand: Use NameConflictTreeWalk to identify file-dir changes
Adding a path that already exists but is changing type such as
from symlink to subdirectory requires a NameConflictTreeWalk to
match up the two different entry types that share the same name.

NameConflictTreeWalk needs a bug fix to pop conflicting entries
when PathFilterGroup aborts the walk early so that it does not
allow DirCacheBuilderIterator to copy conflicting entries into
the output cache.

Change-Id: I61b49cbe949ca8b4b98f9eb6dbe7b1f82eabb724
2015-12-29 11:33:35 -08:00
Andrey Loskutov 0505657d6a commit command: allow to specify path(s) argument(s)
This fixes the command below:

jgit commit a -m "added file a"

which currently fails with:

org.eclipse.jgit.api.errors.JGitInternalException: The combination of
arguments --all and --only is not allowed

Bug: 484973
Change-Id: I37a4ccd68101a66520ef99110f7aa0cbdcc8beba
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-29 15:00:09 +01:00
Andrey Loskutov 241b50be31 Simplify development of commands: added main() to CLIGitCommand
This will execute git commands (with arguments) specified on the command
line, handy for developing/debugging a sequence of arbitrary git
commands working on same repository.

The git working dir path can be specified via Java system property
"git_work_tree". If not specified, current directory will be used.

Change-Id: I621a9ec198c31e28a383818efeb4b3f835ba1d6f
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-28 22:59:59 +01:00
Shawn Pearce ef757a7e12 DirCacheEditor: Cleanup DeleteTree constructor
Neaten up formatting and avoid strings, which prevents the need for
NLS comment tags.  Instead check the last character using char
literal, and append a char literal instead of a string.

Change-Id: Ib68e017769a1f5c03200354a805769d585a48c8b
2015-12-28 11:43:31 -08:00
Shawn Pearce 110d3ca595 DirCacheEditor: Fix formatting to avoid , at start of line
Change-Id: I1b1d614470c67fe4736fdc9c26ae26fb38dd58b5
2015-12-28 11:43:30 -08:00
Shawn Pearce 21cec184b7 Merge "AddCommand: Avoid unnecessary string conversions" 2015-12-28 14:24:33 -05:00