Commit Graph

4322 Commits

Author SHA1 Message Date
Shawn Pearce 2cc80187d3 Revert "Remove deprecated Tree, TreeEntry, FileTreeEntry and friends"
This reverts commit 0f8743d4d7.

JGit is unable to iterate its API.

Change-Id: Ie3d6a28e622a5c0cf54768a2299f1c44c0114c19
2016-01-15 10:45:36 -05:00
David Pursehouse 559e493648 RevParse: Remove superfluous semicolon
Change-Id: I5975bc58d7933dafb3e7a8d891e9f6878b98a9a1
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-14 15:17:46 +09:00
David Pursehouse 0352276da7 RefUpdateTest: Use try-with-resource for auto-closable types
Change-Id: I25c8db0e410aa2a4a53dc7d8863e4a6efefcf7cb
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-14 15:09:53 +09:00
David Pursehouse 56634de0cc RefUpdateTest: Add null check to prevent potential NPE
Change-Id: I64b1850c3bb80b09409044461ca3fdde1c46d4bb
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-14 15:06:10 +09:00
David Pursehouse 1ed5382b37 CommitCommand: Remove redundant null check
Repository.getWorkTree is annotated as @NonNull, so the check
for it returning null is redundant.

Change-Id: I597b0f774ff857b8900519f14a1a17a904cf7c6f
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-14 15:04:34 +09:00
Shawn Pearce 95bdc589b8 Merge "PackWriter: Declare preparePack object sets as @NonNull" 2016-01-12 19:45:40 -05:00
Shawn Pearce 4e650c0d76 PackWriter: Declare preparePack object sets as @NonNull
Require callers to pass in valid sets for both want and have
collections. Offer PackWriter.NONE as a handy constant for an
empty collection for the have part of preparePack instead of null.

Change-Id: Ifda4450f5e488cbfefd728382b7d30797e229217
2016-01-12 19:30:32 -05:00
Shawn Pearce 17580101f5 Merge "GC: Pack RefTrees in their own pack" 2016-01-12 18:49:02 -05:00
Shawn Pearce 40051505d7 GC: Pack RefTrees in their own pack
The RefTree graph needs to be quickly accessed to read references.
It is also distinct graph disconnected from the rest of the
repository. Store the commit and tree objects in their own pack.

Change-Id: Icbb735be8fa91ccbf0708ca3a219b364e11a6b83
2016-01-12 12:15:53 -08:00
Shawn Pearce 9e3d391953 Change to extensions.refsStorage
git-core just rerolled the extensible backends series with refsStorage
as the configuration key. Update JGit to match git-core.

Change-Id: If345a2403a996e358b29cfa2a2298f6e8d59d96b
2016-01-11 20:44:10 -08:00
Shawn Pearce 398d8e877f RefTree: Change peel suffix to " ^" (space carrot)
Using ^{} as the peel suffix has caused problems when projects used
tags like v2.1 and then v2.1.1, v2.2.2, etc.  The peeled value for
v2.1 was stored very far away in the tree relative to v2.1 itself as
^ sorts in the ASCII/UTF-8 encoding after all other common tag
characters like digits and dots.

Use " ^" instead as space is not valid in a reference name, sorts
before all other valid reference characters (thus forcing next entry
locality) and this looks like a peeled marker for the prior tag.

Change-Id: I26d2247a0428dfe26a9c319c02159502b3a67455
2016-01-11 15:27:14 -08:00
Shawn Pearce 088c2fc6e3 FileRepository: Support extensions.refsBackendType = RefTree
This experimental code can be enabled in $GIT_DIR/config:

  [core]
    repositoryformatversion = 1

  [extensions]
    refsBackendType = RefTree

When these are set the repository will read references from the
RefTree rooted by the $GIT_DIR/refs/txn/committed reference.

Update debug-rebuild-ref-tree to rebuild refs/txn/committed only from
the bootstrap layer.  This avoids misuse by rebuilding using packed-refs
and $GIT_DIR/refs tree.

Change-Id: Icf600e4a36b2f7867822a7ab1f1617d73c710a4b
2016-01-11 15:27:07 -08:00
Shawn Pearce 48e245fc60 RefTreeDatabase: Ref database using refs/txn/committed
Instead of storing references in the local filesystem rely on the
RefTree rooted at refs/txn/committed.  This avoids needing to store
references in the packed-refs file by keeping all data rooted under
a single refs/txn/committed ref.

Performance to scan all references from a well packed RefTree is very
close to reading the packed-refs file from local disk.

Storing a packed RefTree is smaller due to pack file compression,
about 49.39 bytes/ref (on average) compared to packed-refs using
~65.49 bytes/ref.

Change-Id: I75caa631162dc127a780095066195cbacc746d49
2016-01-11 15:26:42 -08:00
Shawn Pearce 8f31aa7c28 Merge "RevCommit: Better support invalid encoding headers" 2016-01-11 15:48:16 -05:00
Shawn Pearce 31d92ace5b RevCommit: Better support invalid encoding headers
With this support we no longer need the 'utf-8' alias. UTF-8 will be
automatically tried when the encoding header is not recognized and used
if the character sequence cleanly decodes as UTF-8.

Modernize some of the references to use StandardCharsets.

Change-Id: I4c0c88750475560e1f2263180c4a98eb8febeca0
2016-01-11 12:30:35 -08:00
Shawn Pearce 0f8743d4d7 Remove deprecated Tree, TreeEntry, FileTreeEntry and friends
These types were deprecated in 0.9.1 (aka 384a19eee0).
If anyone is still using them, its time to stop.

Change-Id: I3f73347ba78c639e0c6a504812bc1a0702f829b1
2016-01-09 09:45:33 -08:00
Shawn Pearce 1243e25aad Paths.pathCompare: Utility to sort paths from byte[]
Consolidate copies of this function into one location.

Add some unit tests to prevent bugs that were accidentally
introduced while trying to make this refactoring.

Change-Id: I82f64bbb8601ca2d8316ca57ae8119df32bb5c08
2016-01-09 09:44:57 -08:00
Shawn Pearce bace383507 Add Paths utility class
Simple container for some path related utility functions.

Change-Id: Ice2bec6ad12b1e2cea15988c01aa9dd4e016a849
2016-01-08 16:58:23 -08:00
Shawn Pearce 24a72b6772 Merge changes from topic 'reftree'
* changes:
  debug-rebuild-ref-tree: Simple program to build a RefTree
  RefTree: Store references in a Git tree
2016-01-07 19:21:56 -05:00
Shawn Pearce 52eb040c55 debug-rebuild-ref-tree: Simple program to build a RefTree
This tool scans all references in the repository and writes out a new
reference pointing to a single commit whose root tree is a RefTree
containing the current refs of this repository.

It alway skips storing the reference it will write to, avoiding the
obvious cycle.

Change-Id: I20b1eeb81c55dc49dd600eac3bf8f90297394113
2016-01-07 15:43:50 -08:00
Shawn Pearce 73eb32be82 RefTree: Store references in a Git tree
A group of updates can be applied by updating the tree in one step,
writing out a new root tree, and storing its SHA-1. If references
are stored in RefTrees, comparing two repositories is a matter of
checking if two SHA-1s are identical. Without RefTrees comparing two
repositories requires listing all references and comparing the sets.

Track the "refs/" directory as a root tree by storing references
that point directly at an object as a GITLINK entry in the tree.
For example "refs/heads/master" is written as "heads/master".

Annotated tags also store their peeled value with ^{} suffix, using
"tags/v1.0" and "tags/v1.0^{}" GITLINK entries.

Symbolic references are written as SYMLINK entries with the blob of
the symlink carrying the name of the symbolic reference target.

HEAD is outside of "refs/" namespace so it is stored as a special
"..HEAD" entry. This name is chosen because ".." is not valid in
a reference name and it almost looks like "../HEAD" which names
HEAD if the reader was inside of the "refs/" directory.

A new Command type is required to handle symbolic references and
peeled references.

Change-Id: Id47e5d4d32149a9e500854147edd7d93c1041a39
2016-01-07 15:40:26 -08:00
Andrey Loskutov 7780a4ee31 Make sure CLIGitCommand and Main produce (almost) same results
Currently execution of tests in pgm uses CLIGitCommand which
re-implements few things from Main. Unfortunately this can results in a
different test behavior compared to the real CLI runtime.

The change let CLIGitCommand extend Main and only slightly modifies the
runtime (stream redirection and undesired exit() termination).

Change-Id: I87b7b61d1c84a89e5917610d84409f01be90b70b
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-06 17:27:31 -05:00
Andrey Loskutov fb5056c2c5 branch command: print help if requested, even if arguments are wrong
git branch -d -h reports an error (because of missing -d option value)
but does not print the help as expected.

To fix this, CmdLineParser must catch, print but do not propagate
exceptions if help is requested.

Bug: 484951
Change-Id: I51265ebe295f22da540792c6a1980b8bdb295a02
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-06 17:27:15 -05:00
Andrey Loskutov 24468f09e3 Added CLIText.fatalError(String) API for tests
In different places (Main, TextBuiltin, CLIGitCommand) we report fatal
errors and at same time want to check for fatal errors in the tests.
Using common API simplifies the error testing and helps to navigate to
the actual error check implementation.

Change-Id: Iecde79beb33ea595171f168f46b0b10ab2f339bb
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-06 17:26:43 -05:00
Shawn Pearce 24cd8e170d GitServletResponseTest: Fix testObjectCheckerException
The recent ObjectChecker changes to pass in AnyObjectId as part
of the checkCommit method signature meant the override here was no
longer throwing an exception as expected.

Change-Id: I0383018b48426e25a0bc562387e8cd73cbe13129
2016-01-06 15:18:15 -05:00
Shawn Pearce 5001038f52 Merge "InMemoryRepository: Abort BatchRefUpdate if a command previously failed" 2016-01-06 15:18:04 -05:00
Shawn Pearce 4c9eda17be InMemoryRepository: Abort BatchRefUpdate if a command previously failed
If any command has already been marked as failing, fail the entire batch.

Change-Id: I1692240841aa4f4cb252bdccbc6f11d9246929c1
2016-01-06 12:05:46 -08:00
Shawn Pearce f0d634eed7 DFS: Allow other RefDatabase implementations
Permit a DfsRepository implementation to use a different RefDatabase
than DfsRefDatabase.

Change-Id: Ia263285f547bde1943993cc994d0222185021a16
2016-01-06 11:54:45 -08:00
David Ostrovsky 4c574b39b4 buck: Make :jgit_src target work in cross-cell environment
This artifact is used from unzip utility in Gerrit Code Review
build toolchain and thus the file must exist on the file system.
Moreover, trying to use java_binary() didn't work either, as the
zip layout was wrong: all files contained 'org.eclipse.jgit/src/'
prefix.

Change-Id: I00e3269a7a1a6c6d1fe7e60d1bf1c69b8e57d79d
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2016-01-04 00:53:06 +01:00
Shawn Pearce 8d5d144530 Merge changes I487a30fb,I82a01b5f
* changes:
  Ensure all http tests are run and fix broken tests
  buck: run http tests
2016-01-03 18:24:34 -05:00
Shawn Pearce af36abd02f Merge "buck: set Bundle-Version for :jgit_bin" 2016-01-03 13:22:59 -05:00
Shawn Pearce 776c3003aa buck: set Bundle-Version for :jgit_bin
Run git describe during the build to determine the lineage of this
working directory and stamp this information into the binary.

Change-Id: I0ad24125c31e4280ccf900bac4065924087b05aa
2016-01-03 09:30:16 -08:00
Matthias Sohn 91a3f17474 Merge changes Idaed9310,I62bd7c0b
* changes:
  ObjectChecker: honor some git-core fsck.* options
  ObjectChecker: allow some objects to skip errors
2016-01-03 10:27:37 -05:00
Andrey Loskutov 63bb0bcd4a branch command: provide convenient and meaningful options help
Added tests for all options, fixed multi-valued options parsing.

Bug: 484951
Change-Id: I5558589049544ea6c84932bc01f1f9df09e1f682
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-02 17:41:59 +01:00
Andrey Loskutov d6deb190e6 Simplify pgm tests: allow varargs and trim output for toString()
Change-Id: Ia5bcd9e560b90cf872fef75c2800c889ef1cc85a
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-02 17:41:58 +01:00
Andrey Loskutov 35db319e8a Fixed few locale dependent pgm tests
See https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03040.html

Change-Id: If51f3c750684d82cb6443f1578636c9f5ca56e2b
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2016-01-02 17:41:58 +01:00
Matthias Sohn 80bf52dfe4 Ensure all http tests are run and fix broken tests
HttpClientTests were broken. This wasn't discovered since
maven-surefire-plugin's by default only executes test classes
matching **/*Test.java. Fix this by also including **/*.Tests.java
and fix the failing tests.

Change-Id: I487a30fb333de993a9f8d8fff491d3b0e7fb02cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-02 02:16:13 +01:00
Matthias Sohn f1b79c3c89 buck: run http tests
Running tests using buck reveals that HttpClientTests are broken
and weren't executed by Maven since these test classes don't match the
maven-surefire-plugin's default for test classes **/*Test.java.
Will be fixed in a follow-up change.

Change-Id: I82a01b5fd3f0a930bec2423a29a256601dadc248
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-02 02:02:46 +01:00
Shawn Pearce da3174a812 buck: set vm_args for tests
Maven pom files force the local encoding to UTF-8 to ensure there are
no differences between machines.  They also set the JVM max heap to
256m. Match both in Buck so that results are consistent.

Change-Id: Ice5476dd09352a444a0c97aa0dc28806fddf2ab4
2016-01-01 11:04:11 -08:00
Shawn Pearce 887debc527 buck: pin to stable version
Like with Gerrit, pin JGit to a single version of Buck that is known
to work with current Buck files and JUnit tests. Notably a more recent
version of Buck used by Gerrit (01a0c54d827) fails WalkEncryptionTest.

Change-Id: I6b94c332e4bde97a1910f48cf12eb8698f97d540
2016-01-01 10:43:20 -08:00
Shawn Pearce 163be57d0f Merge changes from topic 'add-df'
* changes:
  DirCache: Do not create duplicate tree entries
  DirCacheEditor: Replace file-with-tree and tree-with-file
  AddCommand: Use NameConflictTreeWalk to identify file-dir changes
2016-01-01 12:58:35 -05:00
Shawn Pearce 09500165a8 Fix "remote: Counting objects: ..." formatting
Trailing whitespace is usually removed in properties files so
JGitText did not supply a space between : and the remote message.
Ensure the space exists at runtime by reading the localized string
and appending a space if it is missing.

Messages should be dynamically fetched and not held in a static
class variable, as they can be changed using thread locals.

Change-Id: If6a3707d64094253b1a5304fbfafcf195db7497a
2015-12-31 17:37:16 -08:00
Shawn Pearce c426a964ed clone: display progress messages
Also support -q/--quiet flag to disable progress.

Change-Id: I979277502c990f6dec052d095461c996ff8fe577
2015-12-31 17:37:16 -08:00
Shawn Pearce 2f8d787b5f buck: run tests
Compile each test in its own java_test() target so they can run in
parallel, reducing total time spent testing on large machines.

$ buck test --all
[-] PROCESSING BUCK FILES...FINISHED 0.3s [100%]
[-] BUILDING...FINISHED 2.9s [100%] (351/383 JOBS, 351 UPDATED, 0.0% CACHE MISS)
[-] TESTING...FINISHED 98.1s (3360 PASS/15 SKIP/0 FAIL)

Change-Id: I8d6541268315089299f933ed23d785b1b3431133
2015-12-31 17:34:03 -08:00
Shawn Pearce 34de70a5d4 buck: build standalone jgit binary
Construct the java_application JAR wrapped with the shell script
header.  This is enough to clone a repository over HTTPs:

  $ buck build :jgit_bin
  $ buck-out/gen/jgit_bin/jgit_bin clone https://...

Change-Id: I4aceb4e77b2ec9be76a32ec93d94f2dafe9acce6
2015-12-31 17:18:50 -08:00
David Ostrovsky 13502fef8f Implement Buck driven build
Today there are plenty of modern build tool systems available in the
wild (in no particular order):

* http://bazel.io
* https://pantsbuild.github.io
* http://shakebuild.com
* https://ninja-build.org
* https://buckbuild.com

The attributes, that all these build tools have in common, are:

* reliable
* correct
* very fast
* reproducible

It must not always be the other build tool, this project is currently
using. Or, quoting Gerrit Code Review maintainer here:

  "Friends, don't let friends use <the other build tool system>!"

This change is non-complete implementation of JGit build in Buck,
needed by Gerrit Code Review to replace its dependency with standlone
JGit cell. This is very useful when a developer is working on both
projects and is trying to integrate changes made in JGit in Gerrit.

The supported workflow is:

  $ cd jgit
  $ emacs <hack>
  $ cd ../gerrit
  $ buck build --config repositories.jgit=../jgit gerrit

With --config repositories.jgit=../jgit jgit cell is routed through
JGit development tree.

To build jgit, issue:

  $ buck build //:jgit
  [-] PROCESSING BUCK FILES...FINISHED 0,0s

Yes, you can't measure no-op build time, given that Buck daemon is
used.

Change-Id: I301a71b19fba35a5093d8cc64d4ba970c2877a44
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
2015-12-31 10:08:55 -08:00
Shawn Pearce 2fbd98dbe8 Merge "Fix encoding problem from curl repostory on github" 2015-12-30 23:47:28 -05:00
Shawn Pearce 80edcac06f Merge "Fix hanging fetch via SSH" 2015-12-30 23:27:09 -05:00
Eryk Szymanski 53c3bbe42f Fix encoding problem from curl repostory on github
Pushing curl repository to gerrit fails with a message:
remote: error: internal error while processing changes
java.nio.charset.IllegalCharsetNameException: 'utf8'

curl repository url: https://github.com/bagder/curl.git

To avoid this problem encodingAliases in RawParseUtils have
been extended to contain "'utf8'" (single quoted utf8) string.

Change-Id: I40f613cfdcabf0dc9455bee45116ab8d8c7dd6ee
Signed-off-by: Eryk Szymanski <eryksz@gmail.com>
2015-12-30 23:24:12 -05:00
Andrey Loskutov 02ade82b34 Make sure tests don't blindly continue if a command is "silently" failed
Make the default execute() function fail fast on first command printed
"fatal: " to output.

Introduced executeUnchecked() for few tests which wanted to test fatal
output.

Change-Id: I5b09aad9443515636811fc4d00bf8b8b9587a626
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2015-12-31 00:48:11 +01:00