Commit Graph

3254 Commits

Author SHA1 Message Date
Shawn Pearce 68d8a51d08 Introduce specific WantNotValidException for servers
Capture the internal "want X not valid" state as a specific subclass
of PackProtocolException, allowing this to be more easily identified
in server stack traces and wrapper application code.

Change-Id: I4b1adb7497f396432da420b0f600ad25a261f912
2016-02-16 16:41:51 -08:00
Shawn Pearce e7a6e85b95 smart HTTP server: Pass along "want X not valid" to client
If the client sends a SHA-1 that the server does not recognize echo
this back to the client with an explicit error message instead of
the generic "internal server error".

This was always the intent of the implementation but it was being
dropped on smart HTTP due to the UploadPackServlet catching the
PackProtocolException, discarding the buffered message UploadPack
meant to send, and sending along a generic message instead.

Change-Id: I8d96b064ec655aef64ac2ef3e01853625af32cd1
2016-02-16 15:09:05 -08:00
Matthias Sohn ff5c756c79 Merge branch 'stable-4.2'
* stable-4.2:
  Revert "Fix warnings about unchecked conversion of MergeResult"

Change-Id: I31c2e0679ad4cff77190858ac9a570d04841c386
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-16 01:06:11 +01:00
Shawn Pearce 8c1d44e436 Merge "Revert "Fix warnings about unchecked conversion of MergeResult"" into stable-4.2 2016-02-15 19:00:50 -05:00
Shawn Pearce 591e39bf9d Revert "Fix warnings about unchecked conversion of MergeResult"
This reverts commit 979fa19110.
Breaks API.

Change-Id: I54af657898d49f64d6906fe3edfb6b08e996d901
2016-02-15 18:45:55 -05:00
Matthias Sohn 514b11ddcc Merge branch 'stable-4.2'
* stable-4.2:
  Don't use deprecated LockFile constructor
  Fix warnings about unchecked conversion of MergeResult
  MockServletConfig: Fix warning about unchecked conversion of Enumeration
  HugeFileTest: Make Git a class member and open in try-with-resource
  Suppress "unchecked cast" warnings related to UploadPackFactory.DISABLED
  DiffAlgorithms: Fix warnings about variable hiding
  DirCacheBasicTest: Open ObjectInserter.Formatter in try-with-resource
  DirCacheBuilderIteratorTest: Open TreeWalk in try-with-resource
  DirCacheCGitCompatabilityTest: Open TreeWalk in try-with-resource
  DirCacheCheckoutMaliciousPathTest: Open Git and RevWalk in t-w-r
  DirCacheIteratorTest: Open TreeWalk instances in try-with-resource
  ForPathTest: Open TreeWalk in try-with-resource
  GitConstructionTest: Open Git instance in try-with-resource
  IndexDiffTest: Open Git instances in try-with-resources
  ManifestParserTest: Don't use deprecated StringBufferInputStream
  InMemoryRepository: Remove unused RevWalk from batch method signature
  IndexModificationTimesTest: Open Git instances in try-with-resource
  InterIndexDiffFilterTest: Open TreeWalk in try-with-resource
  LockFileTest: Open Git instance in try-with-resource
  JGit v4.1.2.201602141800-r
  MergeCommandTest: Use JUnit's assume to check preconditions
  MergeCommandTest: Open Git instances in try-with-resource

Change-Id: Ie5dba6b9132a29e86958a04fa2b76465bcd2c6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-15 23:28:18 +01:00
Shawn Pearce 621b0740b3 Merge "Allow deletion of HEAD ref if the repository is bare." 2016-02-15 13:02:05 -05:00
David Pursehouse e96cb22a43 Don't use deprecated LockFile constructor
Change-Id: Ibc3e2f3372e1a65732dd6d3c71cec53fb1aa15e2
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 18:44:24 +09:00
David Pursehouse 979fa19110 Fix warnings about unchecked conversion of MergeResult
Change-Id: I1490b2209fa7b39676849c624adbc262a672f6df
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 18:22:53 +09:00
David Pursehouse f04bd20fc6 InMemoryRepository: Remove unused RevWalk from batch method signature
The RevWalk given in the arguments is not used. According to the
comment at the top of the method, a new RevWalk is intentionally
used in the implementation.

Remove the unused argument.

Change-Id: Iec81a1341d5bf377801475845b96a465753096ef
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 14:57:39 +09:00
Ivan Motsch 975aa88685 Add Attribute Macro Expansion
Attributes MacroExpander implements macros used in git attributes. This
is implemented inside the TreeWalk using a lazy created MacroExpander.
In addition, the macro expander caches the global and info attributes
node in order to provide fast merge of attributes.

Change-Id: I2e69c9fc84e9d7fb8df0a05817d688fc456d8f00
Signed-off-by: Ivan Motsch <ivan.motsch@bsiag.com>
2016-02-11 17:08:49 +01:00
Matthias Sohn 28e2fed761 Merge branch 'stable-4.2'
* stable-4.2:
  RepoProject: Fix warnings about variable hiding
  RepoTest: Open Git in try-with-resources
  RepositoryResolveTest: Open Git in try-with-resource
  RepositoryTestCase: Open autocloseable types in try-with-resource
  ResetCommandTest: Use Git member in testHardResetAfterSquashMerge
  ResolveMergerTest: Open Git in try-with-resource
  RevCommitListTest: Open Git and RevWalk in try-with-resource
  RevCommitParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevObjectTest: Open RevWalk in try-with-resource
  RevTagParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevertCommandTest: Open Git in try-with-resource
  SquashMessageFormatterTest: Open git in try-with-resource
  StatusCommandTest: Open Git in try-with-resource
  SubmoduleAddTest: Open Git in try-with-resource
  SymlinksTest: Open git and TreeWalk in try-with-resource
  T0003_BasicTest: Open autocloseable types in try-with-resource
  TextHashFunctions: Fix warnings about variable hiding
  TreeFilterTest: Open TreeWalk in try-with-resource
  TreeWalkJava7Test: Open TreeWalk in try-with-resource
  Fix diff for added and removed submodule

Change-Id: If3ecc63f6dfac55474d3c1dd2f4105371f3d24fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-10 00:51:01 +01:00
Matthias Sohn 26012958a3 Merge branch 'stable-4.1' into stable-4.2
* stable-4.1:
  Fix diff for added and removed submodule

Change-Id: I37dd71ed19b06e9bbcffe37370081ab875c6d8d4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-10 00:45:18 +01:00
David Pursehouse 4e5255c59a RepoProject: Fix warnings about variable hiding
Method parameter names were hiding class members of the same
name.

Change-Id: I182f2715894ac4259b09a371cb4e0eb24f52518a
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-05 18:22:58 +09:00
Saša Živkov a0e1374e22 Allow to reuse disableSslVerify method, move it to HttpSupport
The disableSslVerify method will be used in the follow up change.

Change-Id: Ie00b5e14244a9a036cbdef94768007f1c25aa8d3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-04 17:49:43 +01:00
Matthias Sohn 3bae524f6f Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.

Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.

At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:

  [lfs]
	  url = http://host:port/lfs
	  batch = true
  [lfs "http://host:port/lfs"]
	  access = none

the git-lfs client appends the "objects/batch" to the lfs.url.

Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.

The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].

- upload and download of large files is probably network bound hence use
  an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
  concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
  doesn't match id given in URL

The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.

[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api

Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
2016-02-04 17:49:43 +01:00
Matthias Sohn 536db18cc6 Implement SHA-256 abstraction
The Large File Storage extension specified by GitHub [1] uses SHA-256 to
compute the ID of large files stored by the extension. Hence implement a
SHA-256 abstraction similar to the SHA-1 abstraction used by JGit.

[1] https://git-lfs.github.com/

Bug: 470333
Change-Id: I3a95954543c8570d73929e55f4a884b55dbf1b7a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-04 17:49:42 +01:00
Hugo Arès df904a4227 Fix diff for added and removed submodule
Since If13f7b406, submodule difference are shown as a hunk. The issue
was that added and removed submodule were considered as Edit.REPLACE
instead of Edit.INSERT and Edit.DELETE in the DiffFormatter result.

Change-Id: I4330c2aa3f10e29d7d6b0b2e5286e59293a06239
Signed-off-by: Hugo Arès <hugo.ares@ericsson.com>
2016-02-04 10:27:50 -05:00
Matthias Sohn 2de33d7678 Remove unused API problem filters
Change-Id: I8b1c99b108b4cea6356e0a62e6bfc0731df44cf5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-03 10:39:08 +01:00
Matthias Sohn b0facdc113 Merge branch 'stable-4.2'
* stable-4.2:
  DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
  CommitCommand: Remove declaration of unthrown exception
  Branch: Fix variable hiding warning
  ApplyCommandTest: Open Git in try-with-resource
  PackFileTest: Open ObjectInserter.Formatter in try-with-resource

Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-03 10:30:52 +01:00
David Pursehouse 521f513d6d CommitCommand: Remove declaration of unthrown exception
Change-Id: I1795b1e510ab170f062ee6c5988d7a7567abe260
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-02 18:27:47 +09:00
Matthias Sohn 9400d0cb17 Merge "Prevent ChainingCredentialsProvider from adding delegated providers twice" 2016-01-28 18:51:14 -05:00
Mike Williams 48ad6842fc Allow deletion of HEAD ref if the repository is bare.
Change-Id: I2281d818c9f76019e7e053e89b20214f2d663957
Signed-off-by: Mike Williams <miwilliams@google.com>
2016-01-28 11:43:59 -05:00
Yuxuan 'fishy' Wang 7960fa8735 [gitrepo] Support revision in remote tag.
Repo manifest file allows revision attribute in <remote> tag. This change
teaches JGit to read that information.

Change-Id: I1c878a2505b9d09fa09fbd404a119b71f2fb8fdb
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
2016-01-27 16:01:06 -08:00
Rüdiger Herrmann bd67084d2b Prevent ChainingCredentialsProvider from adding delegated providers twice
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Change-Id: Icc0907af4fe26877d066db3c813dba37213a3ccc
2016-01-26 18:55:39 +01:00
Thomas Wolf 9441d61e37 Fix a @since tag
Ketch didn't make it into 4.2

Change-Id: Iab3ddfb926a2f8d31622e212c845c825369b09d5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2016-01-23 23:51:14 +01:00
Matthias Sohn fc4d117511 Prepare 4.2.1-SNAPSHOT builds
Change-Id: Ic9eac53b10ac97b2038b334d388ae3d21393f993
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-22 01:48:52 +01:00
Matthias Sohn 20c2079f70 JGit v4.2.0.201601211800-r
Change-Id: I89f33b0da81ff499315baa5311f95f31724fa510
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 23:47:12 +01:00
Terry Parker f321741cff Merge "Remove unused variable from DfsBlockCache" 2016-01-21 16:21:15 -05:00
Terry Parker 4d178be021 Remove unused variable from DfsBlockCache
Change-Id: I8ea167e44e9ebcac7ca08920e806d8bcb8188bf0
Signed-off-by: Terry Parker <tparker@google.com>
2016-01-21 13:02:39 -08:00
Matthias Sohn b30917ce21 Prepare 4.3.0-SNAPSHOT builds
Change-Id: Idcf0479529693b023042becd96698f9afd344bd4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 21:49:41 +01:00
Laurent Delaigue 4409751516 Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.

This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.

Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 17:08:04 +01:00
Chris Gavin 82f47ace3d Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.

Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.

Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7
Signed-off-by: Chris Gavin <chris@chrisgavin.me>
2016-01-21 17:07:57 +01:00
Laurent Delaigue 1e1fb9fc59 Add progress monitor to Merger
Monitoring progress of merges can be useful for users for large
repositories or complex merge processes that take some time.

This enables setting a monitor. Existing merge implementations in jgit
do not yet report progress if a monitor is set. This will be added in a
later change.

Change-Id: I17b978b3fc91750dd88649638b90a46820a0877c
Signed-off-by: Laurent Delaigue <laurent.delaigue@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-21 11:37:26 +01:00
Shawn Pearce 08aac5904e Merge "Fix TransportException when reading bundle" 2016-01-21 01:15:12 -05:00
Chris Gavin d2bf41e7dd Fix TransportException when reading bundle
When reading a bundle file, commit messages who's oneline format is
longer than 982 characters caused JGit to treat subsequent text in
the commit as a SHA, then throw a TransportException because it's
not a valid SHA.

Now the readLine method will read all the way to the end of the
line, not just the first 1024 characters of it.

Change-Id: If15b491aa9a1e4fd9b8bbed2dd9e6be47a64ccb7
Signed-off-by: Chris Gavin <chris@chrisgavin.me>
2016-01-21 01:14:56 -05:00
Shawn Pearce 8fe9a8baf5 Merge changes from topic 'reftree'
* changes:
  debug-rebuild-ref-tree: Copy HEAD into RefTree
  debug-rebuild-ref-tree: Add --enable flag to turn the database on
  RefTreeDatabase: Allow ORIG_HEAD, etc. on non-bare repositories
  RefTreeDatabase: Expose bootstrap refs in getAdditionalRefs
2016-01-21 00:32:17 -05:00
Shawn Pearce 8d0551dd64 Merge changes from topic 'ketch'
* changes:
  daemon: Add --ketch=LEADER flag
  Ketch: Intercept push and route it through Ketch
  Ketch: Simple map of running leader instances
  Ketch: Basic replication system
2016-01-20 23:15:09 -05:00
Shawn Pearce 7b6122908b Merge "TreeWalk: Remove CorruptObjectException from addTree(AbstractTreeIterator)" 2016-01-20 11:03:47 -05:00
Shawn Pearce 2262a794b4 TreeWalk: Remove CorruptObjectException from addTree(AbstractTreeIterator)
This form of addTree() does not parse any objects and cannot throw the
declared checked exception.  Callers are being forced to try-catch
CorruptObjectException that cannot occur when the iterator instance
has already been constructed.

Change-Id: Id338035302903bab81569d1576eab063eee0885a
2016-01-20 07:41:08 -08:00
Christian Halstrick 7182cb2a26 Fix ResetCommand to return the resulting ref
ResetCommand was not returning the updated ref as a result of the call()
method. Since the ResetCommand is always updating the same ref (HEAD)
this should always be the HEAD ref.

Bug: 440750
Change-Id: I7974975c3ab05e68c208384e69cf0692ded6e8db
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-20 11:27:31 +01:00
Christian Halstrick da43d8d798 Add option to allow empty commits to CommitCommand
CommitCommand should allow to specify whether empty commits (commits
having the same tree as the sole predecessor commit) are allowed or not.
Similar to native git's "--allow-empty" flag.

The defaults differ between JGit and native git even after this change.
When not specifying paths then by default JGit allows to create empty
commits while native git does not. It would be API breaking to change
this now.

Bug: 460301
Change-Id: I88feb0c3ffb2c686b1d0594e669729b065cda4cb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-01-20 11:14:19 +01:00
Shawn Pearce da5ef91ad0 daemon: Add --ketch=LEADER flag
Experimental flag to turn on the KetchLeader within this daemon JVM.
This is a manually elected leader process, set from the command line.
Remote followers for each repository are configured per-repository
using remote sections with ketch-type = FULL. For example:

Manually elected leader's $GIT_DIR/config:

  [ketch]
    name = A

  [remote "A"]
    ketch-type = FULL

  [remote "B"]
    url = git://127.0.0.1:9421/sample.git
    ketch-type = FULL

  [remote "C"]
    url = git://127.0.0.1:9422/sample.git
    ketch-type = FULL

Replica B and C daemons:

  git daemon \
    --export-all \
    --enable=receive-pack \
    --listen=127.0.0.1 --port=9421 \
    --base-path=$HOME/ketch_test/follower_one \
    $HOME/ketch_test/follower_one &

  git daemon \
    --export-all \
    --enable=receive-pack \
    --listen=127.0.0.1 --port=9422 \
    --base-path=$HOME/ketch_test/follower_two \
    $HOME/ketch_test/follower_two &

Change-Id: I165f85970a77e16b5263115290d685d8a00566f5
2016-01-19 23:05:28 -08:00
Shawn Pearce 1f9d205043 Ketch: Intercept push and route it through Ketch
Capture commands and pass to the in-process KetchLeader, allowing
it to replicate to followers.

Change-Id: I25dfeb2a93821af65354337f391480a72bae2210
2016-01-19 23:04:41 -08:00
Shawn Pearce b718e3d304 Ketch: Simple map of running leader instances
Supports use in basic daemons where RepositoryCache is also effective.

Change-Id: Ia736b32c2c710b5a802233badc0d30818de763cd
2016-01-19 23:03:35 -08:00
Shawn Pearce 9b33f4aeeb Ketch: Basic replication system
Git Ketch is a multi-master Git repository management system.  Writes
are successful only if a majority of participant servers agree.  Acked
writes are durable against server failures as a majority of the
participants store all required objects.

Git Ketch is modeled on the Raft Consensus Algorithm[1].  A ketch
sailing vessel is faster and more nimble than a raft.  It can also
carry more source codes.

Git Ketch front-loads replication costs, which vaguely resembles a
ketch sailing vessel's distinguishing feature of the main mast on the
front of the ship.

[1] https://raft.github.io/
Change-Id: Ib378dab068961fc7de624cd96030266660b64fb4
2016-01-19 23:03:32 -08:00
David Pursehouse aca07fac46 Merge branch 'stable-4.2'
* stable-4.2:
  CheckoutCommandTest: Create Git instances in try-with-resource
  BranchCommandTest: Create Git instances in try-with-resource
  CheckoutTest: Create Git instances in try-with-resource
  BranchTest: Create Git instances in try-with-resource
  URIishTest: Use @Test annotation's `expected` argument
  Suppress "The allocated object is never used" warning in tests
  Add $NON-NLS to suppress "Non-externalized string literal" warnings
  Don't use deprecated constructors of CmdLineException
  Prepare 4.2.0-SNAPSHOT builds
  Remove org.eclipse.jgit.updatesite project from tools/version.sh
  RevParse: Remove superfluous semicolon
  RefUpdateTest: Use try-with-resource for auto-closable types
  RefUpdateTest: Add null check to prevent potential NPE
  CommitCommand: Remove redundant null check
  JGit v4.2.0.201512141825-rc1

Change-Id: I2179859289b2f2e3d0b7c6d02ef7e7890c467f7b
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-01-20 09:45:48 +09:00
Dave Borowitz 2ccea7f05a ChangeIdUtil: Don't throw IOException
This could have only happened during the getBytes call. Instead, use
Constants.encode, which is a non-throwing implementation.

This change is binary compatible with existing code compiled against
older versions of JGit, although it might break compilation of
previously compiling code due to dead catch blocks.

Change-Id: I191fec5cac718657407230de141440e86d0151fb
2016-01-19 17:59:19 -05:00
Shawn Pearce a693d7b33c Merge "Transport: Implement AutoCloseable" 2016-01-19 17:36:39 -05:00
Shawn Pearce 6b662af76e Transport: Implement AutoCloseable
After creating a Transport instance callers should always call
its close() method. Use AutoCloseable to document this idiom
and allow use of try-with-resources.

Change-Id: I0c6ff3e39ebecdd7a028dbcae1856a818937b186
2016-01-19 14:27:11 -08:00