Commit Graph

548 Commits

Author SHA1 Message Date
James Melvin 91132bb05e gc: Add options to preserve and prune old pack files
The new --preserve-oldpacks option moves old pack files into the
preserved subdirectory instead of deleting them after repacking.

The new --prune-preserved option prunes old pack files from the
preserved subdirectory after repacking, but before potentially
moving the latest old packfiles to this subdirectory.

These options are designed to prevent stale file handle exceptions
during git operations which can happen on users of NFS repos when
repacking is done on them. The strategy is to preserve old pack files
around until the next repack with the hopes that they will become
unreferenced by then and not cause any exceptions to running processes
when they are finally deleted (pruned).

Change-Id: If3f729f0d9ce920ee2c3e6acdde46f2068be61d2
Signed-off-by: James Melvin <jmelvin@codeaurora.org>
2017-01-19 11:00:18 +01:00
Matthias Sohn 5fee071f6a Prepare 4.7.0-SNAPSHOT builds
Change-Id: I20754d13007e6591d36aae5766f3a9a82b24e120
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-27 01:45:50 +01:00
Matthias Sohn 3857c3168f Prepare 4.6.1-SNAPSHOT builds
Change-Id: I6b05a6f6c3f92365c272e1bdaf76093ca01f2d58
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 15:51:54 +01:00
Matthias Sohn 73a4325149 JGit v4.6.0.201612231935-r
Change-Id: Iaa88fe1b195dfe6be99a7b4cb064684e75563715
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 01:42:38 +01:00
David Pursehouse a45cfee7a3 Organize imports
Change-Id: I7c545d06b1bced678c020fab9af1382bc4416b6e
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-11-14 13:25:20 -08:00
David Pursehouse 03046d0f60 CheckoutCommand: Add method to add multiple paths
The new method addPaths(List<String>) allows callers to add multiple
paths without having to iterate over several calls to addPath(String).

Change-Id: I2c3746a97ead7118fb0ed5543a2c843224719031
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-10-22 10:11:54 +09:00
Ned Twigg e49025386e Checkout: Add the ability to checkout all paths.
Change-Id: Ie1e59c566b63d0dfac231e44e7ebd7f3f08f3e9f
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-10-22 10:08:07 +09:00
David Pursehouse 4e3c5e1f13 Main: Add missing $NON-NLS tag
Change-Id: I030910b88a8f60ca174e38f0a213959f9b0a776f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-10-18 13:18:29 +09:00
David Pursehouse 7e542cbe19 Status: remove unused lineFormat member
Change-Id: I3c4d83583edb1a6e1fbee1ea496dcf93302831b3
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-10-18 01:06:30 +02:00
David Pursehouse 08649a9fd0 LfsStore: Don't invoke toString on String variable
Change-Id: I15d234e5d907d0bbb22a95cf781e915798bead30
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-10-18 01:06:29 +02:00
Christian Halstrick 293e8beacc Fix JGit CLI to follow native git's interpretation of http_proxy...
Native git (as many other tools) interprets the environment variables
http_proxy, HTTP_PROXY, ... in a specific way. "http_proxy" has to be
lowercase while "https_proxy" can be lowercase or uppercase (means:
"HTTPS_PROXY"). Lowercase has precedence. This can be looked up in
"ENVIRONMENT" section of [1]. Teach JGit CLI to behave similar.

Additionally teach JGit not to interpret the environment variables if
the java process was explicitly started with the system properties
telling JVM which proxy to use. A call like "http_proxy=proxy1 java
-Dhttp.proxyHost=proxy2 ..." should use proxy2 as proxy.

[1] https://curl.haxx.se/docs/manpage.html

Change-Id: I2ad78f209792bf8f1285cf2f8ada8ae0c28f8e5a
2016-10-14 15:14:21 +02:00
Matthias Sohn 32c89d2844 Add dependency to org.apache.commons.logging used by Apache HttpClient
Without this dependency running JGit CLI in Eclipse will hit
ClassNotFoundExceptions when HttpClient is trying to log something.

Change-Id: I2d50d9a18fac4c302de2c3a16c07f90ce3e5072e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-10-13 12:29:44 +02:00
Matthias Sohn 8f6b75f7ad Merge branch 'stable-4.5'
* stable-4.5:
  pgm: Fix misspelled key of an externalized string
  Add missing online help for Ketch server type option in CLI daemon
  Remove wrong junit dependencies in org.eclipse.jgit.pgm

Change-Id: I9cac024c0b488101b539c713b0f5ffc8c01b55bd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-10-11 00:34:55 +02:00
Matthias Sohn 15341ce577 pgm: Fix misspelled key of an externalized string
Bug: 502107
Change-Id: I76d0981c8463b63bd049f50cdc7d549fa0604b3c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-10-02 01:39:34 +02:00
Matthias Sohn d2312f2921 Add missing online help for Ketch server type option in CLI daemon
Change-Id: I19d27bbdbfdb1c7a5a688e41dfcba73a142a1afd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-10-02 01:39:33 +02:00
Matthias Sohn 9cb8192d19 Remove wrong junit dependencies in org.eclipse.jgit.pgm
Bug: 503010
Change-Id: I8fa99f53020af41eb15c1f63b6f3ba094d56bfef
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-10-02 01:15:57 +02:00
David Pursehouse d47320bd36 pgm, test: Add missing dependency for buck build
Change-Id: I26d69fb6d35c6fb120360ef143d1b1f565d4014c
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-09-22 15:18:47 +02:00
Matthias Sohn cb617e7ffc Prepare 4.5.1-SNAPSHOT builds
Change-Id: I3305e8a09a3fb06f25a316cff2bdbb551d3ade68
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-09-21 17:05:27 +02:00
Matthias Sohn e7142a3937 JGit v4.5.0.201609210915-r
Change-Id: Idc02a1a1d74f84605d764c239803f0cfbad94eb7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-09-21 15:06:18 +02:00
Matthias Sohn ab79ce91c1 Change JGit minimum execution environment to JavaSE-1.8
Bug: 500059
Change-Id: I47f3f6749a67da52029f84e002d9b155ed56d2b7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-09-20 11:32:36 +02:00
Christian Halstrick bcb5a431a5 Add built-in LFS smudge filter for local case
Adds a JGit built-in implementation of the "git lfs smudge" filter. This
filter should do the same as the one described in [1] besides that it
only supports the local case when the lfs objects are already present in
the media directory. Remote cases where download of LFS objects from an
LFS server is needed will be done in a later commit.

[1] https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-smudge.1.ronn

Change-Id: I8ff661d4edd3667ef7f86f3b4fa33e568eb4c8f4
2016-09-20 10:58:05 +02:00
Christian Halstrick 45ee55d0d9 Add built-in LFS clean filter
Adds a JGit built-in implementation of the "git lfs clean" filter. This
filter should do the same as the one described in [1]. But since this
filter is written in Java and can be called by JGit without forking new
processes it should be much faster

[1]
https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-clean.1.ronn

Change-Id: If60e387e97870245b4bd765eda6717eb84cffb1d
2016-09-20 10:11:27 +02:00
Matthias Sohn 9d4ffcc1ab Prepare 4.6.0-SNAPSHOT builds
Change-Id: Id2eafc331ee32c332c2a9b867b05c260beb0d10f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-09-19 17:06:42 +02:00
Ned Twigg b67df51203 CLI: implement option -d for deleting tags
Change-Id: I438456b76aefd361384729686271288186d3be3b
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-08-31 00:16:23 +02:00
Ned Twigg 34673f0536 Added a clean command.
Change-Id: I05d5392789b5b64e6ee44f678556cf25dc30d7ba
Signed-off-by: Ned Twigg <ned.twigg@diffplug.com>
2016-08-30 22:21:05 +02:00
Stefan Beller 609de52600 push: Do not use push options unless requested
Unless the user passed --push-option, the client does not intend to
pass push options to the server.

Without this change, all pushes to servers without push option support
fail.

Not enabling the feature (instead of enabling it and sending an empty
list of options) in this case is more intuitive and matches the
behavior of C git push's --push-option parameter better.

Bug: 500149
Change-Id: Ia4f13840cc54d8ba54e99b1432108f1c43022c53
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-08-24 15:24:23 -07:00
David Pursehouse bb9988c236 LfsProtocolServlet: Pass request and path to getLargeFileRepository
Passing the request and path to the method will allow implementations
to have more control over determination of the backend, for example:

- return different backends for different requests
- accept or refuse requests based on request characteristics
- etc

Change-Id: I1ec6ec54c91a5f0601b620ed18846eb4a3f46783
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-07-27 15:54:03 +09:00
David Pursehouse 4528d5f56d Ignore 'The value of exception parameter is not used' warning
Change-Id: I50407e4a33e35b718ca40503fdd436f1f9f70fba
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2016-07-26 10:16:49 +09:00
Dan Wang 7f9fb80002 Push implementation of option strings
Example usage:
$ ./jgit push \
  --push-option "Reviewer=j.doe@example.org" \
  --push-option "<arbitrary string>" \
  origin HEAD:refs/for/master
Stefan Beller has also made an equivalent change to CGit:
http://thread.gmane.org/gmane.comp.version-control.git/299872

Change-Id: I6797e50681054dce3bd179e80b731aef5e200d77
Signed-off-by: Dan Wang <dwwang@google.com>
2016-07-22 15:40:54 -07:00
Matthias Sohn 75b3301939 Prepare 4.5.0-SNAPSHOT builds
Change-Id: I572fe9fea0e5ca0bec4648c916ae95a5b1ccf125
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-06-01 23:04:07 +02:00
Matthias Sohn 4c236ff4bb Replace use of deprecated method Repository.getRef()
Change-Id: Iecf2b8deafc4991cc3333702fb9fa0638be7b914
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-05-31 00:11:11 +02:00
Stefan Beller 063548b9af RepoCommand: remove --record-remote-branches
This function only works in bare mode, and the command line works in
non-bare mode, so there's no point in pretending to support it here.

Change-Id: Ia017aded93d072bdb9947ec08c424256c3abe438
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-05-25 17:11:21 -07:00
Rüdiger Herrmann 8ff08994a4 JGit CLI: allow to call git init with specific directory
With this change it is possible to initialize repositories with
'git init /path/to/new/repo'

Change-Id: Ia9b288b4fb34887e8f5e2f3f270e521c44d41b6a
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-05-22 22:11:03 +02:00
Rüdiger Herrmann b3f4abdda1 Redirect all Show output to outs
The Show command wrote all diffs to System.out instead of 'outs.

Bug: 494143
Change-Id: Ibc78e9a058b7a32d536e03e294c1d933b6c26ba7
Signed-off-by: Rüdiger Herrmann <ruediger.herrmann@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-05-21 01:52:28 +02:00
Matthias Sohn ccf8363693 Prepare 4.4.0-SNAPSHOT builds
Change-Id: If3162f4cc4ae6319b9f1e3293549485b039cfe7f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 17:59:59 +02:00
Matthias Sohn f2ee537d55 Prepare 4.3.1-SNAPSHOT versions
Change-Id: I52c98ba8fb3a303269a1f9380af114b6dd8c5009
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 00:56:43 +02:00
Matthias Sohn 4f0daa3bb2 JGit v4.3.0.201604071810-r
Change-Id: I902cdf1ff92ce8c6e9d80c4965d8d5bd8b9ac6c1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-08 00:10:10 +02:00
Matthias Sohn 57e6274bcf JGit v4.3.0.201604071045-r
Change-Id: Iafab78d6be34d31a13f979b7be67611135c0f8bd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-07 16:41:24 +02:00
Matthias Sohn 46bef2bcb6 Prepare 4.3-SNAPSHOT builds
Change-Id: Ib831f8870938113bd5338763f90a07d5c108b1de
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-04-05 23:10:06 +02:00
Matthias Sohn bf32c9102f JGit v4.3.0.201603230630-rc1
Change-Id: I10835e5aa3618e5033424595942cc1649152cb24
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-23 11:27:04 +01:00
Matthias Sohn 846ef78a02 Fix RebuildRefTree trying to add HEAD twice to RefTree
14dfa70520 fixed the problem that HEAD wasn't added to the reftree when
rebuilding the reftree in an empty repository where HEAD isn't yet
resolvable. Since non-resolvable refs are filtered out by
RefDatabase.getRefs(ALL) we have to add HEAD to the reftree explicitly
in this special case.

This fix resulted in another bug: rebuilding the reftree in a repository
which has a resolvable HEAD failed with a DirCacheNameConflictException
in RefTree.apply(). If HEAD is resolvable RefDatabase.getRefs(ALL) does
not filter out HEAD. This results in two identical CREATE commands for
HEAD which RefTree.apply() refuses to execute.

Fix this by no longer creating a duplicate CREATE command for HEAD.

See: I46cbc2611b9ae683ef7319dc46af277925dfaee5
Change-Id: I58dd6bcdef88820aa7de29761d43e2edfa18fcbe
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-05 01:03:34 +01:00
Matthias Sohn be8c525fa0 Fix help text for option --enable of command debug-rebuild-ref-tree
This fixes a MissingResourceException thrown when executing
jgit debug-rebuild-ref-tree --help

Change-Id: I637ea55084a913f5105ebf4cf2baef8b81877938
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-03-01 23:18:34 +01:00
Matthias Sohn f496177a37 Support Amazon S3 based storage for LFS
Add a storage implementation storing large objects in Amazon S3.
The AmazonS3Repository pre-signs download and upload requests.

AWS access and secret key are expected to be in the
$HOME/.aws/credentials file in the following format:

[default]
  accessKey = ...
  secretKey = ...

Use AWS version 4 request signing [1] because it is more secure and
supported by all regions. The version 3 signing is not supported in
newer regions.

In follow up changes we should:

- implement getVerifyAction() and do actual verification. Subclasses of
S3Repository can implement caching for object meta data (size) in order
to avoid extra roundtrips to S3. Verification should ensure that meta
data store and content of S3 storage are in sync

- HEAD request used in S3Repository.getSize() seems to always return
Content-length 0 in contrast to the documentation [2]. So getSize() does
detect if the object exists in S3 or not but in case the object exists
it always returns size 0

[1] http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
[2] https://forums.aws.amazon.com/thread.jspa?threadID=223616

Change-Id: Ic47f094928a259e5264c92b3aacf6d90210907a8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
2016-02-17 11:36:58 -05:00
Matthias Sohn 514b11ddcc Merge branch 'stable-4.2'
* stable-4.2:
  Don't use deprecated LockFile constructor
  Fix warnings about unchecked conversion of MergeResult
  MockServletConfig: Fix warning about unchecked conversion of Enumeration
  HugeFileTest: Make Git a class member and open in try-with-resource
  Suppress "unchecked cast" warnings related to UploadPackFactory.DISABLED
  DiffAlgorithms: Fix warnings about variable hiding
  DirCacheBasicTest: Open ObjectInserter.Formatter in try-with-resource
  DirCacheBuilderIteratorTest: Open TreeWalk in try-with-resource
  DirCacheCGitCompatabilityTest: Open TreeWalk in try-with-resource
  DirCacheCheckoutMaliciousPathTest: Open Git and RevWalk in t-w-r
  DirCacheIteratorTest: Open TreeWalk instances in try-with-resource
  ForPathTest: Open TreeWalk in try-with-resource
  GitConstructionTest: Open Git instance in try-with-resource
  IndexDiffTest: Open Git instances in try-with-resources
  ManifestParserTest: Don't use deprecated StringBufferInputStream
  InMemoryRepository: Remove unused RevWalk from batch method signature
  IndexModificationTimesTest: Open Git instances in try-with-resource
  InterIndexDiffFilterTest: Open TreeWalk in try-with-resource
  LockFileTest: Open Git instance in try-with-resource
  JGit v4.1.2.201602141800-r
  MergeCommandTest: Use JUnit's assume to check preconditions
  MergeCommandTest: Open Git instances in try-with-resource

Change-Id: Ie5dba6b9132a29e86958a04fa2b76465bcd2c6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-15 23:28:18 +01:00
David Pursehouse e96cb22a43 Don't use deprecated LockFile constructor
Change-Id: Ibc3e2f3372e1a65732dd6d3c71cec53fb1aa15e2
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 18:44:24 +09:00
David Pursehouse 3a437dd655 DiffAlgorithms: Fix warnings about variable hiding
Local variables/parameters named 'db' and 'cmp' were hiding class
member variables of the same name.

Change-Id: I98b770587aaf73744a93e6a3ee33d131a9fa91e9
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-15 17:29:20 +09:00
Matthias Sohn 28e2fed761 Merge branch 'stable-4.2'
* stable-4.2:
  RepoProject: Fix warnings about variable hiding
  RepoTest: Open Git in try-with-resources
  RepositoryResolveTest: Open Git in try-with-resource
  RepositoryTestCase: Open autocloseable types in try-with-resource
  ResetCommandTest: Use Git member in testHardResetAfterSquashMerge
  ResolveMergerTest: Open Git in try-with-resource
  RevCommitListTest: Open Git and RevWalk in try-with-resource
  RevCommitParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevObjectTest: Open RevWalk in try-with-resource
  RevTagParseTest: Open ObjectInserter.Formatter in try-with-resource
  RevertCommandTest: Open Git in try-with-resource
  SquashMessageFormatterTest: Open git in try-with-resource
  StatusCommandTest: Open Git in try-with-resource
  SubmoduleAddTest: Open Git in try-with-resource
  SymlinksTest: Open git and TreeWalk in try-with-resource
  T0003_BasicTest: Open autocloseable types in try-with-resource
  TextHashFunctions: Fix warnings about variable hiding
  TreeFilterTest: Open TreeWalk in try-with-resource
  TreeWalkJava7Test: Open TreeWalk in try-with-resource
  Fix diff for added and removed submodule

Change-Id: If3ecc63f6dfac55474d3c1dd2f4105371f3d24fb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-10 00:51:01 +01:00
David Pursehouse 84ae3f4991 TextHashFunctions: Fix warnings about variable hiding
The local variables/arguments named 'db' were hiding the member
variable of the same name in the superclass TextBuiltin.

Change-Id: Ic49dcb7cc2a1b3fb46ad78450042c8c8fb464a80
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
2016-02-05 11:46:45 +09:00
Matthias Sohn 3bae524f6f Support LFS protocol and a file system based LFS storage
Implement LfsProtocolServlet handling the "Git LFS v1 Batch API"
protocol [1]. Add a simple file system based LFS content store and the
debug-lfs-store command to simplify testing.

Introduce a LargeFileRepository interface to enable additional storage
implementation while reusing the same protocol implementation.

At the client side we have to configure the lfs.url, specify that
we use the batch API and we don't use authentication:

  [lfs]
	  url = http://host:port/lfs
	  batch = true
  [lfs "http://host:port/lfs"]
	  access = none

the git-lfs client appends the "objects/batch" to the lfs.url.

Hard code an Authorization header in the FileLfsRepository.getAction
because then git-lfs client will skip asking for credentials. It will
just forward the Authorization header from the response to the
download/upload request.

The FileLfsServlet supports file content storage for "Large File
Storage" (LFS) server as defined by the Github LFS API [2].

- upload and download of large files is probably network bound hence use
  an asynchronous servlet for good scalability
- simple object storage in file system with 2 level fan-out
- use LockFile to protect writing large objects against multiple
  concurrent uploads of the same object
- to prevent corrupt uploads the uploaded file is rejected if its hash
  doesn't match id given in URL

The debug-lfs-store command is used to run the LfsProtocolServlet and,
optionally, the FileLfsServlet which makes it easier to setup a
local test server.

[1]
https://github.com/github/git-lfs/blob/master/docs/api/http-v1-batch.md
[2] https://github.com/github/git-lfs/tree/master/docs/api

Bug: 472961
Change-Id: I7378da5575159d2195138d799704880c5c82d5f3
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
2016-02-04 17:49:43 +01:00
Matthias Sohn b0facdc113 Merge branch 'stable-4.2'
* stable-4.2:
  DirCacheCheckoutTest: Open Git and TreeWalk in try-with-resource
  CommitCommand: Remove declaration of unthrown exception
  Branch: Fix variable hiding warning
  ApplyCommandTest: Open Git in try-with-resource
  PackFileTest: Open ObjectInserter.Formatter in try-with-resource

Change-Id: I8484b10fad5a4c35fcfaedc1cdf8ccf97471618e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-02-03 10:30:52 +01:00