Commit Graph

4870 Commits

Author SHA1 Message Date
Matthias Sohn a4feeb0194 Don't rely on default locale when using toUpperCase() and toLowerCase()
Otherwise these methods may produce unexpected results if used for
strings that are intended to be interpreted locale independently.
Examples are programming language identifiers, protocol keys, and HTML
tags. For instance, "TITLE".toLowerCase() in a Turkish locale returns
"t\u0131tle", where '\u0131' is the LATIN SMALL LETTER DOTLESS I
character.

See
https://docs.oracle.com/javase/8/docs/api/java/lang/String.html#toLowerCase--
http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html

Bug: 511238
Change-Id: Id8d8f37d84d62239c918b81f8d883ed798d87656
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-28 15:06:15 +01:00
David Pursehouse 2eb1bebd60 Repository: Include repository name when logging corrupt use count
Logging the repository name makes it easier to track down what is
incorrectly closing a repository.

Change-Id: I42a8bdf766c0e67f100adbf76d9616584e367ac2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-27 15:59:09 +09:00
Thirumala Reddy Mutchukota c9f55032a2 Record the estimated size of the pack files.
The Compacter and Garbage Collector will record the estimated size of
the newly going to be created compact, gc or garbage packs. This
information can be used by the clients to better make a call on how to
actually store the pack based on the approximated expected size.

Added a new protected method DfsObjDatabase.newPack(PackSource
packSource, long estimatedPackSize), so that the clients can override
this method to make use of the estimatedPackSize while creating a new
PackDescription object. The default implementation of this method is
equivalent to
newPack(packSource).setEstimatedPackSize(estimatedPackSize). I didn't
make it abstract because that would force all the existing sub classes
of DfsObjDatabase to implement this method. Due to this default
implementation, the estimatedPackSize is added to DfsPackDescription
using a setter instead of a constructor parameter (even though
constructor parameter would be a better choice as this value is set only
during the object creation).

Change-Id: Iade1122633ea774c2e842178a6a6cbb4a57b598b
Signed-off-by: Thirumala Reddy Mutchukota <thirumala@google.com>
2017-01-26 12:01:59 -08:00
David Pursehouse 635bff2d50 Merge branch 'stable-4.6'
* stable-4.6:
  Update Orbit to S20170120205402 and com.jcraft.jsch to 0.1.54
  Fix preparation of 4.6.1-SNAPSHOT builds

Change-Id: Ifd1f81cb199a0b5bd35e8652cac116e377136b2d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-26 11:01:32 +09:00
Lars Vogel 71edc8bd6f Fixes Javadoc error in org.eclipse.jgit created with I59539ac
Adds the param information to the private method. These are generated
via tooltip to resolve the compile errors.

Bug: 511043
Change-Id: I9ba551978eab750326d1a067b296e3ae93925871
Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
2017-01-25 12:40:59 -04:00
Jonathan Nieder 061d24f6d5 Remove @since tags from internal packages
These packages don't use @since tags because they are not part of the
stable public API.  Some @since tags snuck in, though.  Remove them to
make the convention easier to find for new contributors and the
expectations clearer for users.

Change-Id: I6c17d3cfc93657f1b33cf5c5708f2b1c712b0d31
2017-01-24 14:41:24 -08:00
David Turner 8bec98cec0 gc: loosen unreferenced objects
An unreferenced object might appear in a pack.  This could only happen
because it was previously referenced, and then later that reference
was removed.  When we gc, we copy the referenced objects into a new
pack, and delete the old pack.  This would remove the unreferenced
object.  Now we first create a loose object from any unreferenced
object in the doomed pack.  This kicks off the two-week grace period
for that object, after which it will be collected if it's not
referenced.

This matches the behavior of regular git.

Change-Id: I59539aca1d0d83622c41aa9bfbdd72fa868ee9fb
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Jonathan Nieder <jrn@google.com>
2017-01-24 14:22:45 -08:00
Matthias Sohn efbecb9764 Update Orbit to S20170120205402 and com.jcraft.jsch to 0.1.54
CQ: 12553
Change-Id: Ie72b48d00f363e3dc05b9733018a6ddca32c3e6e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-23 20:29:43 +01:00
Matthias Sohn fd874b4024 Fix preparation of 4.6.1-SNAPSHOT builds
Change-Id: Ib241c4f45192672712526599898c0bd0bd356436
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-23 20:29:42 +01:00
Matthias Sohn d3c4c0622f [infer] Mark ManifestParse.getFilteredProjects non-null
Change-Id: I05653df7a0337443d2c8e53f47f4e95ec9ca1a9c
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-23 19:55:20 +01:00
Matthias Sohn b686c8468c [infer] Fix potential NPE in DiffFormatter
Change-Id: Ia33e2af9ce3393d9173ca0dc7efefd86c965d8c8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-23 19:55:18 +01:00
Matthias Sohn 423a583fcc [infer] Fix potential NPE in CloneCommand
Change-Id: Ie7eeba3ae719ff207c7535d535a9e0bd6c9e99e6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-23 19:55:12 +01:00
David Pursehouse 9d1474431f Implement Bazel build for http-apache, lfs, lfs-server
Test plan:

  $ bazel build all

  $ unzip -t bazel-genfiles/all.zip
    Archive:  bazel-genfiles/all.zip
      testing: libhttp-apache.jar       OK
      testing: libjgit-archive.jar      OK
      testing: libjgit-lfs-server.jar   OK
      testing: libjgit-lfs.jar          OK
      testing: libjgit-servlet.jar      OK
      testing: libjgit.jar              OK
      testing: libjunit.jar             OK
    No errors detected in compressed data of bazel-genfiles/all.zip.

Change-Id: I9e6c60898ccc6d2a4557ec7544c297442a9702b4
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-22 22:34:12 +01:00
David Pursehouse dd5e500a57 Format Bazel files with buildifier
Change-Id: I934114315d2c7cab917f1011b8e55c52367d429f
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-22 22:34:11 +01:00
Shawn Pearce 131b09106f Change StreamGobbler to Runnable to avoid unused Future
It can be considered a programming error to create a Future<T>
but do nothing with that object. There is an async computation
happening and without holding and checking the Future for done
or exception the caller has no idea if it has completed.

FS doesn't really care about these StreamGobblers finishing.
Instead use Runnable with execute(Runnable), which doesn't
return a Future.

Change-Id: I93b66d1f6c869e66be5c1169d8edafe781e601f6
2017-01-21 09:44:14 +01:00
Matthias Sohn f503a9f5b7 Add missing @since tags on new API constants
Change-Id: Ia8b861da07fba99644ccc9eb5578a46cc39600a1
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-19 23:56:25 +01:00
James Melvin 91132bb05e gc: Add options to preserve and prune old pack files
The new --preserve-oldpacks option moves old pack files into the
preserved subdirectory instead of deleting them after repacking.

The new --prune-preserved option prunes old pack files from the
preserved subdirectory after repacking, but before potentially
moving the latest old packfiles to this subdirectory.

These options are designed to prevent stale file handle exceptions
during git operations which can happen on users of NFS repos when
repacking is done on them. The strategy is to preserve old pack files
around until the next repack with the hopes that they will become
unreferenced by then and not cause any exceptions to running processes
when they are finally deleted (pruned).

Change-Id: If3f729f0d9ce920ee2c3e6acdde46f2068be61d2
Signed-off-by: James Melvin <jmelvin@codeaurora.org>
2017-01-19 11:00:18 +01:00
David Pursehouse d3148f300a Make ObjectDownloadListener public
The same was already done for ObjectUploadListener in I5b0fb1220.

Change-Id: Ie8a79f715fe69bed9331962fb478f7e35acf8680
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-19 13:42:59 +09:00
David Ostrovsky e92a0c3adc Implement initial framework of Bazel build
The initial implementation only builds the packages consumed by
Gerrit Code Review.

Test build and execution is not implemented.

We prefer to consume maven_jar custom rule from bazlets repository,
for the same reasons as in the Gerrit project:

* Caching artifacts across different clones and projects
* Exposing source classifiers and neverlink artifact

TEST PLAN:

  $ bazel build :all
  $ unzip -t bazel-genfiles/all.zip
  Archive: bazel-genfiles/all.zip
    testing: libjgit-archive.jar      OK
    testing: libjgit-servlet.jar      OK
    testing: libjgit.jar              OK
    testing: libjunit.jar             OK
  No errors detected in compressed data of bazel-genfiles/all.zip.

Change-Id: Ia837ce95d9829fe2515f37b7a04a71a4598672a0
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-18 19:13:16 -04:00
Wim Jongman b667c182cb Normalizer creating a valid branch name from a string
Generic normalization method for a possible invalid branch name.
The method compresses dividers between spaces, then replaces spaces
and non word characters with underscores.

This method is needed in preparation for subsequent EGit changes.

Bug: 509878
Change-Id: Ic0d12f098f90f912a45bcc5693d6accf751d4e58
Signed-off-by: Wim Jongman <wim.jongman@remainsoftware.com>
2017-01-18 22:05:28 +01:00
Christian Halstrick 8a46b60371 Merge "Fix StashApplyCommand for stashes containing untracked changes." 2017-01-16 03:45:00 -05:00
Thomas Wolf 46af7192a2 Fix StashApplyCommand for stashes containing untracked changes.
If there are untracked changes, apply only the untracked tree
after a successful merge. The merge tree from merging untracked
with HEAD would also contain files already reset before (changes
in tracked files) and try to reset those again,leading to false
checkout conflicts.

Bug: 505804
Change-Id: Iaced4d277623334d11e3d1cca5969590d7c5093e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2017-01-15 21:54:12 +01:00
Marc Strapetz 1c4b3f8c45 Fix possible InvalidObjectIdException in ObjectDirectory
ObjectDirectory.getShallowCommits should throw an IOException
instead of an InvalidArgumentException if invalid SHAs are present
in .git/shallow (as this file is usually edited by a human).

Change-Id: Ia3a39d38f7aec4282109c7698438f0795fbec905
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-15 15:05:51 +01:00
Zhen Chen d3148f9410 Add missing slf4j dependency in BUCK file for jgit-lfs-server
Change-Id: I0c7fcae123f94ebd8e5eb2700e3d601a1b199be9
Signed-off-by: Zhen Chen <czhen@google.com>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-14 15:49:14 +09:00
Zhen Chen d6b354f60f Skip pack header bytes in DfsPackFile
The 12 bytes `PACK...` header is written in PackWriter before reading
CachedPack files. In DfsPackFile#copyPackBypassCache, the header was not
skipped when the first block is not in cache.

Change-Id: Ibbe2e564d36b79922a936657f286addb1044d237
Signed-off-by: Zhen Chen <czhen@google.com>
2017-01-13 22:10:42 -08:00
David Pursehouse 55c629a9f3 LfsProtocolServlet#LfsRequest: Add operation type helper methods
Server implementations may need to check what type of operation is
being performed. Add helper methods to make it a bit simpler.

Change-Id: Ia33ed6699ebcb9000ef514ce79bcddbebf94e1c5
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-14 01:07:52 +01:00
David Pursehouse 56fe217784 Expose LFS operation strings as public constants
So that they can be used either by third party LFS server implementations
or from within other parts of JGit's internal LFS implementation.

Change-Id: I58da6230247204588e017c010ce5b14e4615448d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-13 19:23:33 -04:00
David Pursehouse 590141163f LfsProtocolServlet: Improve error on getLargeFileRepository failure
Externalize the error message and make it more specific. Also emit
an error log.

Change-Id: Ie7b1c90c54673bfb8e133fb0aa19a117d4ca6587
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-14 00:05:56 +01:00
David Pursehouse 7245aa0316 Add support for refusing LFS request due to invalid authorization
Add a new exception type that server implementations can throw when a
client attempts to make an unauthorized LFS operation, which will result
in HTTP 401 Unauthorized being returned to the client.

An example of this is a Gerrit server that rejects a request to perform
an LFS operation on a ref that is not visible to the caller.

As defined in the LFS spec [1] the request may include authentication,
and per RFC 2616 [2], "401 response indicates that authorization has been
refused for those credentials".

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
[2] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Change-Id: I2aa22e2144df5fb7972df0e3285b77b08ecc63f2
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2017-01-13 10:46:15 +09:00
Dariusz Luksza 0e187f1484 Add LfsPointerFilter TreeFilter
Add new variation of TreeFilter in order to detect LFS pointer files in
the repository.

Additionally, update LfsPointer to support the legacy version URL [1] as
described in [2], and to allow arbitrary fields in the pointer file.

[1] https://hawser.github.com/spec/v1
[2] https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md

Change-Id: I621eb058619fb1b78888a54c4b60bb110a722fc3
Signed-off-by: Dariusz Luksza <dariusz@luksza.org>
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2017-01-10 00:13:24 +01:00
Shawn Pearce db77610256 Pack refs/tags/ with refs/heads/
This fixes a nasty performance issue for repositories that have many
objects referenced through refs/tags/, but not in refs/heads/.
Situations like this can arise when a project has made releases like
refs/tags/v1.0, and then decides to orphan history and start over for
version 2. The v1.0 objects are not reachable from master anymore,
but are still live due to the v1.0 tag.

When tags are packed in the GC_OTHER pack, bitmaps are not able to
cover the repository's contents. This may cause very slow counting
times during git clone, as the server must enumerate the ancient
history under refs/tags/ to respond to the client.

Clients by default always ask for all tags when asking for all heads
during clone. This has been true since git-core commit 8434c2f1afedb
(Apr 27 2008), when clone was converted to a builtin. Including tags
in the main GC pack should still allow servers to benefit from the
fast full pack reuse path when serving a clone to a client.

Change-Id: I22e29517b5bc6fa3d6b19a19f13bef0c68afdca3
2017-01-03 14:46:41 -08:00
Marc Strapetz 6087031469 Get rid of javax.servlet API dependency for core org.eclipse.jgit
Change-Id: I57d5d4fab7e0b1bd4cf5f1850e8569c8ac5def88
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
2017-01-03 18:50:55 +01:00
Matthias Sohn 5dc30db56e [findBugs] PackWriter.NONE should be final
Change-Id: I4b5621bcb4db82e0560408b3cde6f18b0cc55b29
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-30 01:19:58 +01:00
Matthias Sohn 29ddbf7fcd [findBugs] Remove reliance on default encoding in Base64
Change-Id: I6901da975a86c460ce7c783a519669d8be8e23bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 19:50:29 +01:00
Matthias Sohn f211bfc841 [findBugs] Fix potential NPE in CleanFilter
Change-Id: Ibc20e9fc16be62f61748b3f59f6d70c1ebf63671
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 01:09:03 +01:00
Matthias Sohn fc6f02cdba [findBugs] Fix potential NPE in ObjectDirectoryTest
Change-Id: I0307d1e9bbc0bfc8172eed10f8e93788d6e2c8b4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 01:06:06 +01:00
Matthias Sohn a85979c8b2 [findBugs] Fix potential NPE in GcPackRefsTest
Change-Id: I273c06b9b2ce36c8232803cb7f02d7f846ed49e5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 01:04:25 +01:00
Matthias Sohn f63267134f [findBugs] Fix potential NPE in GC
Change-Id: I59cda76b2c5039e08612f394ee4f7f1788578c49
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 00:59:33 +01:00
Matthias Sohn 48b772bec1 [findBugs] Fix potential NPE in DirCacheCheckoutTest
Change-Id: I842cbdd14bf56cb2bd599255fe90a5a3aea0d304
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 00:59:31 +01:00
Matthias Sohn 44936a1216 Remove unused API warning filters
Change-Id: If6d343a55245219a652e35c9cfde69349dc09889
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-29 00:59:02 +01:00
James Melvin d980a3fa85 Fix keep pack filename
Previously it was looking for a keep file with the name of a pack file
(extenstion included) appended with a '.keep'. However, the keep file
name should be the pack file name  with a '.keep' extension

Change-Id: I9dc4c7c393ae20aefa0b9507df8df83610ce4d42
Signed-off-by: James Melvin <jmelvin@codeaurora.org>
2016-12-27 14:08:56 -07:00
Matthias Sohn 5fee071f6a Prepare 4.7.0-SNAPSHOT builds
Change-Id: I20754d13007e6591d36aae5766f3a9a82b24e120
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-27 01:45:50 +01:00
Matthias Sohn b97fa05941 Merge branch 'stable-4.6'
* stable-4.6:
  Update tycho version to 0.26
  Update maven plugins
  Skip javadoc generation for org.eclipse.jgit.http.test
  Prepare 4.6.1-SNAPSHOT builds
  JGit v4.6.0.201612231935-r

Change-Id: Ia70329af7ec8bdb6d8bd6f917e00024a3ea4ac1f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-27 01:14:19 +01:00
Matthias Sohn 16e515a59e Update tycho version to 0.26
Change-Id: I1cf3402c057de89d1afe161d2d4106896fa8753f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-26 23:21:46 +01:00
Matthias Sohn dbd859a48d Update maven plugins
Change-Id: I2d18f69d25362f565df071c320970cbc973199cd
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-26 23:20:17 +01:00
Matthias Sohn 828612355f Skip javadoc generation for org.eclipse.jgit.http.test
This fixes the error the javadoc plugin raises when generating the maven
site for this bundle.

Change-Id: I72026aa33be86960747a246af4a70f6a91b40102
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 16:00:41 +01:00
Matthias Sohn 3857c3168f Prepare 4.6.1-SNAPSHOT builds
Change-Id: I6b05a6f6c3f92365c272e1bdaf76093ca01f2d58
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 15:51:54 +01:00
Matthias Sohn 73a4325149 JGit v4.6.0.201612231935-r
Change-Id: Iaa88fe1b195dfe6be99a7b4cb064684e75563715
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 01:42:38 +01:00
Matthias Sohn 5274da3c3c Merge branch 'stable-4.5'
* origin/stable-4.5:
  Fix one case of missing object

Change-Id: Ia6384f4be71086d5a0a8c42c7521220f57dfd086
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-24 00:30:00 +01:00
Matthias Sohn 1fb2319c18 [infer] Fix resource leak in IndexDiff
We only need the tree id to add it to a TreeWalk so change tree's type
to AnyObjectId.

Bug: 509385
Change-Id: I98dd5fef15cd173fe1fd84273f0f48e64e12e608
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2016-12-21 23:51:50 +01:00