Commit Graph

8994 Commits

Author SHA1 Message Date
Thomas Wolf b12ec8f377 LFS: clean up messages
Remove an unused message, and remove unnecessary quotes.

Change-Id: I9da3eeb3db33324fa07bebd434bca3c0a4da27ab
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-21 04:58:50 -04:00
Thomas Wolf 0ae37d4000 DirCacheCheckout: use a LinkedHashMap instead of HashMap
This guarantees that updates are checked out in git order, which
is important for LFS if a .lfsconfig file is used. That file comes
early in git order, and the LFS smudge filter will consider the
working tree version. To ensure that on branch switches the correct
version of that file is used, the checkout order must be stable and
should be the git order.

Change-Id: I20f6d11bf08558f9d5adfd2be71e36321460038c
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-19 21:41:55 +01:00
Thomas Wolf 7b1c8cf147 Re-try reading a file when there are concurrent writes
Git and JGit are very careful to replace git files atomically when
writing. The normal mechanism for this is to write to a temporary
file and then to rename it atomically to the final destination. This
works fine on POSIX-compliant systems, but on systems where renaming
may not be atomic, exceptions may be thrown if code tries to read
the file while the rename is still ongoing. This happens in particular
on Windows, where the typical symptom is that a FileNotFoundException
with message "The process cannot access the file because it is being
used by another process" is thrown, but file.isFile() == true at the
same time.

In FileBasedConfig, a re-try was already implemented for this case.
But the same problem can also occur in other places, for instance
in RefDirectory when reading loose or packed refs. Additionally,
JGit has similar re-tries when a stale NFS file handle is detected,
but that mechanism wasn't used consistently (only for git configs
and packed refs, but not for loose refs).

Factor out the general re-try mechanism for reading into a new method
FileUtils.readWithRetry() and use that in all three places. The
re-try parameters are hardcoded: at most 5 times for stale NFS handles,
and at most 5 times with increasing backoff delays (50, 100, 200, 400,
and 800ms) for the above concurrent write case.

Bug: 579116
Change-Id: If0c2ad367446d3c0f32b509274cf8e814aca12cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-19 21:33:51 +01:00
Thomas Wolf ac78c17523 [sideband] Ensure last bit of progress channel is written
If the last sideband progress message didn't end in \r or \n, there
may still be a buffered message at the end of a fetch or push. Ensure
that message gets written, too, even if it may be only partial.

Bug: 575629
Change-Id: I38edccb5cffb89e00e468480b43c7d951fb63e8e
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-19 12:13:37 +01:00
Thomas Wolf 45287f1097 Add tests for SideBandInputStream
Bug: 575629
Change-Id: I4591a64a2c07c7841417ec7a5c7883134ffda1be
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-19 12:12:05 +01:00
Fabio Ponciroli a820ff0896 WinPipeConnector: Resolve warnings by using "equals" instead of "=="
Errorprone raises the following warning: "[ReferenceEquality] Comparison
using reference equality instead of value equality".

Change-Id: Ic73305b2af628f66e7659ea1b839b3a640e3cd8d
2022-03-17 16:03:16 -05:00
Thomas Wolf 2c1a8798ca LsRemoteCommand: apply url.*.insteadOf
If LsRemoteCommand is used without repository, the remote URI was not
translated. If the git user config contains e.g.

  [url "ssh://git@github.com/"]
    insteadOf = https://github.com/

and LsRemoteCommand is called with an HTTPS GitHub repository URL, the
command should actually rewrite this to an SSH URI and use the SSH
transport.

Actually this same problem may exist everywhere Transport is used with
an URIish instead of with a remote name. However, doing this translation
in Transport.open(URIish) and in Transport.open(Repository, URIish,
String) if no remote name is given would change the behavior and might
break assumptions made in existing clients. For instance, EGit's
PushOperation assumes that the URI obtained from PushResult.getURI()
was the same as was passed in to Transport.open(Repository, URIish).

URIs obtained from a RemoteConfig have this translation applied
transparently, and in Transport we cannot know for sure whether or
not a URI has already been translated, if needed. So doing this in
Transport might also lead to translating URIs twice.

Therefore this commit does the translation in LsRemoteCommand, where
we can be sure that it won't affect other use cases. If other cases
besides LsRemoteCommand are found where such a URI translation is
missing, it'll have to be done at higher levels, possibly even in
client code directly.

Bug: 544769
Change-Id: I5df54a925d30b55d98e21f37f2851fe79649b064
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-15 20:58:52 +01:00
Thomas Wolf c127a1a652 Factor out URL replacement from RemoteConfig
Add a new UrlConfig that encapsulates the basic URL replacement logic
for git configs url.<base>.insteadOf and pushInsteadof.

Change-Id: Iecc61d1c5e6089533552afa1d1e684ae72393b38
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-15 20:43:51 +01:00
Thomas Wolf 2d9ed3cf2a Fix MANIFEST.MF in the LFS bundles
Somehow new dependencies added in commit a229072f got lost when merging
stable-6.1 into master.

Change-Id: I19c16c6be73d07e05f37241a44cf25133ff467a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-12 17:28:47 +01:00
Matthias Sohn 13eb2ce3e2 Update Tycho to 2.6.0
Change-Id: Ic8b2b352721040ccc0164cf0c8637102518e5ce9
2022-03-09 22:34:16 +01:00
Matthias Sohn 338f2adf16 Merge branch 'stable-6.1'
* stable-6.1:
  Prepare 6.1.1-SNAPSHOT builds
  JGit v6.1.0.202203080745-r
  [checkout] Use .gitattributes from the commit to be checked out
  Don't use final for method parameters
  [push] support the "matching" RefSpecs ":" and "+:"
  [push] Call the pre-push hook later in the push process
  IndexDiff: use tree filter also for SubmoduleWalk
  Run license check with option -Ddash.projectId=technology.jgit
  Exclude transitive dependencies of sshd-sftp
  Update DEPENDENCIES for 6.1.0 release
  Add dependency to dash-licenses
  Fix typos of some keys in LfsText
  Sort LfsText entries alphabetically
  Support for "lfs.url" from ".lfsconfig"

Change-Id: I1b9f0c0ed647837e00b9640d235dbfab2329c5a6
2022-03-09 21:45:35 +01:00
Matthias Sohn 8f7ef245f0 Prepare 6.1.1-SNAPSHOT builds
Change-Id: Ifc80355025d8459245843be1c24dc5a286913e77
2022-03-08 17:04:10 +01:00
Matthias Sohn e982de3fcb JGit v6.1.0.202203080745-r
Change-Id: I8766ed400020c9571f321bbbfe34b0688af0107d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2022-03-08 13:45:56 +01:00
Thomas Wolf f26ab4ebee [checkout] Use .gitattributes from the commit to be checked out
JGit used only one set of attributes constructed from the global and
info attributes, plus the attributes from working tree, index, and
HEAD.

These attributes must be used to determine whether the working tree is
dirty.

But for actually checking out a file, one must use the attributes from
global, info, and *the commit to be checked out*. Otherwise one may not
pick up definitions that are only in the .gitattributes of the commit
to be checked out or that are changed in that commit with respect to
the attributes currently in HEAD, the index, or the working tree.

Maintain in TreeWalk different Attributes per tree, and add operations
to determine EOL handling and smudge filters per tree.

Use the new methods in DirCacheCheckout and ResolveMerger. Note that
merging in JGit actually used the attributes from the base, not those
from ours, which looks dubious at least. It now uses those from ours,
and for checking out the ones from theirs.

The canBeContentMerged() determination was also done from the base
attributes, and is newly done from the ours attributes. Possibly this
should take into account all three attributes, and only if all three
agree the item can be content merged, a content merge should be
attempted? (What if the binary/text setting changes between base, ours,
or theirs?)

Also note that JGit attempts to perform content merges on non-binary
LFS files; there it used the filter attribute from base, too, even for
the ours and theirs versions. Newly it takes the filter attribute from
the correct tree. I'm not convinced doing content merges on potentially
huge files like LFS files is really a good idea.

Add tests in FilterCommandsTest and LfsGitTest to verify the behavior.

Open question: using index and working tree as fallback for the
attributes of ours (assuming it is HEAD) is OK. But does it also make
sense for base and theirs in merging?

Bug: 578707
Change-Id: I0bf433e9e3eb28479b6272e17c0666e175e67d08
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-07 18:45:25 +01:00
Michael Keppler 600e96dfdc Refer to target platform as file
Since Tycho 2.3 the target platform can be referenced as file, without
using Maven coordinates.

Signed-off-by: Michael Keppler <Michael.Keppler@gmx.de>
Change-Id: I120223dd3dc740fdff13a69201624b4cdc02c6f6
2022-03-07 15:52:58 +01:00
Matthias Sohn 72bba7bd53 Don't use final for method parameters
See https://wiki.eclipse.org/EGit/Contributor_Guide#Use_of_the_.22final.22_modifier

Change-Id: Idc6ed85483e381689e5085c4a1bacd75d26f5489
2022-03-06 18:52:59 +01:00
Thomas Wolf 8a2c769417 [push] support the "matching" RefSpecs ":" and "+:"
The implementation of push.default=matching was not correct.
It used the RefSpec "refs/heads/*:refs/heads/*", which would push
_all_ local branches. But "matching" must push only those local
branches for which a remote branch with the same name already exists
at the remote.

This RefSpec can be expanded only once the advertisement from the
remote has been received.

Enhance RefSpec so that ":" and "+:" can be represented. Introduce a
special RemoteRefUpdate for such a RefSpec; it must carry through the
fetch RefSpecs to be able to fill in the remote tracking updates as
needed. Implement the expansion in PushProcess.

Bug: 353405
Change-Id: I54a4bfbb0a6a7d77b9128bf4a9c951d6586c3df4
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 17:30:01 +01:00
Thomas Wolf 90df7c123e [push] Call the pre-push hook later in the push process
Call the pre-push hook only after having received the remote
advertisement and having determined rejections, like C git does.
Also similar to C git, don't pass rejected or up-to-date updates
to the pre-push hook.

Bug: 578852
Change-Id: I51d379ea7bd8234ec815f8f4a9fa325816f476cf
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 17:30:01 +01:00
Thomas Wolf 72ae234e79 IndexDiff: use tree filter also for SubmoduleWalk
The only uses of IndexDiff.setFilter() in JGit and EGit set a path
filter. Passing the filter on to the SubmoduleWalk gives the desired
result, which is consistent with command-line git.

Bug: 565251
Change-Id: I8eca1ed73eb1d237b8785f369352f72af9e0e168
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-03-06 11:11:26 +01:00
Matthias Sohn 46d58b84c9 Run license check with option -Ddash.projectId=technology.jgit
This ensures that works-with-dependencies are attributed properly.

Change-Id: Ie41733cdecc83ae0b532fb4bd9f692bc09e16107
2022-03-04 07:56:50 +01:00
Matthias Sohn c77284ab7f Exclude transitive dependencies of sshd-sftp
We don't need the dependencies of sshd-sftp to sshd-common and
sshd-core since they are contained in sshd-osgi.
Excluding them helps the dash IP log tool to not list them as required
dependencies of jgit.

Change-Id: I85bce4dca1b99ba880cf3e909fac9669c75e7854
2022-03-04 00:30:36 +01:00
Matthias Sohn aed354a1ed Update DEPENDENCIES for 6.1.0 release
Change-Id: I9ea925bd444952dc204e2538b9d16fe631202e0c
2022-03-03 22:11:06 +01:00
Matthias Sohn 996e651c14 Add dependency to dash-licenses
This is required to run the license check using the license tool [1]
required by the Eclipse project handbook [2]:

mvn org.eclipse.dash:license-tool-plugin:license-check -Ddash.summary=DEPENDENCIES

Note: the tool still requires Java 11 hence it needs to be run in a
separate build step and is not yet integrated in the build which runs
on Java 8.

[1] https://github.com/eclipse/dash-licenses
[2] https://www.eclipse.org/projects/handbook/#ip-license-tool

Change-Id: Ib41d54de246c3c9499cc3be9f026294c39fdfd99
2022-03-03 22:00:28 +01:00
Matthias Sohn d0ed6016d4 Prepare 6.2.0-SNAPSHOT builds
Change-Id: Ic2dde88bee3242169d6fa50956f8938f3fc4ba8e
2022-03-03 10:49:58 +01:00
Matthias Sohn 877c79dc69 Fix typos of some keys in LfsText
Change-Id: I86dee0b68e627e26cbd29976162bc7b953ebf276
2022-03-03 10:47:46 +01:00
Matthias Sohn 5ac27a55b0 Sort LfsText entries alphabetically
Change-Id: I3021cb246b51d861d51258808c75d5c6843ec82d
2022-03-03 10:47:46 +01:00
Matthias Fromme a229072fcd Support for "lfs.url" from ".lfsconfig"
- New class LfsConfig to enrich repository configuration by settings
from ".lfsconfig" file respecting configuration file precedence.
- Adapted LfsConnectionFactory to use LfsConfig instead of directly
using configuration from repository to calculate url of the lfs
repository

Bug: 578020
Change-Id: I156f4ec137c2e428136a2ca9b8a4011ecee2d915
2022-03-03 10:47:45 +01:00
Matthias Sohn dc6be13640 Merge branch 'stable-6.1'
* stable-6.1:
  Update Orbit to R20220302172233 for 2022-03
  Prepare 6.1.0-SNAPSHOT builds
  JGit v6.1.0.202203021511-rc1
  Prepare 6.1.0-SNAPSHOT builds
  JGit v6.1.0.202202221755-m3

Change-Id: Iff4b13ca8ae5e13a3f2f45c0d4653405f1ceb2f4
2022-03-02 23:02:21 +01:00
Matthias Sohn e2a484f33d Update Orbit to R20220302172233 for 2022-03
Change-Id: Ief504bf7f926d461e38186bf00711082d619fc08
2022-03-02 22:31:45 +01:00
Matthias Sohn 60d52c9cfd Prepare 6.1.0-SNAPSHOT builds
Change-Id: I525fc1258a729c36d63fdb3c8170e9f04ad55cec
2022-03-02 22:28:46 +01:00
Matthias Sohn 4b9fb3161c JGit v6.1.0.202203021511-rc1
Change-Id: I4c75a58fd76102e773af4f1f8a1487d5e7ffc7cf
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2022-03-02 21:11:54 +01:00
Matthias Sohn 7a372d1d00 Merge branch 'master' into stable-6.1
* master:
  Describe: add support for core.abbrev config option
  Add a typed config getter for integers confined to a range
  Remove odd prefix of PersonIdent test class
  PersonIdent: Add ctors that accept Instant in addition to Date
  Remove ignored potentiallyUnclosedCloseable check
  Make precedence more explicit
  [pgm] Add describe --abbrev option
  Cap describe abbrev option
  DescribeCommand: Add support for --abbrev=0

Change-Id: I1daa3501a38d57b628800fadb96b6b71eea8cbb3
2022-03-02 21:10:03 +01:00
Matthias Sohn 6f175ea6c4 Describe: add support for core.abbrev config option
If core.abbrev is unset or "auto" estimate abbreviation length like C
git does:
- Estimate repository's object count by only considering packed objects,
  round up to next power of 2
- With the order of 2^len objects, we expect a collision at 2^(len/2).
  But we also care about hex chars, not bits, and there are 4 bits per
  hex. So all together we need to divide by 2; but we also want to round
  odd numbers up, hence adding one before dividing.
- For small repos use at least 7 hexdigits
- If object database fails to determine object count use 7 hexdigits as
  fallback

If it is set to "no" do not abbreviate object-ids.

Otherwise set it to the configured value capped to the range between 4
and length of an unabbreviated object-id.

Change-Id: I425f9724b69813dbb57872466bf2d2e1d6dc72c6
2022-03-02 19:29:48 +01:00
Matthias Sohn 9244c07d73 Add a typed config getter for integers confined to a range
Use Integer#MIN_VALUE to denote unset option.

Change-Id: I4d65f2434013111f25520c0ed2b9a9dc8123c6cf
2022-03-02 19:28:14 +01:00
Matthias Sohn 9284ed5db7 Remove odd prefix of PersonIdent test class
Change-Id: I05ede49902527c435691fb2c0f0d4da7ba077b7a
2022-03-02 16:38:45 +01:00
David Ostrovsky 67097f5de4 PersonIdent: Add ctors that accept Instant in addition to Date
Error Prone is flagging Date-API as obsolete and recommends to migrate
to Instant and LocalDate. Given that more JGit users starting to migrate
to new Time API, offer ctors that accept Instant type and also add new
getter that returns when attribute as Instant type.

Change-Id: I64a36bf40f191495c6889c1dff314ede06848880
2022-03-02 16:38:44 +01:00
Fabio Ponciroli 889b82f0e5 Remove ignored potentiallyUnclosedCloseable check
potentiallyUnclosedCloseable currently is not checked and it creates
a warning when compiling with Maven.

Change-Id: I0715ead433270937f77f56a19aa203f8d6f5322f
2022-03-02 16:32:34 +01:00
Fabio Ponciroli c543b8ee1a Make precedence more explicit
Errorprone was failing the build with: "[OperatorPrecedence]. Use
grouping parenthesis to make the operator precedence explicit"

Add parentheses to silence it.

Change-Id: I81f1f249e38fd2543f5412b3501b0179d0759f55
2022-03-02 16:24:44 +01:00
Matthias Sohn 9c27002c49 [pgm] Add describe --abbrev option
Change-Id: I8adf2fad21db71b43266d3f274143eee6bc9dce2
2022-03-02 09:56:04 +01:00
Matthias Sohn 85d8b31cb2 Cap describe abbrev option
- minimum is 4 [1]
- maximum is length of a full ObjectId

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-coreabbrev

Change-Id: I145bde1a218f71b87b8d8260761dd0853770bb76
2022-03-02 09:56:03 +01:00
Matthias Sohn a2d5650b8f DescribeCommand: Add support for --abbrev=0
Setting --abbrev=0 suppresses long format and only shows the closest tag
[1].

[1] https://git-scm.com/docs/git-describe#Documentation/git-describe.txt---abbrevltngt

Change-Id: Ifcf4d7786dd0f0fb0315d8093fdb54384ed9d5f9
2022-03-02 09:56:03 +01:00
Matthias Sohn 8845f2e91b Prepare 6.1.0-SNAPSHOT builds
Change-Id: I30a89277373ad62c857151532942e135e34d1d0c
2022-03-01 17:21:11 +01:00
Matthias Sohn 261f46dc43 JGit v6.1.0.202202221755-m3
Change-Id: Idcd46e05ca3eec376f8ac83209dba0978e004f9a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2022-02-22 23:55:35 +01:00
Fabio Ponciroli 00bb7cc9a5 Remove SuppressWarnings since currently ignored
The following warning was raised by Eclipse:
"At least one of the problems in category
'unused' is not analysed due to a compiler option being ignored"

The org.eclipse.jdt.core.compiler.problem.unusedTypeParameter compiler
option is set to ignore, hence the warning suppression is redundant.

Change-Id: I5fd6efc9968be133c71e95c53803c44310d75e85
2022-02-22 23:21:28 +01:00
Fabio Ponciroli 73831b34b8 Add comment to explain why try-with-resource is not used
Change-Id: I904f9890428334bdf2971511e0b2681f3da5a82a
2022-02-22 13:55:27 +01:00
Sebastian Schuberth a7386ffe3a DescribeCommand: Support configuring the hash abbreviation
Bug: 537883
Signed-off-by: Sebastian Schuberth <sebastian.schuberth@bosch.io>
Change-Id: Ic52dcebc564bbb0d934cc3a6205704b7aeaee30e
2022-02-19 22:04:28 +01:00
Thomas Wolf 69ef598bd9 Simplify implementation of WorkingTreeIterator
All the filtering in WorkingTreeIterator is for check-in, i.e., clean
filtering. The implementation was in some parts too general, passing
around an OperationType. But since it's always CHECKIN_OP, that's not
actually necessary.

Change-Id: I73f8bc059e485a073e456962868f52b3a3db4fc1
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-02-16 18:51:28 +01:00
Matthias Sohn d73b7cdeb4 Update Orbit to S20220215213605
and
- org.apache.httpcomponents.httpcore to 4.4.15.v20220209-2345

Change-Id: I0c7a8cf5a282596fe3d3dac422716480b3d76c83
2022-02-16 13:09:15 +01:00
Thomas Wolf 63c1c6e4d8 Transport: load all refs only if push refspecs have wildcards
There is no need to load all refs if there are no wildcard push
refspecs. Load them lazily on the first wildcard refspec encountered
instead of loading them up-front.

Change-Id: I6d0e981f9ed4997dbdefeb7f83f37ff4f33e06a5
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-02-14 10:45:15 +01:00
Thomas Wolf 855a734875 PushCommand: determine remote from git config if not given
Add ConfigConstants and expose branch.<name>.pushRemote in the
BranchConfig. Use the branch configuration and remote.pushDefault
if no remote is given explicitly. If nothing is configured, fall
back to "origin".

Bug: 578676
Change-Id: I6bb141ff02c8b04980ec34b26ef248b72614c3c9
Signed-off-by: Thomas Wolf <thomas.wolf@paranor.ch>
2022-02-14 10:45:15 +01:00