jgit/org.eclipse.jgit
Shawn Pearce 5d446f410d Support cutting existing delta chains longer than the max depth
Some packs built by JGit have incredibly long delta chains due to a
long standing bug in PackWriter. Google has packs created by JGit's
DfsGarbageCollector with chains of 6000 objects long, or more.

Inflating objects at the end of this 6000 long chain is impossible
to complete within a reasonable time bound. It could take a beefy
system hours to perform even using the heavily optimized native C
implementation of Git, let alone with JGit.

Enable pack.cutDeltaChains to be set in a configuration file to
permit the PackWriter to determine the length of each delta chain
and clip the chain at arbitrary points to fit within pack.depth.

Delta chain cycles are still possible, but no attempt is made to
detect them. A trivial chain of A->B->A will iterate for the full
pack.depth configured limit (e.g. 50) and then pick an object to
store as non-delta.

When cutting chains the object list is walked in reverse to try
and take advantage of existing chain computations. The assumption
here is most deltas are near the end of the list, and their bases
are near the front of the list. Going up from the tail attempts to
reuse chainLength computations by relying on the memoized value in
the delta base.

The chainLength field in ObjectToPack is overloaded into the depth
field normally used by DeltaWindow. This is acceptable because the
chain cut happens before delta search, and the chainLength is reset
to 0 if delta search will follow.

Change-Id: Ida4fde9558f3abbbb77ade398d2af3941de9c812
2013-04-05 10:07:14 -07:00
..
.settings Harmonize the JDT settings within JGit 2012-11-16 10:25:45 -08:00
META-INF Grant access to jgit internals to junit and http.server bundles 2013-03-22 21:35:16 +01:00
findBugs [findBugs] Silence warning about Transport initialization 2012-09-14 00:01:47 +02:00
resources/org/eclipse/jgit/internal Allow to get repo statistics from GarbageCollectionCommand before gc 2013-03-20 00:46:27 +01:00
src/org/eclipse/jgit Support cutting existing delta chains longer than the max depth 2013-04-05 10:07:14 -07:00
.classpath Externalize strings from JGit 2010-05-19 14:37:16 -07:00
.fbprefs Initial JGit contribution to eclipse.org 2009-09-29 16:47:03 -07:00
.gitignore Finish removing Apache Felix maven-bundle-plugin 2010-01-12 11:46:55 -08:00
.project Revert "Hide Maven target directories from Eclipse" 2010-08-28 09:50:50 +02:00
about.html Add missing about.html files to all shipped bundles 2011-06-08 21:51:51 +02:00
build.properties Add "resources/" as a source folder 2010-06-05 14:39:27 +02:00
plugin.properties Remove incubation marker 2011-05-31 22:53:53 +02:00
pom.xml JGit 3.0: move internal classes into an internal subpackage 2013-03-18 09:30:43 -07:00