How to tune EGit for large repositories?

Question

The problem:

I find EGit great and use it intensively, but it can be incredibly slow. It can get frustrating when it takes several minutes to complete operations that the C version of git (Cgit) does in less than a couple of seconds.

All operations are significantly slower than Cgit. For example switching branches will take 10's of seconds compared to near instant. A rebase can take several minutes compared to less than a couple of seconds.

Some details:

History size: 10114 commits as reported with: git rev-list HEAD --count

Current Working directory size: 63.7 MB

Current .git size: 77.4 MB

Largest file size: 4.0 MB

OS: Linux - CentOS 5.5

File System: ext3

JVM: Oracle - Java(TM) SE Runtime Environment (build 1.7.0_21-b11)

EGit and JGit version: 3.0.0.201306101825-r

I was previously running 2.3 but did not notice any change in performance after upgrading.

Could suitable window cache settings help:

I found the following quote in JGit's bugzilla here:

...EGit had to expose UI to allow users to configure it when working on bigger repositories.

Which sounds like it fits my case. So I looked around in eclipse and under Window -> Preferences -> Team -> Git found these Git Window Cache settings:

But how do I use them?

What do the different controls actually do? Has anyone had any success in getting EGit to be more responsive by using them?

Answer 1

EGit 3.5.0 will bring huge performance fix for large repositories - without "tuning" anything. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=440722

You can use nightly EGit build update site to get the fix immediately: http://download.eclipse.org/egit/updates-nightly

Answer 2

Recommendations

As Matthias Sohn suggested, the Window cache limit appears to be most significant of these parameters.

For me, increasing this from "10 m" to "500 m" made a huge difference to how responsive egit was.

Details of each parameter

From the source code^† of WindowCacheConfig.java:

Windows Size

packedGitWindowSize: size in bytes of a single window mapped or read in from the pack file

Default: 8 k

Window cache limit

packedGitLimit: maximum number bytes of heap memory to dedicate to caching pack file data.

Default: 10 m

Delta base cache limit

deltaBaseCacheLimit: maximum number of bytes to cache in delta base cache for inflated, recently accessed objects, without delta chains.

Default: 10 m

Stream File Threshold

streamFileThreshold: the size threshold beyond which objects must be streamed.

Objects smaller than this size can be obtained as a contiguous byte array, while objects bigger than this size require using anObjectStream.

Default: 50 m

Use virtual memory mapping

packedGitMMAP: true enables use of Java NIO virtual memory mapping for windows; false reads entire window into a byte[] with standard read calls.

Default: Unchecked

Not presented on the preferences page

packedGitOpenFiles: maximum number of streams to open at a time. Open packs count against the process limits.

Default: 128

---

^{† Thanks Jens Theeß for their comment on Matthias Sohn's answer containing a pointer to the source code.}

Answer 3

• increase the window cache limit to a larger value, it's defining the size of jgit's cache used to map pack files into memory
• do you run gc (either from egit or native git) on your repository ? JGit/EGit don't run it automatically (yet). YOu may check the number of loose object in EGit's repositories view (click "Properties" and select tab "Statistics")
• how many commits does your repository have (age in year doesn't say anything since it could have an age of 10 years with 2 commits or 2 mio commits)
• which OS / filesystem are you using ?