Version control server performance when using CruiseControl (StarTeam or alternatives)

Question

We are using CruiseControl with a StarTeam server and having problems with the StarTeam server crashing. We are wondering if we are hitting the server too hard. Across 3 CruiseControl machines and a total of about 30 projects, we are logging into StarTeam and checking for modifications every minute or so. Most of our projects have ~20,000 files in them. Does anyone have experience with the performance limitations in this type of a scenario with StarTeam?

I'm also interested in performance metrics for CruiseControl with other version control systems -- such as TFS, Perforce, SVN, etc... Do they have scalability issues when using CruiseControl and a large number of projects with lots of files?

Answer 1

I've some experience with using Star Team for continous integration - though using TeamCity, not CruiseControl. In our case, the regular connections from TeamCity to StarTeam barely register as a blip on our performance monitoring.

Have you looked at the StarTeam log files on your server? - usually located in the root directory of your code vault or hive. I've usually found the logs sufficient to troubleshoot any issues.

Answer 2

You want to look into using the "compound modification set" as described here:

• Page 69 of Pragmatic Project Automation
• http://www.nabble.com/How-I-can-use-compound-modification-set-with-triggers--td19397718.html
• http://cruisecontrol.sourceforge.net/main/configxml.html#compound

Does StarTeam support "triggers"? Instead of having your CruiseControl machines checking the repository every minute for a change, you'd have your StarTeam machine letting CruiseControl know when the code has been changed by touching a file.

Basically, when an update is made to a project your VCS touches a file that CruiseControl monitors (like project-a-update.txt). Once it notices that the file has change CruiseControl knows to go bother doing an update from your VCS. So it's polling a single text file per project every N minutes, instead of the entire repository.

Answer 3

Our biggest performance improvement when running CC on several active projects was to install an MPX cache agent on the same machine and make sure it was storing file attributes and contents. (Well, that and making sure to use ccache in the builds.)

The idea of triggers exists in the StarTeam SDK, but I'm not sure how integrated it is with CC. To check if a build is required, we keep the last pull from the server in a clean directory and use stcmd list to compare for changes. This is very speedy.

Answer 4

As far as I can tell, StarTeam does not support triggers. The CruiseControl.NET StarTeam task only supports polling, and I have not seen any evidence anywhere that StarTeam supports triggers natively.

I am experiencing the exact same issue though of pushing StarTeam to it's limit. I am not certain, but I am beginning to think that the crashing issue is related to simultaneous accesses since the problems happen much more often when a project is highly active by developers as well as CruiseControl.

The penalty for polling for changes is also getting to be quite significant. Even worse is that the penalty is doubled. The first call to StarTeam is a call of "hist" to get the changeset for reporting and triggering a build, then a call of "co" does a second check of the entire source tree and checks out any out-of-date or missing files. If anybody can suggest a way to create a trigger around a StarTeam project that would eliminate either, I would love to give it a try.

Answer 5

An additional bit of information I was able to find.

StarTeam does not always disconnect a user (or agent in the case of CruiseControl accesses), and can result in having dozens of open connections/logins for the same user/agent. Unfortunately, it is very hard to confirm if this is a related issue because I cannot reliably overload the server.

We are using an older version of StarTeam that does not have MPX. I wonder if anyone has experience using StarTeam on a different continuous integration environment that supports the MPX method.

Answer 6

Firstly, I have no experience of StarTeam! I do have experience of CruiseControl and subversion though.

You could change your schedule interval to something less aggressive than 1 minute - 3-5 minutes is usually acceptable for checking projects to see if they need building. But I accept there may be a reason that you need to check very frequently.

Subversion scales very well. This is because whenever cruise checks to see if there is an update all it does is compare the local version number with the remote version number of the project, it does not need to check each file. The speed of this check is therefore not affected by the number of files in the project.

I hope this helps you.