Can I make the SCM polling interval for Jenkins randomised

Question

In most Jenkins' examples the SCM poll value is designated */15 * * * * , ie. poll SCM every 15 minutes. That's fine when you have hundreds of jobs, but not if you have thousands of jobs as it results in thousands of request to the SCM (Subversion in this case) server at 15, 30, 45, and 0 minutes past the hour.

Is there some way to randomize the polling value in Jenkins so as to avoid the above scenario?

On the Jenkins master configuration screen there's a value "Max # of concurrent polling." Should this be set (and to what value) to avoid the above scenario?

Answer 1

Use H instead of * and Jenkins will randomly distribute the polling. Note that at present a good syntax has not been found for a frequency differing from once per hour/day/etc, so

H * * * *

will poll once per hour at a a pre-determined random minute.

H H * * *

will poll once per day at a pre-determined random hour and minute

H H H * *

will poll once per week

0 H * * *

will poll on the hour but once per day at a pre-determined random hour.

Keep in mind that you are allowed multiple cron lines and any will match, so until a good syntax for sub-hour frequency has been settled on, you can get closer (on average) with something like

H * * * *
H * * * *
H * * * *
H * * * *
H * * * *
H * * * *

will give you on average 15 minutes between polling (yes there will be 6 pollings per hour, but that is to give a good chance that at least one polling will fall in each quarter hour)

If you have good suggestions for a syntax distributing within the hour please respond to this thread:

https://groups.google.com/forum/?fromgroups#!jenkinsci-users/VghEjfygWuw/PuIG1o7u1GQJ%5B1-25%5D

Update (April 2013)

Jenkins 1.510 and newer includes a new syntax to allow specifying distributions within the hour

Answer 2

If you set max# of concurrent polling to something between 1 and 3 then the polling requests will simply queue up and be processed serially (or at most 3 in parallel).

Given that all that is involved in the polling for SVN is effectively

svn info branch-url

They should all complete quickly, but at least you will have limited the parallel requests.

Answer 3

@Stephen Connolly's solution is likely a good solution for this.

If you really have thousands of jobs to worry about, consider setting the timing based on some part of the job name (Assuming your job names are reasonably distributed).

For example, if your job starts with 'B', set the timings to */2. If it starts with 'C', set it to */3 etc. ('A' would need something other than 1 though).

It's not a terrific solution, but if you're really looking at managing that many jobs it might be a work-around until you can find a better solution.

Answer 4

H H H * *

I think above will poll once per month not per week