This is most certain MRI's garbage collector swiping the memory. I did some testing with a simple Rails app and a partial:
100.times{Test.new}
I could see the spikes just like you:
Rendered tests/_row.html.erb (3.9ms)
Rendered tests/_row.html.erb (45.3ms)
Rendered tests/_row.html.erb (5.2ms)
...
Rendered tests/_row.html.erb (42.8ms)
Using mini-profiler, it's pretty easy to check how many times the GC was called and, more important, how long it ran. For me, these numbers perfectly matched. If there were 10 spikes, there were also 10 (+/-1) GC calls and also the runtimes of the GC were exactly the difference between the normal renderings and the ones that took longer.
To use the GC timer of mini-profiler append ?pp=profile-gc-time
to your URL. There's also a great post about tuning Ruby with mini-profiler.