Programmatically measure performance of canvas animation

Question

I'm building a test platform for JavaScript performance contest. One of the tasks challenges contestants to optimize JavaScript code responsible for handling a canvas animation. After solution is submitted, server runs it using PhantomJS and reads average number of FPS after 20 seconds of animation. The problem is that I'm getting 3-4FPS for both optimized and unoptimized code. This makes it impossible to tell if code was improved.

Couple of facts:

• I'm 100% sure that phanotmjs is rendering animation correctly (done couple of screenshots)
• in a browser unoptimized code runs at 13FPS, optimized runs at 58FPS
• phantomjs doesn't support requestAnimationFrame so I had to use a polyfill
• I'm using this code below to test number of FPS

frameCounter.js

 var frameCounter = (function() {
    var frames = 0;
    var startTime = new Date();

    function bump() {
        frames++;
        window.requestAnimationFrame(bump);
    }

    bump();

    return {
        getFPS: function() {
            var time = (new Date() - startTime) / 1000;

            return (frames/time).toPrecision(4);
        }
    }
 })();

My question is: how can I programmatically measure performance of canvas animation?

Answer 1

Since phantomjs seems to be unable to produce more than 3-4 FPS on any animation, I ended up using 'real' browser for this task. I was able to automate it thanks to the Chrome's remote debugging protocol.

I made a node.js app that, each time there was new code to test, did the following steps:

• connected to a tab in the Chrome browser (the browser must be running with --remote-debugging-port=9222 flag)
• navigated tab to the test page
• evaluated code inside the tab that tried to render 300 frames of animation as quick as possible
• returned the execution time

Here is a snippet from my code:

//connect to a tab (you can find <tab-debug-id> on http://localhost:9222/json page)
var ws = new WebSocket("ws://localhost:9222/devtools/page/<tab-debug-id>");

ws.onerror = function() {
  //handle error
};

ws.onopen = function()
{
    //when connection is opened hard reload the page before we start
    ws.send(JSON.stringify({
        id: 1,
        method: "Page.reload",
        params: {
            ignoreCache: true
        }
    }));
};

ws.onmessage = function (evt)
{
    var data = JSON.parse(evt.data);

    if(data.id === 1) {
        //reload was successful - inject the test script
        setTimeout(function(){
           ws.send(JSON.stringify({
              id: 2,
              method: "Runtime.evaluate",
              params: {
                expression: '(' + injectedCode.toString() + '());'
              }
           }));
        }, 1000);
    } else if(data.id === 2) {
        //animation has finished - extract the result
        var result = data.result.result.value;
    }
};

Answer 2

I wrote a small script a few months back to specifically measure FPS and consumption for requestAnimationFrame.

I am not sure it will help you 100% but it can give you a good pointer.

The usage is quite simple:

• Initialize the meter somewhere in the code before loop where you specify the div element to be used as meter
• Make sure you grab the argument given by requestAnimationFrame as this will tell how much time is spent (if not it will fallback to using date/time method).
• Do a simple call to its method with this argument.

The green color indicates you are running within optimal FPS (60 in most cases). Yellow means the loop consumes more than the approximate 16.7 ms and the rate is down to about half. Orange means you're using more than double the budget and so forth.

The meter uses weighted FPS to give you a more accurate measurement.

Example:

var meter = new animMeter('divElementId');

function animate(timeArg) {

    /// funky stuff here

    meter.update(timeArg);

    requestAnimationFrame(animate);
}

A demo of this in action can be found here.

You'll find the code for the meter itself at almost the bottom pre-minimized. Feel free to copy and paste. It comes with a MIT license.

And as always when using meters like this: they will consume a few milliseconds themselves in order to update the graphics therefor introducing a tiny error margin.

Another thing to be aware of is that rAF will always run trying to achieve 60 FPS so the meter can never measure higher frame rates than this.

If you need to measure higher frame rates you can call the update method with no argument and use setTimeout instead of rAF, and it will use date/time to measure performance - slightly more inaccurate but you can get higher FPS numbers (that is arbitrary frames as the monitor cannot display more frames than it's synced for anyways.. typically 60 fps).

Answer 3

you can use Date.now() to reduce time wasted for object creation, it should improve precision at least a bit