No, you're not doing anything wrong. Blink has a bug where we do not support this, which was reported to me by someone else just last week, and I filed: https://code.google.com/p/chromium/issues/detail?id=311284. We will get that fixed.
In the meantime, it's actually relatively easy to do a vibrato effect on ANY audio connection (not just buffersourcenodes) using an LFO to drive oscillations on the delayTime of a delayNode - check out the "Vibrato" effect I added to the end of http://webaudiodemos.appspot.com/input/index.html, and the node chain I set up to do it: https://github.com/cwilso/Audio-Input-Effects/blob/master/js/effects.js#L478 is the vibrato subgraph creation routine.