Hm, it seems it needs another anchor. The below works
<div id="customBtn" class="customGPlusSignIn">
<span id="gp" class="gplus"></span>
</div>
Thanks @Dyllen James Owens for suggesting a JSfiddle.
You can find the code at fiddle http://jsfiddle.net/mHcdv/3/