There is an example of using images as labels here:
https://www.rgraph.net/demos/bar-images-as-labels.html
But keep in mind that they're not DOM objects - ie you can't style them with CSS on add mouseover events to them (at least not in the standard way). You could add a mouseover using RGraph though.