- Depending on the complexity of the image (filters, masks, gradients...), it could consume more CPU to render the image; the good thing is that more and more graphic tasks are sent to the GPU, so that's not a big problem
- Not compatible with older browsers
- Browser support isn't perfect yet, so if you're using complex features, there might be differences in the way the final image looks
But I strongly believe that vector graphics have much more advantages than disadvantages, so go for it.