I am running a shop with about 1500 products and over a 100 categories (quite some products are in multiple categories. I run with Joomla 2.5.xx and VM 2.0.xx. Speed was never an issue.
It all of course depends on the environment it will run in/on. Since you do not give any info on that, my answer above will have to do :)