The most excellent Magmi will do that for you. I suggest you read every single page of the Magmi instructions slowly and carefully to comprehend the enormous capabilities of this software tool. But the one you want to concentrate on is here.
So if you use a CSV like:
"sku","url_key"
"00085722994075","my-awesome-product-you-will-fail-without-it"
Then Magmi will update all the product 'slugs' for you.
To answer your question: The core_url_rewrite is the main table but it is built from, among other things, the product attribute 'url_key' and the product attributes are in the EAV so it isn't a single table update. That is why we bow to the genius of Dweeves and his magical Magmi code.