Thanks arkascha.
Turns out I had a couple problems. Here is what I ended up with:
in /shop/.htaccess
RewriteCond %{REQUEST_URI} !^/shop/(media|skin|js|customer|admin|sales|checkout)/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*) /$1 [L]
I needed to not rewrite URLs matching a file or directory. Also, it seems that having /shop/ was a requirement for the RewriteCond, but would not work in the RewriteRule.