Mitchell Hashimoto (the developer of vagrant) put together some test results that show the speeds of different virtual machine file systems. Long story short, native (non shared folders) >>>>>> NFS >> Virtual Box shared folders.
So switching to NFS is a good improvement. However, the latest version of Vagrant added support for Rsync Synced Folders. Using them is a little different from normal shared folders, but the benefit is that in VM file system access is blazing fast. It should be equivalent to the "Native Virtual Box" performance reported in his first blog post.