If Apache is using /var/www as your root, I don't think it will allow access to anything outside of that directory. Have you tried accessing those files directly in a browser, outside of video.js?
Additionally, by setting the source URL to '/home/user/video/video2.mp4', you're telling the browser to look for the file at "www.mywebsite.com/home/user/video/video2.mp4', which apache will translate to "/var/www/home/user/video/video2.mp4".
I'm far from an expert on how Apache works, but I hope that helps.