Found the solution.
brew install libxml2 --with-python
Some files were installed into /usr/local/lib/python2.7/site-packages/libxml2*
. I copied them to my virtualenv's site-packages and voila things worked!
Symlinking might be something better to try.