Using both matplotlib and rpy2 with multi-buildpacks on Heroku

Question

I'm trying to use a multi-buildpack setup on Heroku with these two buildpacks:

https://github.com/virtualstaticvoid/heroku-buildpack-r.git
https://github.com/dbrgn/heroku-buildpack-python-sklearn/

I am using rpy2 to call R from python. I detailed the full process I used to get the slug to compile here.

It works fine for numpy, scipy, and scikit-learn with rpy2. However, I'm also trying to get matplotlib to work with this setup, and I'm getting an error.

I used matplotlib==1.1.0, as suggested by this StackOverflow post.

However, when I have my LD_LIBRARY_PATH set so that rpy2 will work, like this:

LD_LIBRARY_PATH=/app/vendor/R/lib64/R/modules:/app/vendor/R/lib64/R/lib:/app/vendor/gcc-4.3/lib64

I get this error:

>>> from matplotlib import ft2font
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /app/vendor/R/lib64/R/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /app/.heroku/python/lib/python2.7/site-packages/matplotlib/ft2font.so)

If I remove the LD_LIBRARY_PATH settings, then matplotlib works, but rpy2 can't find the R library located in /app/vendor/R/lib64/R/lib. Changing the order of the directories in LD_LIBRARY_PATH doesn't seem to have an effect, for some reason.

So I can get either matplotlib or rpy2 to work, but not at the same time.

I have libraries in these locations:

~ $ find . -name "*libstd*"   
/app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.so.6.0.10
/app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.so.6
/app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.so
/app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.a
/app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.la
/app/vendor/gcc-4.3/gcc-4.3/lib/libstdc++.so.6.0.10
/app/vendor/gcc-4.3/gcc-4.3/lib/libstdc++.so.6
/app/vendor/gcc-4.3/gcc-4.3/lib/libstdc++.so
/app/vendor/gcc-4.3/gcc-4.3/lib/libstdc++.a
/app/vendor/gcc-4.3/gcc-4.3/lib/libstdc++.la
/app/vendor/R/lib64/R/lib/libstdc++.so.6.0.10
/app/vendor/R/lib64/R/lib/libstdc++.so.6
/app/vendor/R/lib64/R/lib/libstdc++.so
/app/vendor/R/lib64/R/lib/libstdc++.a
/app/vendor/R/lib64/R/lib/libstdc++.la

I suspect that matplotlib should be using /app/vendor/gcc-4.3/gcc-4.3/lib64/libstdc++.so.6 (how do I tell?), but I can't seem to get it to use that one.

Any suggestions? I'm totally stuck. I must say the multi buildpack process is pretty messed up for this kind of thing.

Answer 1

Ok, I figured it out. It turns out that the correct library wasn't in /app/vendor/gcc-4.3/gcc-4.3/lib64/ after all, but just in /usr/lib. That explains why changing the order of the directories in LD_LIBRARY_PATH had no effect, since it wasn't in any of them.

There must be some logic that looks in the directories of LD_LIBRARY_PATH, and it doesn't find a match there, then looks in /usr/lib. That is why it works some of the time. The file in /app/vendor/R/lib64/R/lib/ must have been close enough to be considered a match (so it didn't look in /usr/lib), but in the wrong format so it caused a later error.

The fix is just to include /usr/lib in LD_LIBRARY_PATH. I added /usr/local/lib as well, for good measure. You should now use:

heroku config:set LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/app/vendor/R/lib64/R/modules:/app/vendor/R/lib64/R/lib:/app/vendor/gcc-4.3/lib64