$ brew reinstall jpeg
solved the problem for me. After this I could reinstall gnuplot with
brew reinstall gnuplot --cairo --with-x --tests
I assume other reinstall options will work too. Seems like bug in the brew recipe for gunplot. At least I expect a better error message.