OK, you have to use:
WebSettings settings = webView.getSettings();
settings.setLoadWithOverviewMode(true);
settings.setUseWideViewPort(true);
But also, it breaks if you set user-scalable=0 like this:
<meta name='viewport' content='width=640, user-scalable=0'/>