It appears on some devices, with some providers, the TYPE_MOBILE isn't the default MMS gateway, you have to use another one. Here is the solution I found that solved my problem.
I hope this could help someone else.
// Take a wake lock so we don't fall asleep before the message is downloaded.
createWakeLock();
// Let's try every type
int result = -1;
int[] apnTypes = new int[] {ConnectivityManager.TYPE_MOBILE, ConnectivityManager.TYPE_MOBILE_MMS, ConnectivityManager.TYPE_MOBILE_DUN, ConnectivityManager.TYPE_MOBILE_HIPRI, ConnectivityManager.TYPE_MOBILE_SUPL};
for (int i=0; i<apnTypes.length; i++)
{
result = mConnMgr.startUsingNetworkFeature(apnTypes[i], PhoneEx.FEATURE_ENABLE_MMS);
Log.v(TAG, "beginMmsConnectivity: result=" + result);
switch (result)
{
case PhoneEx.APN_ALREADY_ACTIVE:
case PhoneEx.APN_REQUEST_STARTED:
acquireWakeLock();
return result;
}
}
// None found
throw new IOException("Cannot establish MMS connectivity");