Handling an invalid security certificate using MATLAB's urlread command

Question

I'm accessing an internal database using MATLAB's urlread command, everything was working fine until the service was moved to a secure server (i.e. with an HTTPS address rather than an HTTP address). Now urlread no longer successfully retrieves results. It gives an error:

Error downloading URL. Your network connection may be down or your proxy settings improperly configured.

I believe the problem is that the service is using an invalid digital certificate since if I try to access the resource directly in a web browser I get "untrusted connection" warning which I am able to pass through by adding the site to an Exception list. urlread doesn't have an obvious way of handling this problem.

Under the hood urlread is using Java to access web resources, and the error is thrown at this line:

inputStream = urlConnection.getInputStream;

where urlConnection is a Java object: sun.net.www.protocol.https.HttpsURLConnectionImpl.

Anyone suggest a workaround for this problem?

Answer 1

Consider the following Java class. I used this page as reference:

Disabling Certificate Validation in an HTTPS Connection

C:\MATLAB\MyJavaClasses\com\stackoverflow\Downloader.java

package com.stackoverflow;

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.cert.X509Certificate;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import javax.net.ssl.HostnameVerifier;

public class Downloader {
    public static String getData(String address) throws Exception {
        // Create a trust manager that does not validate certificate chains
        TrustManager[] trustAllCerts = new TrustManager[] {
            new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                    return null;
                }
                public void checkClientTrusted(X509Certificate[] certs, String authType) {
                }
                public void checkServerTrusted(X509Certificate[] certs, String authType) {
                }
            }
        };

        // Create a host name verifier that always passes
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        };

        // Install the all-trusting trust manager
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

        // Install the all-trusting host verifier
        HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

        // open connection
        URL page = new URL(address);
        HttpURLConnection conn = (HttpURLConnection) page.openConnection();
        BufferedReader buff = new BufferedReader(new InputStreamReader(conn.getInputStream()));

        // read text
        String line;
        StringBuffer text = new StringBuffer();
        while ( (line = buff.readLine()) != null ) {
            //System.out.println(line);
            text.append(line + "\n");
        }
        buff.close();

        return text.toString();
    }

    public static void main(String[] argv) throws Exception {
        String str = getData("https://expired.badssl.com/");
        System.out.println(str);
    }
}

MATLAB

First we compile the Java class (we must use a JDK version compatible with MATLAB):

>> version -java
>> system('javac C:\MATLAB\MyJavaClasses\com\stackoverflow\Downloader.java');

Next we instantiate and use it MATLAB as:

javaaddpath('C:\MATLAB\MyJavaClasses')
dl = com.stackoverflow.Downloader;
str = char(dl.getData('https://expired.badssl.com/'));
web(['text://' str], '-new')

Here are a few URLs with bad SSL certificates to test:

urls = {
    'https://expired.badssl.com/'       % expired
    'https://wrong.host.badssl.com/'    % wrong host
    'https://self-signed.badssl.com/'   % self-signed
    'https://revoked.grc.com/'          % revoked
};

---

UPDATE: I should mention that starting with R2014b, MATLAB has a new function webread that supersedes urlread.

Answer 2

thanks for the solution. It worked, however, sometimes, I had received the following exception "java.io.IOException: The issuer can not be found in the trusted CA list." and I was not able to get rid of this error.

Therefore, I tried an alternative solution that works well. You can use the following Java code in Matlab function:

 function str = ReadUrl(url)
     is = java.net.URL([], url, sun.net.www.protocol.https.Handler).openConnection().getInputStream(); 
     br = java.io.BufferedReader(java.io.InputStreamReader(is));
     str = char(br.readLine());
 end

Best, Jan

Answer 3

Note also that the "canonical" way to solve this issue is to import the certificate into MATLAB's keystore (i.e., not your JVM's keystore).

This is documented here: Mathworks on using untrusted SSL certificates.