Java httpget seems to work but encountering 401 error

Question

HAve re-written due to the feedback so far...

I'ev got a Java method to try and download a file via httpget within a selenium script, it looks like so...

 private String downloader(WebElement element, String attribute, String filePath) throws IOException, NullPointerException, URISyntaxException {      
        String fileToDownloadLocation = element.getAttribute(attribute);
        fileToDownloadLocation = fileToDownloadLocation.replace("\\", "%5C");
        URL fileToDownload = new URL(fileToDownloadLocation);
        URI FileDL = new URI(fileToDownload.toString());


        File downloadedFile = new File(this.localDownloadPath + filePath);
        if (downloadedFile.canWrite() == false) downloadedFile.setWritable(true);

        CloseableHttpClient httpclient = HttpClients.createDefault();

        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(AuthScope.ANY,new NTCredentials("useraccount", "testpassword", "machinename", "mydomain"));

        HttpClientContext context = HttpClientContext.create();
        context.setCredentialsProvider(credsProvider);

        HttpGet httpget = new HttpGet(FileDL);        

        CloseableHttpResponse response = httpclient.execute(httpget, context);

        this.httpStatusOfLastDownloadAttempt = response.getStatusLine().getStatusCode();

        FileUtils.copyInputStreamToFile(response.getEntity().getContent(), downloadedFile);
        response.getEntity().getContent().close();

        String downloadedFileAbsolutePath = downloadedFile.getAbsolutePath();
        return downloadedFileAbsolutePath;
}

When I use this on a regular link (i.e. downloading a file from the bbc for example) it all works. The trouble is the system I'm using it on is an internal one that uses windows authentication to determine if access is permitted. When I attempt to use it on this internal system my Http response ends up reading: -

HTTP/1.1 401 Unauthorized [Cache-Control: no-cache, no-store, must-revalidate, Pragma: no-cache, Content-Type: text/html, Expires: -1, Server: Microsoft-IIS/7.5, WWW-Authenticate: Negotiate, WWW-Authenticate: NTLM, X-Powered-By: ASP.NET, X-UA-Compatible: IE=edge, Date: Mon, 17 Mar 2014 14:52:43 GMT, Content-Length: 1293]

If I debug this and obtain the URL it uses I can manually paste this into a browser and it works fine.

I also noticed that during debug the httpRequestParameters doesn't seem to have any values in in at all (shows as null) so I'm guessing that I'm still not setting the account parameters properly somehow.

I'll be honest and say I'm really not 100% on what all this does properly and am trying to paste things together and get it to play ball but I'm wondering if the section where I have ((AbstractHttpClient) client).getCredentialsProvider(). is set up properly or if I'm missing something here...

Any further help much appreciated!

Answer 1

I've finally managed to get my code working, the final version is shown above. I'd made a few obvious errors such as my domain was still set to the examples "microsoft.com" and I had actually misspelled my password, other than that it now does exactly what I need it to! :)

@vzamanillo thanks for your help!

Answer 2

You can set the BASIC authentication credentials in two different ways:

Setting the appropiated HTTP header:

HttpClient hc = new DefaultHttpClient();

HttpGet get = new HttpGet("http://...");

String basic_auth = new String(Base64.encodeBase64((username + ":" + password).getBytes()));
get.addHeader("Authorization", "Basic " + basic_auth);

or using the CredentialsProvider (depends on the Apache HttpComponents version)

HttpClient hc = new DefaultHttpClient();

hc.getCredentialsProvider().setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("username", "password"));

HttpGet get = new HttpGet ("http://...");       

hc.execute(get);