401 Unauthorised errors when attempting to download ASP page to file

Question

Issue

Msxml2.ServerXMLHTTP keeps returning 401 - Unauthorised errors each time we attempt to read the contents of a file (ASP) from a web server.

Source server is running IIS6, using NTLM integrated login.

This process has been used successfully before, but only in as far as extracting XML files from external websites, not internal ones.

The proxy settings in the registry of the server on which the script is run has also been updated to bypass the website in question, but to no avail.

All paths identified in the VBScript have been checked and tested, and are correct.

User running the script has correct read/write permissions for all locations referenced in the script.

Solution needed

To identify the cause of the HTTP 401 Unauthorised messages, so that the script will work as intended.

Description

Our organisation operates an intranet, where the content is replicated to servers at each of our remote sites. This ensures these sites have continued fast access to important information, documentation and data, even in the event of losing connectivity.

We are in the middle of improving the listing and management of Forms (those pesky pieces of paper that have to be filled in for specific tasks). This involves establising a database of all our forms.

However, as the organisation hasn't been smart enough to invest in MSSQL Server instances at each site, replication of the database and accessing it from the local SQL server isn't an option.

To work around this, I have constructed a series of views (ASP pages) which display the required data. I then intend to use Msxml2.ServerXMLHTTP by VBScript, so I can read the resulting pages and save the output to a static file back on the server.

From there, the existing replication process can stream these files out to the site - with users having no idea that they're looking at a static page that just happened to be generated from database output.

Code

' Forms - Static Page Generator
' Implimented 2011-02-15 by Michael Harris

' Purpose: To download the contents of a page, and save that page to a static file.

' Target category: 1 (Contracts)
' Target Page:
' http://sharename.fpc.wa.gov.au/corporate/forms/generator/index.asp
' Target path: \\servername\sharename\corporate\forms\index.asp
' Resulting URL: http://sharename.fpc.wa.gov.au/corporate/forms/index.asp

' Remove read only

' Remove read only flag on file if present to allow editing
' If file has been set to read only by automated process, turn off read only


Const READ_ONLY = 1

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile("\\server\sharename\corporate\forms\index.asp")

If objFile.Attributes AND READ_ONLY Then
    objFile.Attributes = objFile.Attributes XOR READ_ONLY
End If

Dim webObj, strURL
Set webObj = CreateObject("Msxml2.ServerXMLHTTP")
strURL = "http://sharename.fpc.wa.gov.au/corporate/forms/generator/index.asp"

webObj.Open "GET", strURL
webObj.send

If webObj.Status=200 Then
Set objFso = CreateObject("Scripting.FileSystemObject")
Set txtFile = objFso.OpenTextFile("file:\\servername.fpc.wa.gov.au\sharename\corporate\forms\index.asp", 2, True)
txtFile.WriteLine webObj.responseText
txtFile.close

ElseIf webObj.Status >= 400 And webObj.Status <= 599 Then
  MsgBox "Error Occurred : " & webObj.Status & " - " & webObj.statusText
Else
  MsgBox webObj.ResponseText
End If

Answer 1

Replace your line:

webObj.Open "GET", strURL 

With:

webObj.Open "GET", strURL, False, "username", "password"

In most cases 401 Unauthorized means you haven't supplied credentials. Also you should specifiy False to indicate you don't want async mode.

Answer 2

I would first test if you can reach your url through a normal browser on the same server X you run your code on (A). I would try then reach the url from another PC. One never used to reach that url but in the same network as server X (B).

If B works but A doesn't I would suspect that for some reason your source server (i.e. that one that serves the url) blocks server X for some reason. Check the security settings of II6 and of NTLM.

If both A and B don't work, there is something wrong more in general with your source server (i.e. it blocks everything or NTML doesn't allow you in).

If A works (B doesn't matter then), the problem has to be somewhere in your code. In that case, I would recommend fiddler. This tool can give you the HTTP requests of both your browser and your code in realtime. You can then compare both. That should give you at least a very strong hint about (if not immediately give you) the solution.

Answer 3

It sounds like the O.P. got this working with the correct proxy settings in the registry (http://support.microsoft.com/kb/291008 explains why proxy configuration will fix this). Newer versions of ServerXMLHTTP have a setProxy method that can be used to set the necessary proxy configuration in your code instead.

In the O.P. code above, after webObj is created, the following line of code would set up the proxy correctly:

webObj.setProxy 2, "0.0.0.0:80", "*.fpc.wa.gov.au" 

ServerXMLHTTP will pass on the credentials of the user running the code if it is configured with a proxy, and if the target URL bypasses that proxy. Since you are bypassing the proxy anyway, you can make it a dummy value "0.0.0.0:80", and make sure your target url is covered by what you specify in the bypass list "*.fpc.wa.gov.au"