SAS PROC SOAP to update Sharepoint List

Question

I am trying to update a Sharepoint (2007) list using SAS (9.3) via PROC SOAP (SAS is living on a Unix GRID). The sharepoint site has basic authentication enabled (since PROC SOAP cannot authenticate through NTLM).

I can successfully pull data down from the Sharepoint list using the XML Libname engine, but I cannot issue via PROC SOAP XML data back to the Sharepoint lists web service. Specifically, I am trying to add and update items within the sharepoint list through http://[intranet_site]/sites/IT/_vti_bin/Lists.asmx

SAS LOG Output :

18399  %let RESPONSE=RESPONSE;
18400  proc soap in=REQUEST
18401            out=&RESPONSE
18402            url="http://[intranet_site]/sites/IT/_vti_bin/Lists.asmx"
18403      webusername="[username]"
18404      webpassword="[password]"
18405      webdomain="[domain]"
18406      SOAPACTION="http://schemas.microsoft.com/sharepoint/soap/UpdateListItems"
18407            ;
18408  run;

ERROR: org.springframework.ws.client.WebServiceTransportException: Unauthorized [401]

I have confirmed, through SOAPUI, that the XML passing into the Lists.asmx web service is valid (I can actually create and update list items as expected in Sharepoint when manually executing through SOAPUI.

As the error explicitly states, for some reason, the user authentication being fed into the PROC SOAP is not making it into Sharepoint (I have administrative rights in Sharepoint so I should have the correct rights). What is super confusing about this is that I can feed in the same credentials through XML Libname and pull back data just fine...

** Questions **

1. If I am passing in my credentials through PROC SOAP, why am I not able to authenticate to Sharepoint?
2. Are there any other work around for passing this XML into Sharepoint (IE: is XML LIBNAME or PROC HTTP able to support this by passing in the XML)?
3. Where does the issue lie : in SAS with PROC SOAP (in the way I am invoking the procedure) or with Sharepoint?

...just for completeness, example XML being passed into PROC SOAP can be seen posted below (SOAP 1.1 - which should be supported by PROC SOAP):

<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <UpdateListItems xmlns="http://schemas.microsoft.com/sharepoint/soap/">
          <listName>{851DDBB5-1765-444D-9012-0210F006A4AF}</listName>
          <updates>
          <Batch OnError="Continue" ListVersion="1">
              <Method ID="1" Cmd="New">
                  <Field Name='ID'>NEW</Field>
                  <Field Name='Title'>DUMMY</Field>
                   [...shortened for space...]
              </Method>
          </Batch>
          </updates>
      </UpdateListItems>
   </soap:Body>
</soap:Envelope>

Answer 1

I gave up on trying to utilize SAS PROC SOAP functionality to interact with SharePoint. NTLM Authentication was too large of a hurdle to overcome. I was looking at a pretty bleak future manually executing the SOAP XML through SOAPUI several times a day indefinitely... SAS is currently lacking when it comes to NTLM authentication functionality (based on discussions with SAS support, there are plans to build in this functionality, but it is still being tested at this point).

Then, I found a solution: cURL !!!
More information about curl found here : http://curl.haxx.se/

Since I am using SAS EG in a Linux environment and CURL is installed on the server, I am able to leverage SAS X Command to pipe the curl commands into the console. Coupling this with sleep, allows curl time to pass the SOAP XML to the SharePoint web service and capture the response. After this, I then pull the response file to parse it for errors. If an error is found, I email a group of supporting users to notify them of the error to being debugging. So far it appears to be working fairly well.

Sample code found below:

DATA _NULL_;
x "cd /prg/SOAP_Request";  /*Navigate to where my XML is stored */
x "curl -X POST -k --negotiate -u 'MyUsername':MyPassword --digest --ntlm -H ""SOAPAction: ""http://schemas.microsoft.com/sharepoint/soap/UpdateListItems"" "" -H ""Content-Type: text/xml; charset=UTF-8"" -d @request_&CLIENTID._&Exets..xml  http://Shrpnt/sites/IT/_vti_bin/Lists.asmx > /prg/SOAP_Response/response_&CLIENTID._&Exets..xml"; /* issue the CURL command */
x=sleep(20,1); /* Have SAS sleep for 20 seconds to allow CURL to process */
run;

(To test out the CURL command, I was using Putty to just ensure that I could successfully issue the HTTP POST and receive/capture the response. Once this was done, I converted the code over so that it could dynamically pull in the saved XML file and save the response using the same methodology.

Hope this helps everyone that runs into this situation. I can now have SAS interact and drive SharePoint Lists!

Answer 2

Ad 1) I assume test with SOAPUI is Windows to Windows test, your situation with SAS is Unix to Windows, that's probably where the issue lies. I'd recommend to contact SAS support with your question, they should have tried already... Also I'd recommend to get some SAS on Windows and try the stuff. From what I searched on unix and sharepoint, the authentication seems to be a source of issues, so it's not SAS specific.

Ad 2) XML libname is actully for reading and writing static xml files - no request, response stuff. PROC HTTP can't do SOAP.

Ad 3) I bet, the issue is with basic authentication vs Unix.