Does this URL exist? RCurl says no

Question

The URL of interest is:

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=/netahtml/PTO/search-adv.htm&r=10&f=G&l=50&d=PTXT&OS=AN/(nortel)&RS=AN/nortel&Query=AN/(nortel)&Srch1=nortel.ASNM.&NextList1=Next 50 Hits

The chosen function to test its existence is:

> url.exists("http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=/netahtml/PTO/search-adv.htm&r=10&f=G&l=50&d=PTXT&OS=AN/(nortel)&RS=AN/nortel&Query=AN/(nortel)&Srch1=nortel.ASNM.&NextList1=Next 50 Hits")
[1] FALSE

Why dis no work? The URL clearly exists and resolves in chrome and using htmlTreeParse on the URL works just fine.

Answer 1

My guess would be that url.exists is using a HTTP HEAD-request, which the server seems unable to handle:

$ telnet patft.uspto.gov 80
Trying 151.207.240.26...
Connected to patft.uspto.gov.
Escape character is '^]'.
HEAD /netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=/netahtml/PTO/search-adv.htm&r=10&f=G&l=50&d=PTXT&OS=AN/(nortel)&RS=AN/nortel&Query=AN/(nortel)&Srch1=nortel.ASNM.&NextList1=Next+50+Hits HTTP/1.1
Host: patft.uspto.gov
Connection: close

Connection closed by foreign host.

So server broken, not RCurl.