Posting a username and password with RCurl

StackOverflow https://stackoverflow.com/questions/22531398

  •  18-06-2023
  •  | 
  •  

Question

I am trying to use R to log into this website: http://www.broadinstitute.org/cmap/signature?servletAction=edit with the following code

library("RCurl")
connectivityMap_detailed_results <- function(up_grp, down_grp){
params <- list('j_username' = 'username',
               'j_password' = 'password')
cmap <- postForm("http://www.broadinstitute.org/cmap/signature?servletAction=edit",
                                                     .params=params, style="POST")

print(cmap)

However, print(cmap) still shows the log-in page.

I also tried 

tmp <- getURL("http://www.broadinstitute.org/cmap/signature?servletAction=edit",
              userpwd = "user:pwd",verbose=TRUE)

with the same result.

How can I use RCurl to log in on that website?

cmap result

[1] "
<html>
<head>
<link href="styles/stylesheet.css" rel="stylesheet" type="text/css">
<link rel="SHORTCUT ICON" href="favicon.ico" >
<title>Connectivity Map sign-in</title>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body>



<div style="overflow: hidden;"><a href="displayServlet?servletAction=mainMenu"><img border="0" src="images/banner1.jpg" alt="cmap" title="cmap"></a></div>

<div align="right" style="text-align: right;" width="100%">

         </div>

        <div id="tabContainer" class="tabContainer"><span id="tabSpace" class="tabSpace">&nbsp;</span></div>
<div class="error"></div>

<br><br>

<form name="login2" target="_top" method="POST" action="j_security_check">
<table>
        <tr>
                <td align="right" valign="top">
                        username:
                </td>
                <td valign="top">
                        <input name="j_username" size="50" value="">
                </td>
        </tr>
        <tr>
                <td align="right" valign="top">
                        password:
                </td>
                <td valign="top">
                        <input type="password" name="j_password" size="50">
                </td>
        </tr>
        <tr>
                <td align="right" valign="top">
                </td>
                <td valign="top">
                        <input type="submit" name="submit" value="sign in" class="little">&nbsp;&nbsp;&nbsp;&nbsp;
                        <br/><br/>
                        <a href="sendPassword.jsp">email me my password</a> | 
                        <a href="userServlet?servletAction=editProfile&userId=0">register as a new user</a>

                </td>
        </tr>
</table>
</form>
<hr/>
<p class="intro">The Connectivity Map (also known as cmap) is a collection of genome-wide transcriptional expression data from cultured human cells treated with bioactive small molecules and simple pattern-matching algorithms that together enable the discovery of functional connections between drugs, genes and diseases through the transitory feature of common gene-expression changes. You can learn more about cmap from our papers in <a target="_blank" href="http://www.sciencemag.org/cgi/content/full/313/5795/1929?ijkey=ylc.v/5sDLGKI&keytype=ref&siteid=sci"><i>Science</i></a> and <a target="_blank" href="http://www.nature.com/nrc/journal/v7/n1/pdf/nrc2044.pdf"><i>Nature Reviews Cancer</i></a>.
<!--table>
<tr><td align="right"><img src="images/scienceMag.png" height="25" style="padding-left: 25px;"></td><td>    read our paper in <a href="http://www.sciencemag.org/cgi/content/full/313/5795/1929?ijkey=ylc.v/5sDLGKI&keytype=ref&siteid=sci">Science Magazine</a><br/>&nbsp;</td></tr>
<tr><td align="right"><img src="http://www.broad.mit.edu/images/homelogo.gif" height="25" style="padding-left: 25px;"></td><td> an <a href="http://www.broad.mit.edu/cgi-bin/news/display_news.cgi?id=2141">article about cmap</a> at The Broad Institute website<br/>&nbsp;</td></tr>
<tr><td align="right"><img src="images/audio.gif" style="padding-left: 25px;"></td><td> <a href="http://www.npr.org/templates/story/story.php?storyId=6167294&ft=1&f=1027">listen to an interview</a> on National Public Radio with Todd Golub about cmap<br/>&nbsp;</td></tr>
</table-->
</p>
<p class="intro">This web interface provides access to the current version (<b>build 02</b>) of Connectivity Map which contains more than 7,000 expression profiles representing 1,309 compounds. It is designed to allow biologists, pharmacologists, chemists and clinical scientists to use cmap without the need for any specialist ability in the analysis of gene-expression data. The previous version (<b>build 01</b>) of Connectivity Map can be accessed <a href="http://www.broad.mit.edu/cmap_build01">here</a>.</p>
<p class="intro">A brief tutorial can be found by clicking 'getting started' under the 'help' tab after log in. Detailed help and a definition of cmap terms can be found by clicking 'topics', also under the 'help' tab. For everything else, please <a href="mailto:cmap-help@broad.mit.edu">contact us</a>.</p>
<p class="intro">The Connectivity Map is based at The Broad Institute of MIT and Harvard in Cambridge, Massachusetts. The cmap team is Justin Lamb, Xiaodong Lu, Dave Peck, Matt Wrobel, Aravind Subramanian, Irene Blat, Josh Modell, Jim Lerner, Elizabeth Liu and Emily Crawford. Jean-Philippe Brunet, Ken Ross, Michael Reich, Paul Clemons, Kathy Seiler, Steve Haggarty, Bang Wong, Maria Nemchuk, Ru Wei, Steve Carr, Christopher Johnson, Stephen Johnson, the MSigDB curation team, and the Genetic Analysis Platform contribute invaluable expertise and assistance. Todd Golub and Eric Lander provide institutional leadership for the project.</p>
<hr/>
<a href="privacy.jsp">privacy statement</a> | <a href="terms_and_conditions.jsp">terms and conditions</a><br/>
<a href="http://www.broad.mit.edu"><img src="images/homelogo.gif" border="0"/></a>
<br/>&copy; 2006 Broad Institute
<!--p class="intro">The Broad Institute is a research collaboration of MIT, Harvard and its affiliated Hospitals, and the Whitehead Institute, created to bring the power of genomics to medicine.</p-->
<script language="javascript">
document.login2.j_username.focus()
</script>
</body>
</html>
"
attr(,"Content-Type")
                  charset 
 "text/html" "ISO-8859-1"
Was it helpful?

Solution

You're using the wrong url. To submit the form you have to post to the url specified in the action=... attribute of the <form> element.

library(RCurl)
url <- "http://www.broadinstitute.org/cmap/j_security_check"
params <- list('j_username' = 'username',
               'j_password' = 'password')
cmap <- postForm(url, .params=params, style="POST")

library(XML)
doc <- htmlTreeParse(cmap, useInternalNodes=T)
xmlValue(doc["//head/title"][[1]])   # page title
# [1] "login failure"

The login fails of course because I don't know the username and password.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top