I want to upload and submit files to the form on http://www.broadinstitute.org/cmap/newQuery?servletAction=querySetup&queryType=quick
(need log-in). The form looks as follows
<form name="form1" enctype="multipart/form-data" method="post" action="newQuery">
<input type="hidden" name="servletAction" value="quickQuery">
<div class="formTable">
<div class="row">
<span class="label" style="width: 100px;">up tag file:</span>
<span class="field"><input type="file" name="ups" size="30" accept="grp"> <font size="-1"><a href="#" onClick="window.open('help_topics_frames.jsp?topic=tag list', 'helpTopicsWindow', 'scrollbars,resizable,height=600,width=700')">tag file help</a></font></span>
</div>
<div class="row">
<span class="label" style="width: 100px;">down tag file:</span>
<span class="field"><input type="file" name="dns" size="30" accept="grp"></span>
</div>
<div class="row">
<span class="label" style="width: 100px;"> </span>
<span class="navigation"><input type="button" onClick="submitForm()" name="submitButton" value="execute query"></span>
</div>
</div>
</form>
First, from the answer https://stackoverflow.com/a/22547541/651779 I get the cookie with the log-in credential
import http.cookiejar
import urllib
from bs4 import BeautifulSoup
import requests
submit_signature_url = 'http://www.broadinstitute.org/cmap/newQuery?servletAction=querySetup&queryType=quick'
login_url = 'http://www.broadinstitute.org/cmap/j_security_check'
login_values = urllib.parse.urlencode({'j_username': 'example',
'j_password': 'example',
'submit': 'sign in'})
payload_submit_signature = bytes(login_values, 'ascii')
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(
urllib.request.HTTPRedirectHandler(),
urllib.request.HTTPHandler(debuglevel=0),
urllib.request.HTTPSHandler(debuglevel=0),
urllib.request.HTTPCookieProcessor(cj))
resp_1 = opener.open(submit_signature_url) #First call to capture the JSESSIONID
resp = opener.open(login_url, payload_submit_signature)
This works correctly. Now I want to post the files to the form. I tried using this answer https://stackoverflow.com/a/12385661/651779
# changed after Brett Lempereur's answer
values = {'ups':open(r'path\to\up.grp','rb'),
'dns':open(r'path\to\down.grp','rb')}
submit_signature_url = 'http://www.broadinstitute.org/cmap/newQuery?servletAction=querySetup&queryType=quick'
req = requests.post(submit_signature_url, files=values, cookies=cj)
soup = BeautifulSoup(req.text)
print(soup.prettify())
This prints
Requested URL: http://www.broadinstitute.org/cmap/newQuery
java.lang.NullPointerException
If you are logged in and go to http://www.broadinstitute.org/cmap/newQuery in your browser you get the same. If I instead use data
instead of files
in the requests.post: req = requests.post(submit_signature_url, data=values, cookies=cj)
, it prints the html of the page containing the form, so it did not post the form.
How can I post to multipart/form-data?
For an example up-file, copy the following into a file a file and call it up.grp
205258_at
221369_at
205751_at
212954_at
219914_at
206703_at
207352_s_at
203548_s_at
203549_s_at
210382_at
212110_at
213805_at
213935_at
218739_at
219737_s_at
219738_s_at
204131_s_at
204132_s_at
210655_s_at
217399_s_at
206791_s_at
206792_x_at
211818_s_at
221523_s_at
221524_s_at
And for an example down file, copy the following in a file and call it down.grp
204725_s_at
211063_s_at
219921_s_at
200618_at
219701_at
220342_x_at
220926_s_at
201323_at
204692_at
221956_at
222017_x_at
90610_at
217755_at
207843_x_at
209366_x_at
215726_s_at
201827_at
201185_at
212411_at
201692_at
214484_s_at
202910_s_at
202381_at
200663_at
201459_at
219770_at
220853_at
201349_at
207015_s_at
207016_s_at
212338_at
220330_s_at