Python POST form login on website

StackOverflow https://stackoverflow.com//questions/22031167

  •  21-12-2019
  •  | 
  •  

Question

I'm trying to get this login form working in Python with a work site, so I can do some scraping and whatnot with the site data.

Now, when I print the contents of r.text it's the same HTML as the login page. No progress is made into the site.

It's also worth mentioning that I'm indifferent about using Requests here. I just find the methods and such from urllib to be ugly, and most info I've found on this site has leaned towards using Requests for the above task.

Below is the information I'm working with.

enter image description here

<form action="login.cgi" autocomplete="off" id="frmLogin_4" method="post" name="frmLogin" onsubmit="return Login(1)">
  <input id="tz_offset_5" name="tz_offset" type="hidden">
  <table border="0" cellpadding="2" cellspacing="0" id="table_LoginPage_3">
    <tr>
      <td valign="top">
        <input id="realm_16" name="realm" type="hidden" value="All Users">

        <table border="0" cellpadding="2" cellspacing="0" id="table_LoginPage_6">
          <tr>
            <td>E-Mail Address</td>

            <td>&nbsp;</td>

            <td><input id="username_5" name="username" size="20" type="text"></td>
          </tr>

          <tr>
            <td>Password</td>

            <td>&nbsp;</td>

            <td><input id="password_5" name="password" size="20" type="password"></td>
          </tr>

          <tr>
            <td></td>
          </tr>

          <tr>
            <td colspan="3">&nbsp;</td>
          </tr>

          <tr>
            <td>&nbsp;</td>

            <td>&nbsp;</td>

            <td><input id="btnSubmit_6" name="btnSubmit" type="submit" value="Sign In">&nbsp;</td>
          </tr>
        </table>
      </td>
    </tr>
  </table>
</form>

And below is the Python I'm using at this point.

import requests

url = 'https://<scrubbed>/dana-na/auth/url_default/welcome.cgi'

payload = {
    'username':'<scrubbed>',
    'tz_offset':'-480',
    'realm':'All Users',
    'password':'<scrubbed>',
    'btnSubmit':'Sign In'
}    

session = requests.session()    
r = requests.post(url, payload)
print r.text
Was it helpful?

Solution

Use the data keyword. From the documentation:

Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made:

payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.post("http://httpbin.org/post", data=payload)
print r.text

Additionally, the HTML you posted suggests the form will actually go to login.cgi and not welcome.cgi. You can see this in the screenshot as well.

Use:

url = 'https://<scrubbed>/dana-na/auth/url_default/login.cgi'
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top