why isn't Requests not signing into a website correctly?

Question

I am trying to sign into linkedin by using the requests library. After looking around the best way to do this is with using a requests.Session() I attempted to do this, but I was not successful. I believe it has something to do with the link I post to.

import requests

payload = {
    'session_key': EMAIL_GOES_HERE,
    'session_password': PASSWORD_GOES_HERE
}

with requests.Session() as s:
    s.post('https://www.linkedin.com/', data=payload)
#program should be signed in here so I am going onto a private page that requeires the user to be signed in.
r=s.get('https://www.linkedin.com/vsearch/p?f_CC=2289109')
#saving the results in an HTML file for easy debugging/viewing 
html= open('testtest.html', 'w')
html.write(r.content)
html.close()

Answer 1

I should start with stating that you really should use their API: http://developer.linkedin.com/apis

There does not seem to be any POST login on the frontpage of linkedin using those parameters?

This is the login URL you must POST to: https://www.linkedin.com/uas/login-submit

Be aware that this probably wont work either, as you need at least the csrfToken parameter from the login form.

You probably need the loginCsrfParam too, also from the login form on the frontpage.

Something like this might work. Not tested, you might need to add the other POST parameters.

import requests
s = requests.session()

def get_csrf_tokens():
    url = "https://www.linkedin.com/"
    req = s.get(url).text

    csrf_token = req.split('name="csrfToken" value=')[1].split('" id="')[0]
    login_csrf_token = req.split('name="loginCsrfParam" value="')[1].split('" id="')[0]

    return csrf_token, login_csrf_token


def login(username, password):
    url = "https://www.linkedin.com/uas/login-submit"
    csrfToken, loginCsrfParam = get_csrf_tokens()

    data = {
        'session_key': username,
        'session_password': password,
        'csrfToken': csrfToken,
        'loginCsrfParam': loginCsrfParams
    }

    req = s.post(url, data=data)

login('username', 'password')