Using CookieJar in Python to log in to a website from “Google App Engine”. What's wrong here?

Question

I've been trying to find a python code that would log in to my mail box on yahoo.com from "Google App Engine" . Here (click here to see that page) I was given this code:

import urllib, urllib2, cookielib

url = "https://login.yahoo.com/config/login?"
form_data = {'login' : 'my-login-here', 'passwd' : 'my-password-here'}

jar = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
form_data = urllib.urlencode(form_data)
# data returned from this pages contains redirection
resp = opener.open(url, form_data)
# yahoo redirects to http://my.yahoo.com, so lets go there instead
resp = opener.open('http://mail.yahoo.com')
print resp.read()

The author of this script looked into HTML script of yahoo log-in form and came up with this script.

That log-in form contains two fields, one for users' Yahoo! ID and another one is for users' password. Here is how HTML code of that page for both of those fields looks like:

User ID field:

<input type="text" maxlength="96" class="yreg_ipt" size="17" value="" id="username" name="login">

Password field:

<input type="password" maxlength="64" class="yreg_ipt" size="17" value="" id="passwd" name="passwd">

However, when I uploaded this code to Google App Engine I discovered that this log-in form keeps coming back to me, which, I assume, means that logging-in process didn't succeed. Why is it so?

Answer 1

You send MD5 hash and not plain password. Also you'd have to play along with all kinds of CSRF protections etc. that they're implementing. Look:

            <input type="hidden" name=".tries" value="1"> 
            <input type="hidden" name=".src" value="ym"> 
            <input type="hidden" name=".md5" value=""> 
            <input type="hidden" name=".hash" value=""> 
            <input type="hidden" name=".js" value=""> 
            <input type="hidden" name=".last" value=""> 
            <input type="hidden" name="promo" value=""> 
            <input type="hidden" name=".intl" value="us"> 
            <input type="hidden" name=".bypass" value=""> 
            <input type="hidden" name=".partner" value=""> 
            <input type="hidden" name=".u" value="bd5tdpd5rf2pg"> 
            <input type="hidden" name=".v" value="0"> 
            <input type="hidden" name=".challenge" value="5qUiIPGVFzRZ2BHhvtdGXoehfiOj"> 
            <input type="hidden" name=".yplus" value=""> 
            <input type="hidden" name=".emailCode" value=""> 
            <input type="hidden" name="pkg" value=""> 
            <input type="hidden" name="stepid" value=""> 
            <input type="hidden" name=".ev" value=""> 
            <input type="hidden" name="hasMsgr" value="0"> 
            <input type="hidden" name=".chkP" value="Y"> 
            <input type="hidden" name=".done" value="http://mail.yahoo.com"> 
            <input type="hidden" name=".pd" value="ym_ver=0&c=&ivt=&sg="> 

Launch Wireshark and play with it. Good luck :)

However if you intend to use it w/ App Engine keep in my mind that using Google IP will almost surely result w/ Captcha challenge. Also Yahoo might block your User-Agent that is being set permanently by Google.