How to stay logged into a website using a Python script (requests library)?

https://stackoverflow.com/questions/21820104

12-10-2022
|

문제

I am attempting to use the Python requests library to login to a website called surfline.com, then get a webpage once logged in (presumably within a persisting session).

The login form that surfline.com uses throughout its pages uses onclick() to map to a JS function called verifyLogin(), which in turn posts to a .cfm file. On success, it refreshes the page, and the user is now kept logged in so long as the cookies persist.

I am using the requests library for the first time, and am unsure how to:

login successfully,
stay logged-in throughout the session,
then print out the homepage as a logged-in user to the terminal.

Here is the login form (HTML):

<form id="loginForm">
    <label for="name">Email Address:</label><br>
    <input type="text" name="username" id="username">
    <label for="mail">Password:</label><br>
    <input type="password" name="password" id="password">
    <input type="hidden" name="top_login" id="top_login" value="true">
    <button class="surfline-button blue1" type="button" onclick="verifyLogin();">Log In</button>
    <button class="surfline-button grey1" type="button" onclick="jQuery('#dialog-login').dialog('close');">Cancel</button>
    <div id="forgot">Forgot Password? <a href="/myaccount/?action=forgot_password">Click Here</a></div>
    <div class="clear"></div>
    <div><input style="float:left; width:18px; height:18px; margin-right:6px; border:none" type="checkbox" name="rememberMe" id="rememberMe" value="true" checked="checked"><div id="remember-me-text" style="float:left; width:200px; text-align:left; margin-top:1px;">Remember me on this computer?</div></div>
    <div class="clear"></div>
    <p>Not a Premium member?  <a href="https://www.surfline.com/subscribe_vindicia/index.cfm?mkt=login&amp;slintcid=LOGIN&amp;slcmpname=LOGIN-MODAL">TRY PREMIUM FREE NOW</a></p>
</form>

Here is the verifyLogin() function in the /05222013_slmenu.js file:

function verifyLogin(){  
    //var username = jQuery("#username").val();
    //var password = jQuery("#password").val();
    var usernameVal = jQuery("#username").val();
    var passwordVal = jQuery("#password").val();
    var rememberMeVal = jQuery("#rememberMe").val();    
    var top_loginVal = jQuery("#top_login").val(); 

    if(usernameVal.length === 0 || passwordVal.length === 0 ){
        jQuery("#login-note").addClass("warning");
        if(usernameVal.length === 0){ jQuery("#username").addClass('warning'); jQuery("#login-note").html("Email Field is Blank"); }else{ jQuery("#username").removeClass('warning');}
        if(passwordVal.length === 0){ jQuery("#password").addClass('warning'); jQuery("#login-note").html("Password Field is Blank"); }else{ jQuery("#password").removeClass('warning'); }
        if(usernameVal.length === 0 && passwordVal.length === 0){jQuery("#login-note").html("Email and Password Fields are Blank"); }
    }else{  
        jQuery("#inner-dialog").fadeOut("slow",function(){
            var htmlData = "<center><div style='padding-top:60px;'><h1>Verifying Login</h1><img src='/global_includes/images/ajax-loader-snake-295284.gif' style='margin-top:24px;'></div></center>";                                                
            jQuery("#verifying").html(htmlData).fadeIn("slow", function(){ 
                //var loginData = jQuery('#loginForm').serialize();

                var loginData = 'username=' + escape(usernameVal) + '&password=' + escape(passwordVal) + '&rememberMe=' + rememberMeVal + '&top_login=' + top_loginVal;

                jQuery.ajax({
                    type:'POST', 
                    url: '/myaccount/inc_login_handler.cfm', 
                    data:loginData, 
                    cache:false,
                    success: function(response){
                        var responseTrimmed = response.replace(/^\s+||\s+$/g,'');
                        if(responseTrimmed != true){    
                             jQuery("#verifying").fadeOut("slow",function(){ jQuery("#inner-dialog").fadeIn("slow"); jQuery("#login-note").html("Unable to find your login information. Please Try Again...").addClass("warning");  jQuery("#username").addClass('warning');  jQuery("#password").addClass('warning'); });
                        }else{ 
                            var successData = "<center><div style='padding-top:60px;'><h1>Success!</h1><img src='/global_includes/images/checkmark.png' style='margin-top:24px; margin-bottom:20px;'><br /> Please Wait, Site Reloading...</div></center>";
                            jQuery("#verifying").fadeOut("slow",function(){  jQuery('#verifying').html(successData).fadeIn("slow"); });
                            setTimeout(function(){ window.location.reload(); }, 1200 ); 
                        }
                    },
                    error:function (xhr, ajaxOptions, thrownError){
                        jQuery("#verifying").fadeOut("slow",function(){ jQuery("#inner-dialog").fadeIn("slow"); jQuery("#login-note").html("There was an error finding your login infomation. Please Try Again...").addClass("warning");  });
                    }  
                });
            });
        })
    }
}

I've tried this code, but can't seem to get the homepage as a logged-in user; it still returns as it would to someone who isn't logged in (my full name should appear in the header, etc.):

>>> import requests, json
>>> s = requests.Session()
>>> page_signed_out = s.get('http://www.surfline.com/home/index.cfm')
>>> form_data = {'type':'POST',
...              'url':'/myaccount/inc_login_handler.cfm',               
...              'data':"username=myemail@example.com&password=mypassword&rememberMe=true&top_login=true",
...              'cache':False}
>>> s.post(url,
...        data=json.dumps(form_data),
...        headers= {'content-type': 'application/json'})
<Response [200]>
>>> page_signed_in = s.get('http://www.surfline.com/home/index.cfm')

Basically, how can I log into surfline.com from a python file, and get a page as a logged in user? I don't mind using a different library, if it is not possible with the requests library. Thank you.

EDIT: Here are the cookies after a POST has been made, as suggested by @André

<[Cookie(version=0, name='CRYPTOPASS', value='%296%25%2A%2521B%3FO%2A%3C%28', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CRYPTOUSER', value='23%252600%2E3Z%5FM%5EEZV1IK%27TFIW%3E', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='LOGGED_OUT', value='true', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='USER_ID', value='259829', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CFID', value='437349255', port=None, port_specified=False, domain='www.surfline.com', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CFTOKEN', value='1082d1233da237c-3E4C2D43-FFC6-4ECC-1E80D9B505E495CE', port=None, port_specified=False, domain='www.surfline.com', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False)]>

해결책

I've tried logging in to that website while monitoring the requests made using Firebug, and it looks like data is in application/x-www-form-urlencoded format rather than application/json, and also you misinterpreted some values from the Javascript code, the type, url, data and cache were parameters for the AJAX call, not the actual data that should be sent to the server.

This is the hopefully working code, replace your email and password in the POST request.

from requests import Session # we can just import Session, no need to import the entire requests

s = Session() # new session

page_signed_out = s.get('http://www.surfline.com/home/index.cfm') # gets signed out page, this will set eventual cookies that may be needed later

s.post("http://www.surfline.com/myaccount/inc_login_handler.cfm", {"username":"myemail@example.com", "password":"mypassword", "rememberMe":"true", "top_login":"true"}) # does the actual login, if successful this will set a session cookie

page_signed_in = s.get('http://www.surfline.com/home/index.cfm') # good, we're logged in

EDIT:

Or, if for whatever reason, surfline.com does not successfully log you in, you can essentially copy the cookies planted in your browser when you physically logged in, and "paste" them into your program, then pass them in with your request.

from requests import Session

s = Session()
s.cookies["s_cc"] = "your_session_cookie" # replace this with your actual login cookie

page_signed_in = s.get("http://www.surfline.com") # you're logged in :)

다른 팁

If you are also a surfer and are just trying to get the streams to play all day I will hint at checking out the RTMP connections their flash player makes..

Even bigger hint:

tcpdump -A -n -s 0 -r file.tcpdump | grep connect

when visiting a cam page and grep for the RTMP stream..

But you might be going for something else and then this would be off topic..

라이센스 : CC-BY-SA ~와 함께 속성

제휴하지 않습니다 StackOverflow