Python how to preserve HTTP cookies

Question

I used this piece to

cj = cookielib.LWPCookieJar()
cookie_support = urllib2.HTTPCookieProcessor(cj)
opener = urllib2.build_opener(cookie_support, urllib2.HTTPHandler)
urllib2.install_opener(opener)

// ..... log in with username and password. 
// urllib2.urlopen() to get the stuff I need. 

Now, how do I preserve the cookie and set the expiration dates to forever, so next time I don't have to log in with username and password again. I can directly use urllib2.urlopen() ?

By "next time" I mean after the program ends, when I start a new program, I can just reload the cookie from the disk and use it

Thanks a lot

Answer 1

I would highly recommend using the Requests HTTP library. It will handle all this for you.

http://docs.python-requests.org/en/latest/

import requests
sess = requests.session()
sess.post("http://somesite.com/someform.php", data={"username": "me", "password": "pass"})
#Everything after that POST will retain the login session
print sess.get("http://somesite.com/otherpage.php").text

edit: To save the session to disk, there are a lot of ways. You could do the following:

from requests.utils import cookiejar_from_dict as jar
cookies = jar(sess.cookies)

Then read the following documentation. You could convert it to a FileCookieJar and save the cookies to a text file, then load them at the start of the program.

http://docs.python.org/2/library/cookielib.html#cookiejar-and-filecookiejar-objects

Alternatively you could pickle the dict and save that data to a file, and load it with pickle.load(file).

http://docs.python.org/2/library/pickle.html

edit 2: To handle expiration, you can iterate over the CookieJar as follows. cj is assumed to be a CookieJar obtained in some fashion.

for cookie in cj:
    if cookie.is_expired():
        #re-attain session

To check if any of the cookies are expired, it may be more convenient to do if any(c.is_expired() for c in cj).