Manually login into website with Typheous

Question

Recently I was using Mechanize for this kind of thing, but I want to use Typhoeus, which I'm already using everywhere else. I want to mimic Mechanize's behavior, the issue is that I would like to log in into a site and perform requests as logged in user. Here is generalised version of the script:

require 'rubygems'
require 'typhoeus'

GET_URL = 'http://localhost:3000'
POST_URL = "http://localhost:3000/admins/sign_in"
URL = "http://localhost:3000/dashboard"
USERNAME_FIELD = 'admin[email]'
PASSWORD_FIELD = 'admin[password]'
USERNAME = "admin@example.com"
PASSWORD = "my_secret_password"

def merge_cookies_into_cookie_jar(response)                            
  if response.headers_hash['set-cookie'].instance_of? Array
    response.headers_hash['set-cookie'].each do |cookie|
      @cookie_jar << cookie.split('; ')[0]
    end
  elsif response.headers_hash['set-cookie'].instance_of? String
    @cookie_jar << response.headers_hash['set-cookie'].split('; ')[0]
  end        
end

# initialize cookie jar
@cookie_jar = []

# for server to establish me a session
response = Typhoeus::Request.get( GET_URL, :follow_location => true )
merge_cookies_into_cookie_jar(response)                                                   

# like submiting a log in form
response = Typhoeus::Request.post( POST_URL,
                                   :params => { USERNAME_FIELD => USERNAME, PASSWORD_FIELD => PASSWORD },
                                   :headers => { 'Cookie' => @cookie_jar.join('; ') }
                                 )
merge_cookies_into_cookie_jar(response)                                                   

# the page I'd like to get in a first place, 
# but not working, redirects me back to login form with 401 Unauthorized :-(                 
response = Typhoeus::Request.get( URL, 
                                  :follow_location => true,
                                  :headers => { 'Cookie' => @cookie_jar.join('; ') }
                                 )

The cookie gets sent to the server, but for some reason I'm not logged in. I tested it on two different sites (which one of them was my Rails application's administration). Any idea what am I doing wrong or maybe a better or more widely applicable solution to this problem?

Answer 1

Here's code that I'm able to run successfully.

First, your cookie jar is an Array, and in my code it needs to be an Array with replacement (or a Hash). When I run the code on a real app of mine, the GET_URL response returns a session cookie, but the POST_URL response returns a different session cookie.

# initialize cookie jar as a Hash
@cookie_jar = {}

Adjust the parsing so you get each cookie's name and value:

def merge_cookies_into_cookie_jar(response)
  x =  response.headers_hash['set-cookie']
  case x
  ...
  when String
    x.split('; ').each{|cookie|
      key,value=cookie.split('=', 2)
      @cookie_jar[key]=value
    }
  end
end

The cookie jar needs to convert to a string:

def cookie_jar_to_s
  @cookie_jar.to_a.map{|key, val| "#{key}=#{val}"}.join("; ")
end

Finally, change your headers to use the new cookie_jar_to_s:

:headers => { 'Cookie' => cookie_jar_to_s }

A bonus would be to make the cookie jar its own class, perhaps something like this:

class CookieJar < Hash

  def to_s
    self.to_a.map{|key, val| "#{key}=#{val}"}.join("; ")
  end

  def parse(*cookie_strings)
    cookie_strings.each{|s|
      s.split('; ').each{|cookie|
        key,value=cookie.split('=', 2)
        self.[key]=value
      }
    }
  end

end

Answer 2

I've fixed @joelparkerhenderson's CookieJar (since it wasn't working here). Here the result:

class CookieJar < Hash
  def to_s
    self.map { |key, value| "#{key}=#{value}"}.join("; ")
  end

  def parse(cookie_strings)
    cookie_strings.each { |s|
      key, value = s.split('; ').first.split('=', 2)
      self[key] = value
    }
    self
  end
end

# Use like this:
response = Typhoeus::Request.get("http://www.example.com")
cookies = CookieJar.new.parse(response.headers_hash["Set-Cookie"])
Typhoeus::Request.get("http://www.example.com", headers: {Cookie: cookies.to_s})

Answer 3

Are your sites using Rails forgery protection?

If so, when you get the form page, Rails is sending a hidden field that is a CSRF token.

It will look something like this in HTML:

<input type="hidden" name="csrf" value="abcdef">

You need to use this hidden value when you post your form:

:params => {"csrf" => "abcdef", USERNAME_FIELD => USERNAME, ...

The hidden field tells Rails that you are the person who made the request for the form page, thus you (and only you) are allowed to post.

Here are my notes on CSRF with links to more info:

http://sixarm.com/about/rails-session-csrf-token-jquery-ajaxprefilter.html

Related StackOverflow CSRF info:

http://stackoverflow.com/questions/941594/understand-rails-authenticity-token