Asynchronous requests with Mechanize

Question

I'm parsing a bunch of documents in parallel, and I'd love to do it using asynchronous web requests.

I tried to integrate typhoeus with mechanize, but encountered a problem with passing cookies from mechanize.

My steps:

1. Login via mechanize and get cookies.
2. Pass cookies to typhoeus and send request.
3. Parse response with nokogiri.
4. Repeat 2-3 many times.

I'm sending request as:

Typhoeus::Request.new("http://example.com", headers: {'Cookie' => agent.cookies})

where agent is a mechanize instance.

Is there a way to integrate Mechanize with Eventmachine or Typhoeus?

Answer 1

agent.cookies is an array. Maybe you want:

agent.cookies.join('; ')