Question

I am trying to scrape a password protected website in python. My code is as follows:

import mechanize
import cookielib
from BeautifulSoup import BeautifulSoup

br = mechanize.Browser()

cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)

br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)

br.addheaders = [('User-agent', 'Chrome')]

br.open('https://monitor1.returnpath.net/login.php')

for f in br.forms():
print f

br.select_form(nr=1)

br.form['email'] = 'email'
br.form['password'] = 'password'

The for loop returns this:

<form1 POST https://monitor1.returnpath.net/login.php application/x-www-form-urlencoded
  <TextControl(email=)>
  <PasswordControl(password=)>
  <CheckboxControl(remember=[1])>
  <SubmitControl(Submit=Sign In) (readonly)>>
<GET http://now.eloqua.com/e/f2.aspx application/x-www-form-urlencoded
  <TextControl(e=)>
  <HiddenControl(lang=NA) (readonly)>
  <HiddenControl(elqSiteID=841) (readonly)>
  <HiddenControl(elqFormName=nLRegFooter-1347904420246) (readonly)>
  <SubmitControl(<None>=Sign Me Up) (readonly)>
  <SubmitButtonControl(<None>=) (readonly)>>
<POST https://monitor1.returnpath.net/send_feedback.php application/x-www-form-urlencoded
  <HiddenControl(size=) (readonly)>
  <HiddenControl(nps=) (readonly)>
  <TextareaControl(desc=)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>
  <IgnoreControl(<None>=<None>)>>

And this error:

mechanize._form.ControlNotFoundError: no control matching name 'email'

The output states that 'email' is found so I'm not sure why it says there is no control matching it?

Was it helpful?

Solution

Its zero-indexed. try the code below:

br.select_form(nr=0)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top