Accepting Terms and Conditions with Python and Mechanize

Question

I'm using Python and Mechanize to search a database and return results. I'm having a problem getting through the Terms and Conditions page to access the actual database search page.

I've tried using Mechanize to manipulate the Agree button on the page but I am getting this response.

<Form1 POST http://www.wataugacounty.org/ias/Search/Disclaimer2.aspx?FromUrl=..%
 2fsearch%2fcommonsearch.aspx%3fmode%3dparid application/x-www-form-urlencoded
  <HiddenControl(__EVENTTARGET=)>
  <HiddenControl(__EVENTARGUMENT=)>
  <HiddenControl(__VIEWSTATE=/wEPDwUKLTk3MzgyMzUzOQ8UKwACZGdkZARSom15z0v0QyobiaK
gUmCf0EcW)>
  <IgnoreControl(btDisagree=<None>)>
  <IgnoreControl(btAgree=<None>)>>
Traceback (most recent call last):
  File "run.py", line 19, in <module>
    br["btAgree"] = 'True'
   File "C:\Python27\Lib\site-packages\mechanize\_form.py", line 2784, in __setit
em__
raise ValueError(str(e))
ValueError: control 'btAgree' is ignored, hence read-only

I'm not sure if Mechanize is capable of processing the acceptance. Where do I go from here?

Answer 1

Mechanize won't cut it because it doesn't evaluate Javascript code and you really need it because the input button triggers a Javascript function. Even if you try to use Mechanize to simply submit the form, you'll get the following message:

The web site is unable to acknowledge your acceptance of the disclaimer message. This situation is typically caused when your web browser is set to prevent the use of Javascript. This site requires that Javascript be enabled in your browser. Javascript is a programming language that allows us to provide you with a richer and more enjoyable experience than would otherwise be the case. This site will not operate without Javascript.

You've got some alternatives though.

• ghost.py is a headless webkit web client (and, as such, it'll run Javascript)
• Selenium
• Also noteworthy (although not written in Python) is PhantomJS, another headless webkit client