Selenium SelectBox Performance

Question

Version :

• Python 2.7.6 - 32 Bits
• Selenium 2.41.0
• Firefox 24.0( Necessary , with the new version FF Hangs)

OS:

• Windows Server 2008 R2, 64 Bits
• 4 vCPU
• 8 GB
• VM With 1:1 Ratio (Private Cloud)

Problems : I have to crawl aprox: 9000 pages and each page contains 20 selectboxes with around 200 options

I only have to grab the curent value and for that I use :

Obj = ElementObj.find_element_by_name(ElementName)
Select(Obj).first_selected_option.get_attribute("value")

Now the issue is all about performance it takes 30 seconds to actually go trough one page. If i try to crawl a page without any selectboxes all goes at a much faster rate (close to 0.4 seconds for page)

So I'm I doing something wrong ?

Thanks

Answer 1

Ok, So in order to increase your performance it's best to do (if possible)

Driver.execute_script("""return document.getElementsByName("%s")[0].value""" % ElementName)

This will give you at least 10x Boost Of Performance.

I hope this helps someone !

Answer 2

Selenium is typically slow, it's likely that you aren't doing anything wrong. I've seen it used with many languages, and it doesn't seem to improve noticeably.

Hope that helps :)