WebDriverWait(selenium) not working

Question

i had written a code in python and selenium. It clicks show-more-results button once. I have used WebDriverWait so that it allows the page to load but no use.

for j in range(100):
    driver.find_element_by_xpath('.//div[@id="show-more-results"]').click()
    WebDriverWait(driver, 50).until(lambda driver : driver.find_element_by_xpath('.//div[@id="show-more-results"]'))

The following error is shown

Traceback (most recent call last):
  File "python23.py", line 20, in <module>
    driver.find_element_by_xpath('.//div[@id="show-more-results"]').click()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 60, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 370, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 166, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: u'element not visible\n  (Session info: chrome=34.0.1847.132)\n  (Driver info: chromedriver=2.9.248304,platform=Linux 3.13.0-24-generic x86_64)' 

Answer 1

Following on from the last answer, I modified the script. You will have to find your own condition to break the loop, I would find some element that is in the very last page and break when you get to it.

import time
driver = webdriver.Firefox()
driver.get("http://www.flipkart.com/mobiles/pr?   p%5B%5D=sort%3Dfeatured&sid=tyy%2C4io&ref=659eb948-c365-492c-99ef-59bd9f0427c6")
time.sleep(3)
driver.maximize_window() # maximize window to see  elements
for i in range(4):
    print "Scrolling"
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") # scroll to bottom of page
    time.sleep(4)
while True:  #
    try:
        print "Scrolling to bottom of page" # need to sroll to make show-more-results visible
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") # scroll to bottom of page
        time.sleep(2)#   wait until you get to bottom of page to avoid "element not visible"
        print "Clicking show more results"
        driver.find_element_by_xpath('//*[@id="show-more-results"]').click()
        # click load more  button, needs to be done until you reach the end.
        time.sleep(10) # adjust sleep to suit your connection
    except Exception as e:
        print "This happened ",e
        print "Trying again"
        continue
elem=[]
elem=driver.find_elements_by_xpath('.//div[@class="pu-title fk-font-13"]')
for e in elem:
   print e.text

It is a quick draft and you will have to modify it but should get you closer to what you are trying to do. In particular if you get disconnected for the net, you want to break on that exception.