Following on from the last answer, I modified the script. You will have to find your own condition to break the loop, I would find some element that is in the very last page and break when you get to it.
import time
driver = webdriver.Firefox()
driver.get("http://www.flipkart.com/mobiles/pr? p%5B%5D=sort%3Dfeatured&sid=tyy%2C4io&ref=659eb948-c365-492c-99ef-59bd9f0427c6")
time.sleep(3)
driver.maximize_window() # maximize window to see elements
for i in range(4):
print "Scrolling"
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") # scroll to bottom of page
time.sleep(4)
while True: #
try:
print "Scrolling to bottom of page" # need to sroll to make show-more-results visible
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") # scroll to bottom of page
time.sleep(2)# wait until you get to bottom of page to avoid "element not visible"
print "Clicking show more results"
driver.find_element_by_xpath('//*[@id="show-more-results"]').click()
# click load more button, needs to be done until you reach the end.
time.sleep(10) # adjust sleep to suit your connection
except Exception as e:
print "This happened ",e
print "Trying again"
continue
elem=[]
elem=driver.find_elements_by_xpath('.//div[@class="pu-title fk-font-13"]')
for e in elem:
print e.text
It is a quick draft and you will have to modify it but should get you closer to what you are trying to do. In particular if you get disconnected for the net, you want to break on that exception.