Download CSV from url in Selenium?

Question

I have a URL that I want to regularly download from. It can only be accessed from a logged in account which requires javascript support to do so, and so I'm forced to use Selenium with PhantomJS, trust me. Otherwise, I would just use urllib for this, but it gives me a sign in error.

It's a CSV file, with a maximum of 1000 rows, and about 6 columns.

I want to eventually get this CSV into a list. Does anyone know how to download a CSV with Selenium Webdriver?

Thanks so much.

Edit: I'm just looking to download a CSV from a URL in Selenium. Nothing else.

Answer 1

its actually pretty simple. Using another answer I gave in stack over flow

https://stackoverflow.com/a/21871600/2423379

EDIT: Running Firefox in Headless mode

Requirements:

• sudo apt-get install xvfb (or equivalent command in ur distro)
• pip install --user xvfbwrapper

And code part

from xvfbwrapper import Xvfb

vdisplay = Xvfb()
vdisplay.start()

# launch stuff inside virtual display here

vdisplay.stop()

Ref: Firefox-selenium in headless mode

Answer 2

How about page_source attribute?

browser.get("http://winterolympicsmedals.com/medals.csv")
csv_file = browser.page_source
print(csv_file)

Try this, my friend. I use Selenium + Python + HTMLUnit, working like a breeze.

Hope it works for your PhantomJS