Morningstar csv download

Question

I am using a Mac and am trying to automate the download of a csv file from Morningstar which is generated from this page:IBM Financials by pressing the "Export" button. The button generates a file from this request: http://financials.morningstar.com/ajax/ReportProcess4CSV.html?&t=XNYS:IBM&region=usa&culture=en-US&cur=USD&reportType=is&period=12&dataType=A&order=asc&columnYear=10&rounding=3&view=raw&r=337541&denominatorView=raw&number=3. I can use the curl command to generate a csv file but if you have not logged into Morningstar, the file returned only has 5 years of data whereas for members it returns 10 years of data.

I have tried using the the following curl command

curl -v -u username:password 'http://financials.morningstar.com/ajax/ReportProcess4CSV.html?&t=XNYS:IBM&region=usa&culture=en-US&cur=USD&reportType=is&period=12&dataType=A&order=asc&columnYear=10&rounding=3&view=raw&r=337541&denominatorView=raw&number=3'

but the website ignores the login and just returns the standard 5 years of data.

I have also tried suppling a cookie with the curl as I thought that it may be referred to by the site and recognise that I was already logged from Safari: curl -v -b /Users/xxx/Library/Cookies/Cookies.binarycookies 'http://financials.morningstar.com/ajax/ReportProcess4CSV.html?&t=XNYS:IBM&region=usa&culture=en-US&cur=USD&reportType=is&period=12&dataType=A&order=asc&columnYear=10&rounding=3&view=raw&r=337541&denominatorView=raw&number=3'

Does anyone have any idea how to tell the website that I am logged in?

Answer 1

I ended up using mechanize via a ruby script to log into the website and then generate the cvs file.