Download .csv from website, Can't find exact .csv URL

Question

I have a problem which is exactly similar to an older posting titled as below:

VBA Insert .csv from website, .csv not in url

It would be great if you can take a look at this posting before addressing my concern.

I need to write a VBA code that downloads the Excel file found on the website (below). I can not find the exact .csv URL as described in the posting I mentioned above.

The website requires a username and password which I don't mind sharing if needed.

Please check below link. The file is located on the top right of the table (The arrow pointing down):

https://www.oceanschedules.com/schedules/schedule-search.do?&originId=230866&destinationId=100166&weeksOut=6&date=06-May-2014&origin=New York, New York,UnitedStates(USNYC)&destination=Alexandria,Egypt(EGALY)&searchType=0&searchSort=2&showSurroundingPorts=Y&lad=05/06/2014&dad=06/17/2014&isp=1&PInUserRole=SchedulesInquiry&PInUserType=Public&PInINTTRAProduct=&carrierScac=

Any thoughts?

Answer 1

I can't give you a specific answer for VBA, but I can give you a general idea of what you need to do.

First off - you need to get the page contents of the link you posted above, as a string. Within that page - around line 68, is a JSON object with all the data of the search results, stored in the variable "aco".

You will then need to parse that JSON so you can use it - again, not sure if VBA has a method for that - but I imagine it should. From there, you can use that data however you need to. If your goal is to still get the xls file, then things get a bit more fun.

The JSP that generates the xls file is at https://www.oceanschedules.com/schedules/downloadschedule.do You need to send a POST request, giving it the following information

• statisticValue (always 4)
• totalResults (number of search results)
• searchResults (should be the same as above, possibly different if the search was filtered)
• csvResults

csvResults is a string, of the comma separated values that the server page uses to generate the xls file. It seems to include formatting of some degree, and here's what that should look like:

HAPAG-LLOYD***CMA CGM EIFFEL***4215***9248112***Wed***07-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***27***Tue***03-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***CMA CGM EIFFEL***INA62E***9248112***Thu***08-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***23***Sat***31-May-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***CMA CGM EIFFEL***INA62E***9248112***Thu***08-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***23***Sat***31-May-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SEASPAN DALIAN***04E19***9227027***Fri***09-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***24-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***MSC DYMPHNA***NU418R***9110391***Sat***10-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***07-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***FRANKFURT EXPRESS***417E***9450442***Mon***12-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***32***Fri***13-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***FRANKFURT EXPRESS***417W***9450442***Mon***12-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***34***Sun***15-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||ALIANÇA***SEASPAN HAMBURG***415E***9224300***Mon***12-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***47***Sat***28-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***SEASPAN HAMBURG***415E***9224300***Mon***12-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***47***Sat***28-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***MAERSK BATUR***419S***9402029***Mon***12-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***24-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MAERSK UTAH***1407***9305300***Tue***13-May-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***12-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***OOCL KOBE***INA64E***9329526***Wed***14-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***24***Sat***07-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***OOCL KOBE***INA64E***9329526***Wed***14-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***24***Sat***07-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***OOCL KOBE***4216***9329526***Wed***14-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***27***Tue***10-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***NAGOYA EXPRESS***418E***9450428***Thu***15-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***36***Fri***20-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SANTA ROBERTA***04E20***9227326***Fri***16-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***01-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***MSC CAROLE***NU419R***8618308***Sat***17-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***14-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SPIRIT OF SINGAPORE***420S***9362396***Mon***19-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***31-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MAERSK DENPASAR***1407***9348663***Tue***20-May-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***19-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***WESTERMOOR***419E***9222106***Wed***21-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***30***Fri***20-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SEASPAN HAMBURG***03E21***9224300***Fri***23-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***08-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***MSC ILONA***NU420R***9225641***Sat***24-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***21-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***JPO LIBRA***INA66E***9297840***Sun***25-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***22***Mon***16-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***JPO LIBRA***INA66E***9297840***Sun***25-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***22***Mon***16-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***JPO LIBRA***4217***9297840***Sun***25-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***23***Tue***17-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***MAERSK BRANI***421S***9409352***Mon***26-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***07-Aug-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MAERSK WISCONSIN***1407***9193252***Tue***27-May-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***26-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***KUALA LUMPUR EXPRESS***420E***9343730***Tue***27-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***31***Fri***27-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***DUBAI EXPRESS***INA68E***9301782***Wed***28-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***26***Mon***23-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***DUBAI EXPRESS***INA68E***9301782***Wed***28-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***26***Mon***23-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***DUBAI EXPRESS***4218***9301782***Wed***28-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***27***Tue***24-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***EM ATHENS***419E***9203538***Wed***28-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***39***Sun***06-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SANTA RICARDA***04E22***9227314***Fri***30-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***15-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***MSC MICHAELA***NU421R***9230488***Sat***31-May-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***28-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***MAERSK BRATAN***422S***9394894***Mon***02-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***14-Aug-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MAERSK VIRGINIA***1407***9235531***Tue***03-Jun-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***03-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***HOUSTON EXPRESS***421E***9294991***Tue***03-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***31***Fri***04-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***APL OMAN***INA70E***9400124***Wed***04-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***23***Fri***27-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***APL OMAN***INA70E***9400124***Wed***04-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***23***Fri***27-Jun-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***APL OMAN***4219***9400124***Wed***04-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***27***Tue***01-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SEASPAN FELIXSTOWE***05E23***9227039***Fri***06-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***22-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***HS COLUMBIA***NU422R***9225079***Sat***07-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***05-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SPIRIT OF MELBOURNE***423S***9362413***Mon***09-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***21-Aug-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MSC FIRENZE***1407***9252101***Tue***10-Jun-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***10-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***FRANKFURT EXPRESS***422E***9450442***Tue***10-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***31***Fri***11-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||CMA CGM***CMA CGM KINGFISH***INA72E***9330991***Wed***11-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***26***Mon***07-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||DELMAS***CMA CGM KINGFISH***INA72E***9330991***Wed***11-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***26***Mon***07-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***CMA CGM KINGFISH***4220***9330991***Wed***11-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***27***Tue***08-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***SEASPAN NINGBO***04E24***9227015***Fri***13-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***46***Tue***29-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MSC***MSC DYMPHNA***NU423R***9110391***Sat***14-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***28***Sat***12-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAPAG-LLOYD***OLGA MAERSK***424S***9251614***Mon***16-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***73***Thu***28-Aug-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||MAERSK LINE***MAERSK KENTUCKY***1407***9193240***Tue***17-Jun-2014***NEWARK, NEW JERSEY, UNITED STATES***USEWR***29***Thu***17-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||HAMBURG SÜD***NAGOYA EXPRESS***423E***9450428***Tue***17-Jun-2014***NEW YORK, NEW YORK, UNITED STATES***USNYC***31***Fri***18-Jul-2014***EL ISKANDARIYA (= ALEXANDRIA), EGYPT***EGALY|||

Simply send all that in a post request, and you'll get the xls data in return. Write that to a file, and you have your excel doc. Now, I haven't gone through the effort of translating the JSON to the csv for your, or the server's needs, but you should be able to piece it together well enough. The POST, and the JSON parts on the other hand, I did verify, so hopefully that gives you what you need.

One side note - you won't need to log in for this to work. Just the download button itself requires being logged in. Kinda the equivalent of locking your front door when you're missing an entire wall.