Well-formed XML, Yahoo Finance API Output

Question

I am trying to use the output from Yahoo Finance API in my application, however, loading the XML output into an XML document keeps failing for me. What I am doing is saving the XML output as an XML file and trying to load that into and XML Document in my code. Here is the output generated by the API:

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20%28%22AAPL%22%29&env=store://datatables.org/alltableswithkeys

And these two are the problems with loading the XML into an XML Document in code. I tried removing these two things and then it works, with them it does not.

The first line, I must remove the 'yahoo:' otherwise it will not work

Then I must also remove the TickerTrend element from the XML

After these two are removed/fixed, i can load the XML into an XML Document in my code.

I am not including any code since I am using Dexterity and the loading code is working fine, only the actual XML is not ok. I tried two validators and one throws errors while the other doesn't.

Throws errors: http://www.w3schools.com/xml/xml_validator.asp
No errors: http://validator.w3.org/check?uri=http%3A%2F%2Fquery.yahooapis.com%2Fv1%2Fpublic%2Fyql%3Fq%3Dselect%2520*%2520from%2520yahoo.finance.quotes%2520where%2520symbol%2520in%2520%2528%2522AAPL%2522%2529%26env%3Dstore%3A%2F%2Fdatatables.org%2Falltableswithkeys&charset=%28detect+automatically%29&doctype=Inline&group=0

Any ideas what I can do? Instead of saving to an XML file and then loading should I just load straight from the URL?

Or how can I automatically remove the 'yahoo:' each time the file is saved? I know I can get rid of the <TickerTrend> by specifying only elements I want int the URL but not sure how to remove the 'yahoo:'

Thanks!

Answer 1

I was about to blame the problem on the fact that using the YQL in this way pulls data from user-generated tables and is not Yahoos own self-generated data, which it will tell you. So they try to absolve themselves of responsibility from this.

However, just looking at the the output from their console, this returns well-formed XML. Looking at the actual supposed REST query for the same operation, it's quite obvious that the return of the query is not well-formed due to lack of definition the "yahoo" namespace.

This output is also going to give pretty much any XML parser a hissy-fit and/or unpredictable results because of the complete lack of version details, document type and encoding information that any normal XML response would generally contain.

So yes, you are correct in stating that there is probably some nutty kind of anomaly going on with Yahoo's YQL service.

Based on an observation you can make for yourself about the state of the YQL forums themselves, which seem to lack any sort of robust activity, and another looking their blog, which hasn't posted anything in almost 2 years, one perhaps might not be far from the actual truth in stating that it's most likely Yahoo is silently deprecating/discouraging use of this API and trying to phase it out.

If you plan on pulling data via YQL REST queries, you're going to need to be able to be aware of these differences, and work accordingly around them, most likely by pulling the entire query into a string rather than trying to use generic XML parsers, using something like regular expressions to insert and define

1. the appropriate proper document type and encoding information and
2. the "yahoo" xml name space

If you do that, then give that to an XML parser you should be fine. But try and be aware of these "difficulties" when working with the YQL API.