YQL Reliability - Example using Google Weather API

Question

Im fairly new to YQL and am having some issues with the reliability. (From a Google XML Source)

My request fails about 50% of the time. I wanted to know why this is and how it can be resolved.

My initial thoughts are;

• Is google preventing YQL in making a successful request?
• Is YQL itself not very reliable?

Here is the ouptut or the YQL Query (You can put it in your address bar and keep refreshing to view its success / failure).

YQL JSON

---

This is the YQL Statement I am using

select * from xml
    where url='http://www.google.com/ig/api?weather=CM12QE'

I am also picking the JSON option on the YQL console.

---

Am I doing something wrong? Is YQL not very reliable to use as a service within my application? Is there good way to guarantee the desired results without making another request?

Answer 1

Is google preventing YQL in making a successful request?

It is far more often the case that rather than YQL being a problem, the data source (whatever that might be) is the root of any problem. In this case, Google is giving a different response to the one that you and YQL expects to get; this is because the YQL engine is being presented with a redirect to a page containing a CAPTCHA input form (example) rather than the desired XML.

Is YQL itself not very reliable?

I've worked with YQL for many years and the single most often cause of problems is not YQL but the external data sources. YQL can fall over from time to time, however.

What can I do?

The first step is to determine what the problem really is. Basic steps include turning on diagnostics and debugging. There are check boxes in the console, or put diagnostics=true and debug=true in the YQL URL. These give details on what external resources are fetched and their status information. Diagnostics are usually enough to determine the problem, but debug details are good to have too since they provide the most fine-grained detail of YQL's interaction with the external resource.

See also

• http://developer.yahoo.com/yql/guide/yql-network-logging.html

Answer 2

After debugging the request a bit more, using &diagnostics=true I found that Google was blocking the YQL servers and requesting human verification via captcha.

See this link for example. http://www.google.com/sorry/?continue=http://www.google.com/ig/api%3Fweather%3DCM12QE

The answer here was to use &diagnostics=true on the end of the url. And get some additional debugging information.

What is rather interesting is that when running the YQL through the console is that the sucess rate is around 100%