How to get the json data from the url using Python script apart from requests library?

Question

I have a URL which gives me the below JSON String if I hit them on the browser -

Below is my URL, let's say it is URL-A and I have around three URL's -

http://hostnameA:1234/Service/statistics?%24format=json

And below is my JSON String -

{
 "description": "",
 "statistics": {
  "dataCount": 0,
 }
}

Now I have written a Python script which is scanning all my 3 URL's and then parse then JSON String to extract the value of dataCount from it. And it should keep on running every few seconds to scan the URL and then parse it.

Below are my URL's

hostnameA       http://hostnameA:1234/Service/statistics?%24format=json
hostnameB       http://hostnameB:1234/Service/statistics?%24format=json
hostnameC       http://hostnameC:1234/Service/statistics?%24format=json

And the data I want to see is like this on the console, here dataCount will be actual number

hostnameA - dataCount
hostnameB - dataCount
hostnameC - dataCount

And I have the below python script which works fine locally on my cygwin but if I am running it on my company's production ubuntu machine it gives an error -

import requests
from time import sleep

def get_data_count(url):
    try:
        req = requests.get(url)
    except requests.ConnectionError:
        return 'could not get page'

    try:
        # this line is giving an error
        return int(req.json['statistics']['dataCount'])
    except TypeError:
        return 'field not found'
    except ValueError:
        return 'not an integer'

def main():
    urls = [
        ('hostnameA', 'http://hostnameA:1234/Service/statistics?%24format=json'),
        ('hostnameB', 'http://hostnameB:1234/Service/statistics?%24format=json'),
        ('hostnameC', 'http://hostnameC:1234/Service/statistics?%24format=json')
    ]

    while True:
        print('')
        for name, url in urls:
            res = get_data_count(url)
            print('{name} - {res}'.format(name=name, res=res))
        sleep(10.)

if __name__=="__main__":
    main()

Below is the error I am getting -

AttributeError: 'Response' object has no attribute 'json'

I am using Python 2.7.3 and Ubuntu 12.04 and the version of requests I am running is 0.8.2 (I guess this is the problem).

In any case, is there any way I can rewrite the above script using some other library apart from requests meaning just the portion of getting the data from server, we can use other libraries right?

Since I guess, I cannot update this package since it's our production ubuntu servers so I need to find some other way to do this.

Answer 1

You can use requests still, just not rely on the response object to do the decoding for you:

import json

# ...
data = json.loads(req.content)
return int(data['statistics']['dataCount'])

Version 0.8.2 is ancient; you can use virtualenv to create a location for you to install a newer version instead:

$ virtualenv venv
New python executable in venv/bin/python2.7
Also creating executable in venv/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
$ cd venv/
$ bin/pip install requests
Downloading/unpacking requests
  Downloading requests-2.2.1.tar.gz (421kB): 421kB downloaded
  Running setup.py egg_info for package requests

Installing collected packages: requests
  Running setup.py install for requests

Successfully installed requests
Cleaning up...
$ bin/python -c 'import requests; print requests.__version__'
2.2.1