Get Json with i.e 200 objects, but only 100 displayd on one site

Question

I'm working in my Twitch IRC Bot and we got a Problem.

We receive alot of information through the twitch API (json) like Who followed, dateOf .. viewercounts.. amount of followers and stuff like that.

Were making a Follow-Function to read all the names out of the whole list and set all into our database. First off we tried to just read all and system.output them but we always get the error: org.json.JSONException: JSONArray[100] not found.

We noticed that "0" is holding an array as well so we set the loop to 0-99 and it should change the link then by adding 100+ (next site) and read the JSON again. then it should just continue the loop with the next site.

Below is the Main code as well for the read-methods.

We tried debugging but we wasn't able to find a solution yet x(

MyBot Main Code Snippet:

JSONObject follower = null;
String followername = null;
int listnumber;

offsetvalue = 0;

for(int i = 0; i < TwitchStatus.totalfollows; i++) { 

    try {
        follower = TwitchStatus.followerarray.getJSONObject(i);
    } catch (JSONException e2) {
        e2.printStackTrace();
    }        

    try {
        followername = follower.getJSONObject("user").getString("display_name");
    } catch (JSONException e) {
        e.printStackTrace();
    } 

    System.out.println("array ist: "+i +" " +followername);

    listnumber = offsetvalue+99; // 0+99

    if (i == listnumber){
        offsetvalue = offsetvalue+100;

        try {
            TwitchStatus.FollowerTicker();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (JSONException e) {
            e.printStackTrace();
        }
    }
    // System.out.println("Follower abgleichs-Liste: "+followername);        
}

And there is the Reader Method:

////////////////////////////////////////////////////////////////////////////////////
// Twitch Follower Ticker
////////////////////////////////////////////////////////////////////////////////////

private String readAll4(Reader rd) throws IOException {
    StringBuilder sb = new StringBuilder();
    int cp;
    while ((cp = rd.read()) != -1) {
        sb.append((char) cp);
    }
    return sb.toString();
  }

  public JSONObject readJsonFromUrl4(String url) throws IOException, JSONException {
    InputStream is = new URL(url).openStream();
    try {
        BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));
        String jsonText = readAll4(rd);
        JSONObject json = new JSONObject(jsonText);
        return json;
    } finally {
        is.close();
    }
  }

  public static void FollowerTicker() throws IOException, JSONException {
    json = readJsonFromUrl2("https://api.twitch.tv/kraken/channels/dotastarladder_en/follows?direction=DESC&limit=100&offset="+MyBot.offsetvalue+"");

    followerarray = json.getJSONArray("follows");

    {
        JSONObject follower = followerarray.getJSONObject(0);
        neuerfollower = follower.getString("created_at");
        fname = follower.getJSONObject("user").getString("display_name");  
        totalfollows = json.getInt("_total");
    }
} 

Answer 1

Note from the API docs:

limit optional integer Maximum number of objects in array. Default is 25. Maximum is 100.

So, what do you do? Query the next one, of course! Here's the bit of JSON from the linked page, and an example next URL. Basically, you just put an offset in, but the URL already declares it, so...

{
  "_total": 1234,
  "_links": {
    "next": "https://api.twitch.tv/kraken/channels/test_user1/follows?direction=DESC&limit=25&offset=25",

How I would solve this problem is something like this:

1. Create an AsyncTask that takes in the URL to parse the JSON text.
2. When the data has been received, start a new task to read the next one.
3. Read everything received in this JSON string
4. Compile everything after it has been downloaded as needed.