How to fetch more than 25 post messages

Question

I'm trying to get all post messages using restfb, my code is as follows

public Connection<Post> publicSearchMessages(Date fromDate, Date toDate) {
    Connection<Post> messages = publicFbClient.fetchConnection("search",
            Post.class,
            Parameter.with("q", "Watermelon"),
            Parameter.with("since", fromDate),
            Parameter.with("until", toDate),
            Parameter.with("type", "post"));

    return messages;
}

This only gives latest 25 post messages.

Parameter.with("limit",100 )

If i set limit parameter, it gives 100 messages but i don't want to limit to fetching post messages. So,

Is there anyway I can get a full list of post messages matching the search criteria without setting limit parameter?

Answer 1

There is no way to fetch unlimited results from FB. The default limit is set to 25. As you know, you can change this using the limit parameter. I have not found an upper border for limit searching the web. Maybe, you can set it to a very high amount.

Answer 2

Maybe you can try using a loop. FB can't get more than 1000 each time, so you can use the loop to get the whole feeds. Use the offset like this:

Parameter.with("limit", 1000));
Parameter.with("offset", offset));

Offset will be a variable and its value will be 1000,2000,3000...

Answer 3

As I tested recently, You don't have to specify anything. Connection class implements Iterable in this way:

• fetch 25 results
• hasNext check if there is next item to process
• if not, it will fetch next page of 25 results

So basically all you need to do is this:

Connection<Post> messages = publicFbClient.fetchConnection("search",
        Post.class,
        Parameter.with("q", "Watermelon"),
        Parameter.with("since", fromDate),
        Parameter.with("until", toDate),
        Parameter.with("type", "post"));

for (List<Post> feedConnectionPage : messages) {
        for (Post post : myFeedConnectionPage) {
                 // do stuff with post
        }
}

If you want to have some kind of method that returns results I would be very careful, because you can be returning thousands of results and crawling through them may take some time (from seconds to minutes or even hours) and result object array is going to be really big. Better idea is to use some asynchronous call and check out results of method periodically.

Though it seems that parameter "since" is ignored. Posts are fetched from newest to oldest and I think that it somehow leave out this parameter when doing paging.

Hope I made it more clear for you :)

Answer 4

We have an Iterator object in Post. So we can do it like this:

Connection<Post> messages = publicFbClient.fetchConnection(...) ;
someMethodUsingPage(messages);
    while (messages.hasNext()) {
        messages = facebookClient.fetchConnectionPage(messages.getNextPageUrl(), Post.class);
        someMethodUsingPage(messages);
    }

Then in each messages we'll have next 25 messages.