Multiple NSUrlRequests To Fetch Different Pages Of WebService To Load More Data In TableView

Question 1

Your issue can be understood if you consider this older question of yours and the solution I proposed.

Specifically, the critical bit has to do with the way you are persisting the information (RSS info + images), which is through archiving your whole channel to a file on disk:

        [channelCopy addItemsFromChannel:obj];
        [NSKeyedArchiver archiveRootObject:channelCopy toFile:pathOfCache];

Now, if you look at fetchEntriesNew:, the first thing that you do there is destroying your current channel. If this happens before the channel has been persisted to disk you enter a sort of endless loop.

I understand you are currently persisting your channel (as per my original suggestion) at the very end of image download.

What you should do is persisting the channel just after the feed has been read and before starting downloading the images (you should of course also persist it at the end of image downloads).

So, if you take this snippet from my old gist:

[connection setCompletionBlock:^(RSSChannel *obj, NSError *err) {

    if (!err) {

        [channelCopy addItemsFromChannel:obj];

        // ADDED
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            dispatch_group_wait(obj.imageDownloadGroup, DISPATCH_TIME_FOREVER);
            [NSKeyedArchiver archiveRootObject:channelCopy toFile:cachePath];
        });
    }
    block(channelCopy, err);

what you should do is adding one more archiveRootObject call:

[connection setCompletionBlock:^(RSSChannel *obj, NSError *err) {

    if (!err) {

        [channelCopy addItemsFromChannel:obj];
        [NSKeyedArchiver archiveRootObject:channelCopy toFile:cachePath];

        // ADDED
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            dispatch_group_wait(obj.imageDownloadGroup, DISPATCH_TIME_FOREVER);
            [NSKeyedArchiver archiveRootObject:channelCopy toFile:cachePath];
        });
    }
    block(channelCopy, err);

This will make things work as long as you do not scroll fast enough so that the channel is destroyed before the feed (without images) is ever read. To fix this you should add a bool to your TheFeedStore class that you set to YES when you call fetchWebService and reset just after executing the newly added archiveRootObject:channelCopy.

This will fix your issues.

Let me also say that from a design/architecture point of view, you have a big issue with the way you manage persistence. Indeed, you have a single file on disk that you write atomically using archiveRootObject. This architecture is intrinsically "risky" from a multi-threading point of view and you should also devise a way to avoid that concurrent accesses to the shared stored have no destructive effects (e.g.: you archive your channel to disk for page 4 at the same time as the images for page 1 have been fully downloaded, hence you try to persist them as well to the same file).

Another approach to image handling would be storing the images outside of your archive file and treat them as a sort of cache. This would fix the concurrency issues and will also get rid of the performance penalty you get from archiving the channel twice for each page (when the feed is first read and later when the images have come in).

Hope this helps.

UPDATE:

At this point, its not destroying and reloading the same batch again and again, exactly what i wanted. However, it doesn't persist images if i scroll multiple times to reach the next page without the images of the previous page were loaded completely.

This is exactly what I meant saying that your architecture (shared archive/concurrent access) would probably lead to problems.

You have several options: use Core Data/sqlite; or, more easily, store each image in its own file. In the latter case, you could do following:

on retrieval, assign to each image a filename (this could be the id of the feed entry or a sequential number or whatever) and store the image data there;
store in the archive both the URL of the image and the filename where it should be stored;
when you need accessing the image, you don't get it from the archived dictionary directly; instead, you get the filename from the it then read the file from disk (if available);
this change would not affect otherwise your current implementation of rss/image retrieval, but only the way you persist the images and you access them when required (I mean, it seems a pretty easy change).

2) I am not sure how to use Bool as he explained in the answer.

add a isDownloading bool to TheFeedStore;
set it to YES in your fetchWebService: method, just before doing [connection start];
set it to NO in the completion block you pass to the connection object (again in fetchWebService:) right after archiving the feed the first time (this you are already doing);
in your scrollViewDidEndDecelerating:, at the very beginning, do:
```
    if ([TheFeedStore sharedStore].isDownloading)
        return;
```
so that you do not refresh the rss feed while a refresh is ongoing.

Let me know if this helps.

NEW UPDATE:

Let me sketch how you could deal with storing images in files.

In your RSSItem class, define:

@property (nonatomic, readonly) UIImage *thumbnail;
@property (nonatomic, strong) NSString *thumbFile;

thumbFile is the the path to the local file hosting the image. Once you have got the image URL (getFirstImageUrl), you can get, e.g., and MD5 hash of it and use this as your local image filename:

NSString* imageURLString = [self getFirstImageUrl:someString];
....
self.thumbFile = [imageURLString MD5String];

(MD5String is a category you can google for).

Then, in downloadThumbnails, you would store the image file locally:

    NSMutableData *tempData = [NSData dataWithContentsOfURL:finalUrl];
    [tempData writeToFile:[self cachedFileURLFromFileName:self.thumbFile] atomically:YES];
    [[NSNotificationCenter defaultCenter] postNotificationName:@"DataSaved" object:nil];

Now, the trick is, when you access the thumbnail property, you read the image from file and return it:

- (UIImage *)thumbnail
{
    NSData* d = [NSData dataWithContentsOfURL:[self cachedFileURLFromFileName:self.thumbFile]];
    return [[UIImage alloc] initWithData:d];
}

in this snippet, cachedFileURLFromFileName: is defined as:

- (NSURL*)cachedFileURLFromFileName:(NSString*)filename {

NSFileManager *fileManager = [[NSFileManager alloc] init];
NSArray *fileArray = [fileManager URLsForDirectory:NSCachesDirectory inDomains:NSUserDomainMask];

NSURL* cacheURL = (NSURL*)[fileArray lastObject];
if(cacheURL)
{
    return [cacheURL URLByAppendingPathComponent:filename];
}
return nil;
}

Of course, thumbFile should be persisted for this to work.

As you see, this approach is pretty "easy" to implement. This is not an optimized solution, just a quick way to make your app work with its current architecture.

For completeness, the MD5String category:

@interface NSString (MD5)

- (NSString *)MD5String;

@end

@implementation NSString (MD5)

- (NSString *)MD5String {
const char *cstr = [self UTF8String];
unsigned char result[16];
CC_MD5(cstr, strlen(cstr), result);

return [NSString stringWithFormat:
        @"%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X",
        result[0], result[1], result[2], result[3],
        result[4], result[5], result[6], result[7],
        result[8], result[9], result[10], result[11],
        result[12], result[13], result[14], result[15]
        ];  
}

@end

Question 2

What you are actually trying to do, is implement paging in a UITableView

Now this is very straightforward and the best idea is to implement the paging in your UITableView delegate cellForRowAtIndexPath method, instead of doing this on the UIScrollView scrollViewDidEndDecelerating delegate method.

Here is my implementation of paging and I believe it should work perfectly for you too:

First of all, I have an implementation constants related to the paging:

//paging step size (how many items we get each time)
#define kPageStep 30
//auto paging offset (this means when we reach offset autopaging kicks in, i.e. 10 items before the end of list)
#define kPageBegin 10

The reason I'm doing this is to easily change the paging parameters on my .m file.

Here is how I do paging:

- (UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    NSUInteger row = [indexPath row];
    int section = indexPath.section-1;

    while (section>=0) {
        row+= [self.tableView numberOfRowsInSection:section];
        section--;
    }

    if (row+kPageBegin>=currentItems && !isLoadingNewItems && currentItems+1<maxItems) {
        //begin request
        [self LoadMoreItems];
    }
......
}

currentItems is an integer that has the number of the tableView datasource current items.
isLoadingNewItems is a boolean that marks if items are being fetched at this moment, so we don't instantiate another request while we are loading the next batch from the server.
maxItems is an integer that indicates when to stop paging, and is an value that I retrieve from our server and set it on my initial request.

You can omit the maxItems check if you don't want to have a limit.

and in my paging loading code I set the isLoadingNewItems flag to true and set it back to false after I retrieve the data from the server.

So in your situation this would look like:

- (UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    NSUInteger row = [indexPath row];
    int section = indexPath.section-1;

    while (section>=0) {
        row+= [self.tableView numberOfRowsInSection:section];
        section--;
    }

    if (row+kPageBegin>=counter && !isDowloading) {
        //begin request
        isDowloading = YES;
        [self fetchEntriesNew:counter];
    }
......
}

Also there is no need to reload your whole table after adding the new rows. Just use this:

for (int i = 0; i < itemDelta; i++) {
    NSIndexPath *ip = [NSIndexPath indexPathForRow:i inSection:0];
    [rows addObject:ip];
}

[self.tableView beginUpdates];
[self.tableView insertRowsAtIndexPaths:rows withRowAnimation:UITableViewRowAnimationBottom];
[self.tableView endUpdates];

Question 3

A simple BOOL is enough to avoid repetitive calls:

BOOL isDowloading;

When the download is done, set it to NO. When it enters here:

 if (endScrolling >= scrollView.contentSize.height)
    {
        NSLog(@"Scroll End Called");
        NSLog(@"New Counter NOW is %d", counter);
        [self fetchEntriesNew:counter];
    }

put it to YES. Also don't forget to set it to NO when the requests fails.

Edit 1:

- (void)scrollViewDidEndDecelerating:(UIScrollView *)scrollView
{
    float endScrolling = scrollView.contentOffset.y + scrollView.frame.size.height;
    if (endScrolling >= scrollView.contentSize.height)
    {
        if(!isDowloading)
        {
           isDownloading = YES;
           NSLog(@"Scroll End Called");
           NSLog(@"New Counter NOW is %d", counter);
           [self fetchEntriesNew:counter];
        }
    }
}

And when you finish fetching, just set it to NO again.