How to read text chunks from a huge text file?

Question

I am trying to read a text file containing characters in billions. Using the function contentOfFile is not working, as my application get crashed due to it.

So anybody please send me the sample code so that I get the chunks according to my requirement.Whichever i need i wanna get that one only. please reply as soon as possible.

Answer 1

I'm guessing this is an iOS app. In that case, you are likely hitting the memory limit by calling contentsOfFile: because that method is trying to read the entire contents of the file into a variable (memory). Remember that on iOS your app must play nice and if it decides to consume too much memory, then the watchdog process will kill your app to save the device from rebooting (which happens because there is no disk to swap to on iOS devices).

Have you had a look at NSFileHandle? NSFileHandle supports seeking within a text a file. With some simple iteration you can use the following to methods to seek within the file and read chunks of data:

- (NSData *)readDataOfLength:(NSUInteger)length;
- (void)seekToFileOffset:(unsigned long long)offset;

It might look something like this. Assume pathToFile is an NSString containing the path to the text file to be read in.

uint64 offset = 0;
uint32 chunkSize = 1024;     //Read 1KB chunks.
NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:pathToFile];
NSData *data = [handle readDataOfLength:chunkSize];

while ([data length] > 0)
{
     //Make sure for the next line you choose the appropriate string encoding.
     NSString *dataString = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] autorelease];

     /* PERFORM STRING PROCESSING HERE */

     /* END STRING PROCESSING */
     offset += [data length];

     [handle seekToFileOffset:offset];
     data = [handle readDataOfLength:chunkSize];
}

[handle closeFile];

Answer 2

A good idea is to look at the textedit source because I've opened massive files with it before and there should be a way to do it. Not sure why your app is crashing though. It shouldn't have a problem.