iOS storage issues with preloaded content in NSBundle and delta updates from API

Question

I'm working on a few apps currently that have a similar problem. I'll try and explain it by detailing what I've done in the past and why that isn't a good solution.

Let's say I have an app that has a list of places and some photos of those places. When a customer first installs the app, all of the places and photos should be loaded and ready to work offline. However, if the content is changed via an online CMS, then the app should download the new data and load it (if the app is online).

The way this works currently is that the CMS generates a zip file containing images and a full dump of all the data as JSON in a txt file. The API allows you to specify a timestamp which means the zip will only contain data modified / added from that date. Before the app is uploaded to the App Store, I download the full zip file (timestamp = 0) and store it within the app bundle. When the app is first run, the zip is copied from [NSBundle mainBundle] to the Documents directory, unzipped, the JSON is imported into CoreData/SQLite, and then the zip is deleted. Next, an internal timestamp is checked and an API call is made (if online) to see if there is any new content; if there is, a new zip file with the changes is generated, downloaded, unzipped, imported, deleted.

The problem with this approach is more data is stored than there should be. If the initial zip is 300MB, then the customer must have around 600MB free after installation (which is a big wifi-only download) in order to copy the zip and unzip the contents. After that first run, you have all the unzipped data taking up ~310MB but the 300MB you no longer need is stuck in the bundle. Similarly, any downloads have to have 2x the space (1x for the zip file, 1x for the unpacked files). This is only temporary (as you'll delete the zip) but still not ideal.

The only solution I can see to limit this somewhat is to store all of the initial assets in the bundle but not in a zip file. The JSON that is imported would specify that each of those assets are bundle assets so when loading the images, get them from there. If that flag wasn't set, it should get them from the Documents directory (as they've been downloaded since). That doesn't solve the issue of downloading a zip file of updates though. Any solution would have to contain the initial content in the bundled app in some way (despite the fact this puts it way over the 100MB OTA limit) as I want the customer to be able to use the app immediately without having to update the internal database; nobody wants to download an app and then wait 5 minutes for it to download a huge update.

Does anybody have any better ideas? I'm keen to know how other developers are performing this task in their apps.

Answer 1

I assume most of your content is images + text. Is it maybe possible to keep a compact index file which would power any search queries in your app and be able to identify which pieces of content will be needed for a specific item to display? When you want to use the data from your ZIP file, selectively extract the required files to display that item - trading access time and computation power for disk space. (Objective-Zip should support extracting individual files)

If it helps maybe split the ZIP file into multiple ZIP files to speed up the selective extraction. The ability to do this type of segmentation greatly depends on the structure of your content and how it's being queried/accessed.

In regards to the update process, if the problem is:

• space needed for extraction: how about splitting the downloaded zip file into multiple individual files? During the update process: download chunk 1, extract chunk 1, download chunk 2.. (download + extraction of the previous chunk may happen in parallel speeding up the update process in general)

• space needed after extraction: incorporate the initial suggestion to the update ZIP files as well.

Obviously, when you increase the amount of independent ZIP files, your compression will be less effective in comparison with a single ZIP file.

I hope this sparks a few new ideas at least.