Java updating large binary file

Question

So im developing Android app, sort of a book reader. The books are stored as a 500mb+ binary files with certain objects stored in it, and in a tiny binary file that contains mappings to particular objects in the big file. I need to find most efficient way to perform an update of certain publication file. The incomming update package will contain relatively small file with offsets + lenghts of objects that need to be deleted from the original big file, a set of new objects that will be added at the end of big file, and the new mapping after the update is done obviously.

Ive googled ALOT, but still I cant find out whether theres any better solution than copying the content i want to be kept into a new file. Is ther any more efficient way to do this? I mean the aproach i mentioned requires double disc space as the main big file takes, and it doesnt seem to be any fast way either.

Thx in advance, Mac

Answer 1

Well I figured out a solution that doesnt require another copy of file. I simply take first offset to delete, jump to offset + its lenght, take a bounch of bytes lets say minimum (distance to next deleted object) and maximum 1k, paste it at the offset of the deleting entry and look for another one to delete, nto sure if i explained it in a proper way but well, asnwering your own question often leads to that kind of issue xD

Answer 2

This is a common question.

Normally, this kind of file is stored in certain "pattern".

That is to say, they are serialized objects on permanent storage.

So, as long as you know the "pattern" - to be more accurate - the structure of the data object, you can use RandomAccessFile to read particular part of this file.

Take an very easy example:

1. You load the index of contents(like several mega bytes) into memory
2. When you want to look up the a word, you first search in the index of contents, and find the "address" of the explanation of this word in the huge file
3. You read the particular part of the file and get the detailed content
4. Done

The similar approach might also applied to the updating of the file, please refer to the java doc of random access file.