Lots of questions about file I/O (reading/writing message strings)

Question

For this university project I'm doing (for which I've made a couple of posts in the past), which is some sort of social network, it's required the ability for the users to exchange messages.

At first, I designed my data structures to hold ALL messages in a linked list, limiting the message size to 256 chars. However, I think my instructors will prefer if I save the messages on disk and read them only when I need them. Of course, they won't say what they prefer, I need to make a choice and justify the best I can why I went that route.

One thing to keep in mind is that I only need to save the latest 20 messages from each user, no more.

Right now I have an Hash Table that will act as inbox, this will be inside the user profile. This Hash Table will be indexed by name (the user that sent the message). The value for each element will be a data structure holding an array of size_t with 20 elements (20 messages like I said above). The idea is to keep track of the disk file offsets and bytes written. Then, when I need to read a message, I just need to use fseek() and read the necessary bytes.

I think this could work nicely... I could use just one single file to hold all messages from all users in the network. I'm saying one single file because a colleague asked an instructor about saving the messages from each user independently which he replied that it might not be the best approach cause the file system has it's limits. That's why I'm thinking of going the single file route.

However, this presents a problem... Since I only need to save the latest 20 messages, I need to discard the older ones when I reach this limit.

I have no idea how to do this... All I know is about fread() and fwrite() to read/write bytes from/to files. How can I go to a file offset and say "hey, delete the following X bytes"? Even if I could do that, there's another problem... All offsets below that one will be completely different and I would have to process all users mailboxes to fix the problem. Which would be a pain...

So, any suggestions to solve my problems? What do you suggest?

Answer 1

You can't arbitrarily delete bytes from the middle of a file; the only way that works is to rewrite the entire file without them. Disregarding the question of whether doing things this way is a good idea, if you have fixed length fields, one solution would be to just overwrite the oldest message with the newest one; that way, the size / position of the message on disk doesn't change, so none of the other offsets are affected.

Edit: If you're allowed to use external libraries, making a simple SQLite db could be a good solution.

Answer 2

You're complicating your life way more than you need to.

If your messages are 256 characters, then use a array of 256 characters to hold each message.

Write it to disk with fwrite, read with fread, delete it by changing the first character of the string to \0 (or whatever else strikes your fancy) and write that to disk.

Keep an index of the messages in a simple structure (username/recno) and bounce around in the file with fseek. You can either brute-force the next free record when writing a new one (start reading at the beginning of the file and stop when you hit your \0) or keep an index of free records in an array and grab one of them when writing a new one (or if your array is empty then fseek to the end of the file and write a complete new record.)

Answer 3

I want to suggest another solution for completeness' sake:

Strings should be ending with a null-byte character, "hello world\0", so you might read the raw binary data until reaching "\0". Other datatypes have fixed bits, beware of byteorder (endian).

Also you could define a payload before each message, so you know its string length:

"11hello world;2hi;15my name is loco"

Thus making it possible to treat raw snippets like data fields.