How to store information? Database vs Data-Structure vs Log files [closed]

Question

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 9 years ago.

Improve this question

Recently I came across a scenario in a question:

There are n websites with n pages each and n users visiting the sites....each visit of the user has to be saved and the pages he/she has visited ( not mentioned whether in database or log files, so it's up to the developer )

I decided to go on with it and do something in datastructures but when I discussed this thing with a friend of mine, he said, we can save it in database and this logically sounded correct too.

So we have 3 ways of storing anything in general...log files data-structure database

Now I am really confused, when should one go with data-structures, databases or simply log files, not only for this particular scenario but in a generic way?

What's the real difference?

---

I understand that this question is primarily opinion based but couldn't get a concrete result while browsing!

Answer 1

Log files are often / usually output-only - these files will rarely, if ever, get read, possibly only read manually. Some types of files may have random access, allowing you to fairly efficiently find a given record by a single index (through binary search), but you can't (easily) have multiple indices on the data in a single file, which is a trivial task for a database. If you just want to log something for manual processing later, a log file can work fine (even if a database can work too).

Databases is the standard in the industry, in that they provide you with persistence, efficient reading and writing, a standard interface and redundancy (but of course they need to be set up correctly).

A pure data structure solution typically doesn't consider persistent storage, as in making sure your data is kept when the program stops running for some reason. If you do want to write to and read from persistent storage, this will often come with a fair bit of complexity to do efficiently and regularly. And multiple / complex indices is a bit of a hassle to cater for. That's not to say data structures can't be used with persistent storage - databases are built using data structures and some data structures are specifically made for disk reads and writes. But you don't want to be figuring this out on a low level - it's best to just let a database take care of it if you need persistence.

You could also combine data structures and databases, using the database as persistent storage and use the data structure to cache the results so you only need to do (slower) writes to the database and you can do (faster) reads from the data structure. This is not uncommon in large systems with external databases. Although anything more complex than a standard map data structure is probably overcomplicating your cache and make indicate a bigger problem with your design.

---

What you have there sounds like an interview question, for which they may be expecting a data structure solution and simply saying "use a database" may be frowned upon. However, if it's a system design question, you'd almost certainly need to include some sort of a database in your design instead of concerning yourself with data structures.