Should I use Log file or DB to track impression?

Question

I have a picture (similar to an ad) in an iframe that will be placed in multiple sites. Should I use the server log file to find out the impression? or should I use DB to keep track of each impression?

which way is faster and can handle large volume of traffic? thanks

Answer 1

The answer for this question really depends on what kind of database you are using, and how busy the web servers disk and the database really are in your deployment environment;

• If you cat a log entry to a file each time an impression is done this is pretty fast. You can daily move this file to a new file and send it to some back-end system to process it and you would most likely never loose an entry (unless you encounter a disk crash)
• A regular mysql database with a table entry might be overkill for this unless you want to use the same table for doing queries on the data. It would probably also scale well, but you could end up with a lot of entries here if you have lots of traffic.
• Using a nosql database for this could be a good match, and this would probably scale it up to scaling like twitter - though most sites aren't like Twitter so it is probably overkill for your needs :)

If you have a regular website you can probably get away with doing it the first way as it is simple. If it does not, you haven't spent a lot of time on it.

Answer 2

Sounds reasonable to me. The simple way with logs could be e.g. to use grep to find the correct calls from the log and count them.

As Knubo said, compressing entries that have already been processed makes sense, as with 10M+/hour you could be looking at quite a log-file :) So e.g. daily do a process which:

1. Starts a new log-file (log rotating)
2. Counts correct entries from yesterdays log-file
3. Archives the processed log-file (gzip, bzip2, 7zip etc.)