Question

I have a MVC framework that has a Request class which handles all requests. I log each request that comes in the database and the columns for the request log table is as follows:

  1. ID (auto-increment)
  2. Controller
  3. Action
  4. Params (GET, POST, PUT, DELETE)
  5. Params (MVC params delimited by slash "/")
  6. Request Method (GET, POST, PUT, DELETE)
  7. Request Type (HTML, JSON, XML)
  8. Is Request Ajax (true/false)
  9. Referer URL
  10. Request URI
  11. Browser (User Agent Browser)
  12. Browser Version
  13. IP
  14. Platform
  15. Request Time (datetime)

Now that I think about it, if I have 100+ users online, each one making around 500+ requests a day I get 100*500 = 50,000 rows per day and per year it comes out around 18 Million.

Now after 1-2 years if I dont clear the log, will it be slower for users to insert their request into the table?

Is it bad idea to log every request?

Reason: the reason i want to log every request is to detect kind of DDOS attack so that if there are more than 10 requests per 30 seconds lock account for 20 minutes.

Was it helpful?

Solution

The amount of slowdown exhibited should mostly depend on the indexes you have on the table. Each record inserted will cause each index for that table to be updated, thereby creating more and more slow down as your table grows.

There are several ways to maintain performance over the long haul, the simplest being have your fresh data written to a MySQL Memory table for insertions and queries, then have a cronjob transfer the data to a permanent table.

There's also message queueing (ZeroMQ, RabbitMQ, etc) or a NoSQL database (MongoDB)

OTHER TIPS

Reason: the reason i want to log every request is to detect kind of DDOS attack so that if there are more than 10 requests per 30 seconds lock account for 20 minutes.

Why not periodically delete logs older than a day or so, then? You're presumably not going to rate limit an account based off two year old accesses.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top