Is it dangerous to use transaction log flush delay - e.g set innodb_flush_log_at_trx_commit to 0?

Question

Is it true that risks of having such an option turned on are actually higher than loosing just last N seconds of transactions?

Let me explain why I'm asking this question. Imagine that DB server just processed a transaction T, and modified a set of index pages as result of this. Few of these index pages are already flushed to disk, but few others aren't. But all operations made by T aren't flushed to on-disk transaction log yet. And now we turn off the power.

So when DB server restarts and tries to recover, it doesn't see any T operations in transaction log. But some of its indexes actually reflect these operations, since some index pages modified by T were flushed before power failure. I.e. as result, T is partially applied - and probably, in such a way that leads to inconsistent index structure (e.g. primary index of table X doesn't reflect any changes made by T, but some of its secondary indexes does).

So I wonder of something like this is really possible if I delay transaction log flushing, and if not, what database server actually does to prevent this (MySQL 5.5 w/InnoDB is the most interesting case for me).

Answer 1

Got a full answer on this question at Quora:

"Checkpoints (dirty page flushing) fsync transaction log too, to avoid the situation you mentioned. The background thread timer isn't the only source of log flushes. Do note, this would raise insane amount of warnings, as LSNs in pages would be ahead of your transaction log LSNs.

Generally it is safe to run InnoDB with transaction log flushing disabled, as long as you really don't need that last data (and if you run replicated setup, you have to deal with data missing on the master, that may exist on slave)."

Answer 2

This PDF from a Percona Live conference from a few years ago has a decent write-up (see page 55).

Short answer: value of 0 doesn't flush or sync on COMMIT, so it's not durable at all. At least with a value of 2, you flush on COMMIT; but even so, you only sync ever second, so that too is often an unacceptable compromise.

Answer 3

innodb_flush_log_at_trx_commit    
1  every commit log buffer flashed, synced,     no data loss
2  every commit log buffer flashed, not synced, data loss in OS crash or power loss
0  every second log buffer flashed, synced,     data loss on mysql crash, OS crash or power loss

I think if you data is not critical and your hardware is limited innodb_flush_log_at_trx_commit=0 can be used. But for money, banking or security solutions innodb_flush_log_at_trx_commit=1 must be used to not to lose any data.

Answer 4

Short answer - it is safe, but you may lose up to one second of committed transaction in case OS or MySQL crashes.

It's up to you to decide if it's acceptable for your application (e.g. financial software won't tolerate data loss, but a blog will survive if a comment gets lost).