Exact meaning of “[ERROR] InnoDB: The age of the last checkpoint is NNN, which exceeds the log capacity NNN”

dba.stackexchange https://dba.stackexchange.com/questions/286996

  •  17-03-2021
  •  | 
  •  

Question

My understanding is that InnoDB will write transactions to the redo logs, and will run out of space in the redo logs if dirty pages exist in the buffer pool with LSNs lower than the lowest LSN observed in the redo logs.

Therefore, I can see two possible meanings of the error message

[ERROR] InnoDB: The age of the last checkpoint is NNN, which exceeds the log capacity NNN:

  1. InnoDB is simply warning that the database was forced to freeze up and write some dirty pages to disk, which is happening because the redo logs are not as large as they should ideally be (as specified by the innodb_log_file_size variable).

  2. InnoDB needed to write a transaction to the redo logs, and even after writing all dirty pages to disk, it was still not possible to evict records from the redo logs in order to make space for the new transaction. Therefore the transaction had to be aborted, since it was not possible to write it into the redo logs in order to allow the commit to complete. Perhaps this was due to a single transaction being larger than the entire redo log capacity, or maybe there was another reason related to the way that fuzzy checkpointing works. Therefore this should be treated as a real ERROR that caused a transaction to fail, and not just as a warning about redo log capacity being lower than the ideal amount.

Which of these two possibilities could have resulted in the ERROR?

This error occurred while shutting down a MariaDB 10.5.8 instance that had been running for several months, with innodb_buffer_pool_size = 2GB and innodb_log_file_size = 256MB. It's confusing me that this only happened during a shutdown. There must be something about the sharp checkpointing at shutdown that triggers the error in a way that ongoing fuzzy checkpointing does not?

Here is the full shutdown log: (done with innodb_fast_shutdown = 0)

[Note] /usr/sbin/mariadbd Normal shutdown
[Note] Event Scheduler: Purging the queue. 0 events
[Note] InnoDB: FTS optimize thread exiting.
[Note] InnoDB: to purge 13 transactions
[Note] InnoDB: Starting shutdown...
[Note] InnoDB: Dumping buffer pool(s) to /mysql/ib_buffer_pool
[Note] InnoDB: Restricted to 32260 pages due to innodb_buf_pool_dump_pct=25
[Note] InnoDB: Buffer pool(s) dump completed at 210314 11:36:56
[ERROR] InnoDB: The age of the last checkpoint is 241591543, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 324335210, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 411544367, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 510534114, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 545226678, which exceeds the log capacity 241590068.
[Note] InnoDB: Waiting for change buffer merge to complete number of bytes of change buffer just merged: 175298332
[Note] InnoDB: Shutdown completed; log sequence number 197482953077338; transaction id 35190036382
[Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
[Note] /usr/sbin/mariadbd: Shutdown complete

The above log was for the master. The slave, which was shut down first (also done with innodb_fast_shutdown = 0), also showed similar errors:

[Note] /usr/sbin/mariadbd Normal shutdown
[Note] Event Scheduler: Purging the queue. 0 events
[Note] InnoDB: FTS optimize thread exiting.
[Note] InnoDB: to purge 27 transactions
[Note] InnoDB: Starting shutdown...
[Note] InnoDB: Dumping buffer pool(s) to /mysql/ib_buffer_pool
[Note] InnoDB: Restricted to 32260 pages due to innodb_buf_pool_dump_pct=25
[Note] InnoDB: Buffer pool(s) dump completed at 210314 11:31:34
[ERROR] InnoDB: The age of the last checkpoint is 241590247, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 322757457, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 406272546, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 496970738, which exceeds the log capacity 241590068.
[ERROR] InnoDB: The age of the last checkpoint is 541325450, which exceeds the log capacity 241590068.
[Note] InnoDB: Waiting for change buffer merge to complete number of bytes of change buffer just merged: 175621505
[Note] InnoDB: Shutdown completed; log sequence number 197499955777400; transaction id 35203811823
[Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
[Note] /usr/sbin/mariadbd: Shutdown complete

Perhaps the errors are related to the [Note] InnoDB: to purge 13 transactions line? I'm not sure what transaction purging is in this context.

No correct solution

Licensed under: CC-BY-SA with attribution
Not affiliated with dba.stackexchange
scroll top