MultiThread insert mongodb made stale data

Question

In my test Envinroment: node1:shard1 primary,shard2 primary node2:shard1 secondary,shard2 secondary node3:shard1 arbiter,shard2 artbiter I wrote a multi-thread to concurrently write the mongo replicat set shard,after 1 hour(the primary had 6g data) I found the secondary status is :recovering I checked the secondary log,said:stale data from primary oplog So the reason was my write request very frequent?then render the secondary cannot replicate in time? or other reasons? I'm puzzling... Thanks in advance

Answer 1

This situation can happen if the size of the OpLog is not sufficient to keep a record of all the operations occurring on the primary, or the secondary just can't keep up with the primary. What will happen in that case is the position in the OpLog where the secondary is will be overwritten by the new inserts from the primary. At this point the secondary will report that it's status is Recovering and you will see a RS102 message in the log, indicating that it is too stale to catch up.

To fix the issue you would need to follow the steps outlined in the documentation.

In order to prevent the problem from happening in the future, you would need to tune the size of the OpLog, and make sure that the secondaries are of equivalent hardware configurations.

To help tune the OpLog you can look at the output of db.printReplicationInfo() which will tell you how much time you have in your OpLog. The documentation outlines how to resize the OpLog if it is too small.