how to configure/design a table with 30,000 insertion in every 15 minutes?

Question

I am trying to design a database table which will be going to face 30,000 insertion in every 15 minutes. So this tables going to face millions of insertion.

A sample table is as follow-

CREATE TABLE `cdr` (`id` bigint(20) NOT NULL AUTO_INCREMENT,
                `dataPacketDownLink` bigint(20) DEFAULT NULL,
                `dataPacketUpLink` bigint(20) DEFAULT NULL,
                `dataPlanEndTime` datetime DEFAULT NULL,
                `dataPlanStartTime` datetime DEFAULT NULL,
                `dataVolumeDownLink` bigint(20) DEFAULT NULL,
                `dataVolumeUpLink` bigint(20) DEFAULT NULL,  
                `dataplan` varchar(255) DEFAULT NULL,  
                `dataplanType` varchar(255) DEFAULT NULL,  
                `createdOn` datetime DEFAULT NULL,  
                `deviceName` varchar(500) DEFAULT NULL,  
                `duration` int(11) NOT NULL,  
                `effectiveDuration` int(11) NOT NULL,  
                `hour` int(11) DEFAULT NULL,  
                `eventDate` datetime DEFAULT NULL,  
                `msisdn` bigint(20) DEFAULT NULL,  
                `quarter` int(11) DEFAULT NULL,  
                `validDays` int(11) DEFAULT NULL,  
                `dataLeft` bigint(20) DEFAULT NULL,  
                `completedOn` datetime DEFAULT NULL,   
            PRIMARY KEY (`id`),   
            KEY `msisdn_index` (`msisdn`),   
            KEY `eventdate_index` (`eventDate`)   
        ) ENGINE=MyISAM AUTO_INCREMENT=55925171 DEFAULT CHARSET=latin1;

How I can retrieve records if use comparison >= in case of date column(result in millions of record). So please help me to design such table, so i can use optimized select statements without effecting insertion operation(lot of indexes create problem for insertion after millions of records). Thank you.

Answer 1

A short answer:

1. Decide how do you want to keep history.

2. Use InnoDB engine.

3. Use partitioning to manage data in chunks easily and fast.

4. Implement current window partitioning - have one table for current set and 1 or more tables for archives and move partitions to older tables.

5. For fastest write performance - remove all indexes and constrains. Cast them after all data is loaded. If there is no pressure on data insertion - leave them, but know what will be the penalty.

6. Ensure you server can handle so many connections / data volume. Plan for next 2-3 years as hardware upgrades take a lot of time.

7. Do load testing, including overloading - know your limits.

And keep researching.