MYSQL automatically insert csv files from folder using mysqlimport

Question

I would like to import around 3000 CSV files from a specified "folder" automatically at a designated time. Everyday the CSV files will be updated to include new data.

I understand that I should use the command line tool "mysqlimport" and --replace option, (as sometimes old data will be altered)

Answer 1

load data local infile 'uniq.csv' into table tblUniq(field1, field2, field3)
fields terminated by ','
enclosed by '"'
lines terminated by '\n'

This is an optional solution, the only thing you'll need is an PHP\Perl\Python script to itreate over all of the files, and insert them one by one

PHP code:

<?php

    $basedir = "/path/to/dir"
    if ($handle = opendir($basedir)) {
      while(false !== ($entry = readdir($handle))) {
        $query = "LOAD DATA LOCAL INFILE $basedir$entry
                    INTO TABLE tableName
                    FIELDS
                    TERMINATED BY ','
                    OPTIONALLY ENCLOSED BY '\"'
                    ESCAPED BY '\\'
                    LINES TERMINATED BY '\n'
                    IGNORE 1 LINES");
        if(DEBUG) { echo $query . "\n"; }
        if(!mysql_query($query)) {
          die('MySQL error: ' . mysql_error());
        }
      }
    } else {
      echo "Could not open $basedir";
    }
    ?>