Doctrine Data Import - Common way to do a fallback?

Question

Rather than backing up the database, modifying data in it, and possibly restoring your database if anything goes wrong, use Doctrine2 or SQL's explicit transaction mechanism.

Essentially, in an explicit transaction you can do multiple insertions, updates, deletions, and more. In an explicit transaction the developer has full control over what data is committed to the database and when. And, if the developer chooses to do so, you can even rollback the explicit transaction so that the database is in the state it was in before the transaction began.

Concepts & More Info

MySQL explicit transaction reference: https://dev.mysql.com/doc/refman/5.0/en/commit.html

Doctrine2 explicit transaction reference: http://docs.doctrine-project.org/en/2.0.x/reference/transactions-and-concurrency.html.

Sample code

I've put together some sample code that reads from a CSV file, customers.csv, and imports that data into the database. Only if the data is inserted properly will it be committed to the database. If something goes wrong the database is rolled back to its previous state.

I have used two beginTransaction()s, however, the inner most one is not necessary for this code sample to work properly. The inner most one is there as a reminder. In you're actual implementation, you probably call a function to import the csv data (rather than copy/pasting it into this block), and the inner most beginTransaction(), commit(), and rollback() should wrap around whatever function you use to import your CSV data.

$className = 'Customer';
$con = $em->getConnection();
$tableName = $em->getClassMetadata($className)->getTableName();

// start an explicit transaction
$con->beginTransaction();
try {
    // remove the old data from the customers table
    $plat = $con->getDatabasePlatform();
    $con->executeQuery('SET FOREIGN_KEY_CHECKS = 0;');
    $con->executeQuery('DELETE FROM '.$tableName);
    $con->executeQuery('SET FOREIGN_KEY_CHECKS = 1;');

    // start another explicit transaction
    $con->beginTransaction();
    try {
            // import new customer data from `customers.csv`
            if (false !== $handle = fopen("customers.csv", "r")) {
                    // read in a row of CSV data
                    while (false !== $data = fgetcsv($handle)) {
                            // $data[0] is the first column in the csv file
                            // and it holds the customer's name.
                            $customer = new Customer();
                            $customer->setName(trim($data[0]));
                            $em->persist($customer);
                    }
            }
            // commit the imported data to the database
            $em->flush();
            $con->commit();
    } catch(\Exception $e) {
            $con->rollback();
            throw $e; // rethrow to restore database to its original state.
    }

    // If we made it here, then all rows in the customer data table were
    // removed and all new customer data was imported properly. So, save
    // everything to the database.
    $em->flush();
    $con->commit();
} catch (Exception $e) {
    // If we made it here, then something went wrong and the database is in
    // an unstable state. So, rollback to the previously stable state.
    $con->rollback();
}

Caveats

Within explicit transactions (everything between beginTransaction() and commit() or rollback()) you should not do data definition language (DDL) statements--e.g., ALTER TABLE, TRUNCATE TABLE etc., because all DDLs perform an implicit commit to the database, and that means rollback() will not perform as expected. I've written more about that here: How to truncate a table using Doctrine 2?

If you read the above code carefully, you will notice that I used DELETE FROM rather than TRUNCATE because TRUNCATE is a DDL statement. It's important to know that DELETE FROM is not equivalent to TRUNCATE. DELETE FROM does not reset AUTO_INCREMENT. So, if your old customer table has 1032 entries in it and you run the above code, your newly inserted customer's will start from an auto increment value of 1033.

How do you work around this? Well, first off--you don't have to reset the auto increment value. All that matters to the database is that the id field is unique. But, if you really must reset the auto increment value, then I'd say you have two choices: explicitly set the customer.id as you import the data, or perform an ALTER TABLE statement. An ALTER TABLE statement sounds easy but it can easily become problematic because it is another DDL statement and it will perform an implicit commit to the database--that's the same reason why the above code uses DELETE FROM instead of TRUNCATE. Based on what I've read about your particular situation, I would explicitly set the id attribute as I imported the data from a CSV file.

Remember to test your implementation thoroughly! Transactions can get messy.