Best way to update over million records in mysql

Question

I have a mysql table having over a million rows with user information. I want to add a column 'password' in that table, randomly generate password and update all the records (using PHP). What is the fastest way to perform this task? Thanks in advance.

Answer 1

A possible scenario

1. Generate all passwords into a file with your php script
2. Create temp table and use LOAD DATA INFILE (which happens to be the fastest method to import data from file) to load data from the password file.
3. Alter your table and add password column
4. Use UPDATE with join to update password column in your original table from temp table
5. Drop temp table

Answer 2

Keep in mind that the operation for altering the table might take very long time, I would recommend to sepearate this two steps.

1) Alter the table, here I would recommend the following steps (pseudocode):

• create new_table like old_table
• alter new_table add column password
• insert into new_table (columns) select * from old_table
• rename old_table to old_table_bck, new_table to old_table
• drop old_table_bck

At this point you have your original table with the new column.

2) Now after you changed your structure you can populate the new column 'password' with your php. If you are using InnoDB as storage engine time doesn´t matter since you are not locking the table with your updates. If you do this within transactions I would suggest to break the update process down to smaller transactions instead of inserting one large bulk.

If you can´t take (minimal) downtime, i would suggest that you look at pt-online-schema-change, we use this tool on major databases where we can´t effort any downtime in order to make schema changes while database are running. It roughly performs above steps (1) and additionally ensures with the help of triggers that data inserted on the original table while performing the alter process is also written to the altered table.