Python/SQLite updating columns

Question

I am trying to update a couple of columns in my SQLite database using:

UPDATE db SET Col1 = 0 WHERE Col1 IS NULL;
UPDATE db SET Col2 = 0 WHERE Col2 IS NULL;

However this seems to take a very long time to even update one column. I have resorted to using:

CASE WHEN Col1 IS NULL THEN 0 ELSE Col1 END

in my SELECT queries which works a lot quicker, however is there a reason why the UPDATE method is so slow? (I only have 670K rows in my database)

NOTE: My computer is fairly high-end and when the UPDATE is running, there doesn't seem to be much pressure on my desktops resources.

Answer 1

UPDATEing requires starting a transaction, then keeping a write-ahead-log of the changes that the update would make, then ensuring that succeeds before then writing changes back to the main table, and then ensuring all of that completes successfully...

SELECTing is reading then doing a simple "in memory if" to swap the values over for each row... so it's bound to be faster (plus disk reads are almost always a lot faster than disk writes (which a SELECT really doesn't require))...

Answer 2

Do you have the tools available to do an "explain" on your query :to see what (if any) indexes your sql query is using. if your sql query is resulting in a full table scan over 670K rows, then it could be slowish.

if you are using ubuntu or a similiar based linux distribution, Sqliteman will do this for you.