How to catch and ignore ActiveRecord::RecordNotUnique exception in Ruby

Question

I had ruby script job running daily for loading data from one table to another until there were duplicate key records found in the source table since that table had no constraints. My target table rejected those records cuz the target table had primary key constraints on columns (head_date, center_id, site_url). My program just halted and showed the error below on the command line. :-(

in `async_exec': PG::Error: ERROR:  duplicate key value violates unique constraint "tst_data_pkey" (ActiveRecord::RecordNotUnique)

I can expect the source table to have other nasty records like this. How can I continue processing and move on to the next record after catching this active record exception?

Below is the code for my updates/inserts:

class SiteSection < ActiveRecord::Base

  SiteSection.establish_connection(
        :adapter  => 'postgresql',
        :host     => 'hoster-of-hosts.com',
        :database => 'super-inven',
        :username => 'user',
        :password => 'pass'
                      )
  self.table_name = 'master.target_tbl'  # << insert into this table
end

conn.query("select * from source_tbl") do |row|
  siteData = SiteSection.find_or_initialize_by_head_date_and_center_id_and_site_url(row[:date_c], row[:com_id], row[:site_link])

  siteData[:head_date]      = row[:date_c]
  siteData[:center_id]     = row[:com_id]
  siteData[:site_url]          = row[:site_link].nil? ? 'unknown' : row[:site_link]
  siteData[:people_cnt]      = row[:persons].nil? ? 0 : row[:persons]
  siteData[:ips]     = row[:ip_adds].nil? ? 0 : row[:ip_adds]

  siteData.save
i = i+1
puts "finished: #{i}" if i % 10000 == 0
end
conn.close

Answer 1

You can use a

begin

rescue => e

end

like this:

class SiteSection < ActiveRecord::Base

  SiteSection.establish_connection(
        :adapter  => 'postgresql',
        :host     => 'hoster-of-hosts.com',
        :database => 'super-inven',
        :username => 'user',
        :password => 'pass'
                      )
  self.table_name = 'master.target_tbl'  # << insert into this table
end

conn.query("select * from source_tbl") do |row|
  siteData = SiteSection.find_or_initialize_by_head_date_and_center_id_and_site_url(row[:date_c], row[:com_id], row[:site_link])

  siteData[:head_date]      = row[:date_c]
  siteData[:center_id]     = row[:com_id]
  siteData[:site_url]          = row[:site_link].nil? ? 'unknown' : row[:site_link]
  siteData[:people_cnt]      = row[:persons].nil? ? 0 : row[:persons]
  siteData[:ips]     = row[:ip_adds].nil? ? 0 : row[:ip_adds]

  begin
    siteData.save
  rescue => e
    puts e.message
    puts "Error happened but I'll just keep chuggin along"
  end
i = i+1
puts "finished: #{i}" if i % 10000 == 0
end
conn.close

The rescue => e is expecting to catch an error. It will swallow that error by not letting it bubble up. Your code will continue to run without the exception crashing it.

Answer 2

If you have unique key you should use it - something like this:

i = 0
puts 'loading records'
conn.query("select * from source_tbl") do |row|
  rec = SiteSection.where(
    :head_date => row[:date_c],
    :center_id => row[:com_id],
    :site_url => (row[:site_link] || 'unknown')
  ).first_or_initialize

  rec.people_cnt = (row[:persons] || 0)
  rec.ips => (row[:ip_adds] || 0)

  rec.save!

  i += 1
  print '.' if i % 10000 == 0
end
puts "done\ntotal records: #{i}"

This way you will save a record (new) or update ones (if found). No ActiveRecord::RecordNotUnique should be here if you didn't change you pkey!