Uploading CSV File to Rails 4 DB via rake task

Question

I am trying to import data from a csv fle using a rake task (rake data:import) and am getting errors. I have been teaching myself rails the past few months but after a day and a half of googling I found many solutions to this, none of which i could get to work.

Here is where I stand so far with my rake file:

require 'CSV'
namespace :data do
  desc "Import teams from csv file"
  task :import => [:environment] do
    file=IO.read('filepath of my csv').force_encoding("ISO-8859-1").encode("utf-8", replace: nil)

    CSV.foreach(file, :headers => true) do |row|
      product.create ([
                       :name => row['name'],
                       :rating => row['rating'],
                       :year => row['year'],
                       :country => row['country'],
                       :state_or_province => row['state_or_province']]
                      )
    end

  end
end

I am not getting any specific error (as far as I can tell). The output which confuses me is:

tasks/dataimport.rake:7:in `block (2 levels) in <top (required)>'

Is that output indicative of anything specific going on?

Answer 1

There's a lot going wrong here so I'll start at the top.

CSV.foreach is meant to open a file and iterate over it all at once. The first argument to CSV.foreach should be a file name not the file's content. That means that this:

CSV.foreach(file, :headers => true) do |row|

goes wrong because file is a string containing the CSV data rather than the file name that CSV.foreach expects. Since you're converting from Latin-1 text to UTF-8, you'll want to get CSV.foreach to deal with that for you and you can use the :encoding option for that:

This method also understands an additional :encoding parameter that you can use to specify the Encoding of the data in the file to be read. [...] For example, encoding: "UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.

Putting that together, we have:

CSV.foreach('filepath of my csv', :headers => true, :encoding => 'ISO-8859-1:UTF-8') do |row|

Once you get the CSV reading and iteration going, you'll see errors like:

NameError: undefined local variable or method `product' for ...

You'd get a NameError because there is no product defined anywhere in your rake task. I suspect that you mean to say Product.create, that would try to create a new instance of your Product model. Ruby is case sensitive so product and Product are different things and Product would be the class.

Once the NameError is dealt with, you'll see complaints like this:

NoMethodError: undefined method `keys' for [{ ... }]:Array

You'll get the NoMethodError because Product.create wants to see a Hash of attributes and their values, not an Array which contains a Hash. You would want to say:

Product.create(
  :name => row['name'],
  :rating => row['rating'],
  :year => row['year'],
  :country => row['country'],
  :state_or_province => row['state_or_province']
)

Of course, if your row contains only those five values, then just hand the whole row to create:

Product.create(row.to_hash)

and if row contains (or might contain) a bunch of other stuff that you don't want create to see, use Hash#slice to grab just the part of row that you're interested in:

Product.create(row.to_hash.slice(*%w[name rating year country state_or_province]))

Note that %w[...] builds an array of strings from a white-space delimited list so these are the same:

%w[a b]
['a', 'b']

Then the splat (*) removes the array wrapper so these are the same:

row.to_hash.slice(*%w[name rating year country state_or_province])
row.to_hash.slice('name', 'rating', 'year', 'country', 'state_or_province')

You can use whichever form is easier on your eyes.

Also note the to_hash calls in there. Your row will be a CSV::Row object, calling to_hash on it will give you the row as a Hash.

---

That should leave your entire rake task looking like this:

CSV.foreach('filepath of my csv', :headers => true, :encoding => 'ISO-8859-1:UTF-8') do |row|
  Product.create(row.to_hash)
end

or

CSV.foreach('filepath of my csv', :headers => true, :encoding => 'ISO-8859-1:UTF-8') do |row|
  Product.create(row.to_hash.slice(*%w[name rating year country state_or_province]))
end

You might want to add some error handling to those create calls too.