COPY CSV file with an ID column

Question

I have a rails application and I am trying to load data into it via PostgreSQL's COPY command. The file is CSV. The table maps to a Rails model and I need to retain the primary-key ID column so I can use model.for_each to play with the data--the table will have 5M+ rows.

My question is, how can I structure my CSV so that I can load data into the table and still allow the ID column to be there? It's a bummer because the only reason I need it is for the for_each method.

Regardless, I tried sending NULLs, as in:

NULL,col1,col2,col3,col4, etc.

But it doesn't work.

Thanks.

Answer 1

Passing in null for the primary key will never work no matter what options you have set for null string. You would be telling the backend to set the primary key to null which it will never allow no matter what the command to insert might be.

I really have no idea what you mean by retaining the primary key. That is something that is going be retained no matter what do. If you mean letting the DB pick the value for you and the primary key is a serial (auto-increment) then explicitly name all the columns but the primary key:

COPY country (colA, colB, colC) FROM '/usr1/proj/bray/sql/country_data'; -- leave out pkey

It also might be quicker to read the documentation on what null string options you would like to use instead of guessing possible values:

http://www.postgresql.org/docs/9.0/static/sql-copy.html

Answer 2

The default when using WITH CSV is an unquoted empty string, such as:

,col1,col2,col3,,col5

Which would create a record that looks like:

NULL,'col1','col2','col3',NULL,'col5'