SQL Server 2012 Bulk Insert with carriage returns in text fields?

Question

I've seen variations of this question all over the place yet can't seem to get this to work. I need to be able to bulk insert data from a flat file where some of the text fields will contain carriage returns.

I have set the flat file up to be delimited by the caret ^ symbol. The Row delimiter is a vertical pipe and the column delimiter is a tab. Why does the import still fail when my text field has a carriage return in it?

I was under the impression that if the row/column delimiter was NOT a CR/LF then a delimited text field could contain a CR/LF (or single CR or single LF). How can I get the import to work? Thanks.

PS - the way I've been testing is to just take a table, export it to a flat file with delimiters set as above, insert a newline in a text field, then try to import the data again using the SQL Server Import Export Wizard in both directions. Here is the error message I see:

Error 0xc02020a1: Data Flow Task 1: Data conversion failed. The data conversion for column "Column 23" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".

Error 0xc020902a: Data Flow Task 1: The "Source - IVREJECTHD_txt.Outputs[Flat File Source Output].Columns[Column 23]" failed because truncation occurred, and the truncation row disposition on "Source - IVREJECTHD_txt.Outputs[Flat File Source Output].Columns[Column 23]" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.

Error 0xc0202092: Data Flow Task 1: An error occurred while processing file "C:\Users\bbauer\Desktop\IVREJECTHD.txt" on data row 2.

Error 0xc0047038: Data Flow Task 1: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Source - IVREJECTHD_txt returned error code 0xC0202092. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.

Answer 1

Bulk Insert can import embedded CR/LF pairs in text fields. Something else is going on with the raw data in your source at the specified column (23) on the second row. There are a number of causes for the "text was truncated" error. Some of them are touched on in this thread. One common cause which particularly bites those using the Wizard is not specifying the target column width. It doesn't matter if your target table is set up correctly; if the column width specified in the import isn't big enough, you'll get this error.

You might consider performing a bulk insert using T-SQL and a format file; if you need to repeatedly test your import process and refine it, it's a lot easier to make modifications and re-run.

Also, as noted in this answer, the embedded CR/LFs will be present even if the tools (e.g. Management Studio) aren't displaying them to you.