SQL Server BULK IMPORT Truncation error

Question

I am trying to import a simple CSV file into a table on SQL Server 2017. The CSV file and table match each other in number of columns and data types. This is an extract of the CSV file:

UID,customerID,DateT,TypeID,AssociatedID,AttributeID,LandingPage,jsonT,ClientIPAddress
9150,2345,2020-09-01T00:00:36Z,1,,859,campaign01.html,{'product': 'string01', 'type': '', 'string011'},192.168.200.2
9151,2356,2020-09-01T00:02:30Z,1,,640,campaign02.html,{'product': 'string02', 'type': '', 'string022'},192.168.200.1

This is my table:

CREATE TABLE [dbo].[activity](
    [UID] [bigint] NOT NULL,
    [customerID] [bigint] NOT NULL,
    [DateT] [datetime] NULL,
    [TypeID] [bigint] NULL,
    [AssociatedID] [float] NULL,
    [AttributeID] [bigint] NULL,
    [LandingPage] [varchar](256) NULL,
    [jsonT] [varchar](1024) NULL,
    [ClientIPAddress] [varchar](64) NULL,
)
GO

This is my import statement:

BULK INSERT dbo.activity
FROM 'C:\tmp\activity.csv'
WITH (
    FIRSTROW = 2,
    FIELDTERMINATOR = ',',
    ROWTERMINATOR = '0x0a',
    BATCHSIZE = 1000,
    MAXERRORS = 2
)
GO

I keep getting an error saying the last column (the client IP) is to be truncated. I checked all the values and they have nothing but IPs. Also the error is not on all rows and I cannot see for my life any difference between one IP format or the other, they are just IPs! I tried simply making the ClientIPAddress field bigger (just for the sake of it) but I just keep getting the same error.

Msg 4863, Level 16, State 1, Line 1
Bulk load data conversion error (truncation) for row 2, column 9 (ClientIPAddress).
Msg 4863, Level 16, State 1, Line 1
Bulk load data conversion error (truncation) for row 5, column 9 (ClientIPAddress).
Msg 4863, Level 16, State 1, Line 1
Bulk load data conversion error (truncation) for row 6, column 9 (ClientIPAddress).
Msg 4865, Level 16, State 1, Line 1
Cannot bulk load because the maximum number of errors (2) was exceeded.
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".

Any idea on what I could check or what I am doing wrong would be highly appreciated.

Thank you!

Answer 1

The import worked for me with your sample data. But if the real data has commas in that "jsonT" column, it's going to get the data in the wrong columns. BULK INSERT interprets that comma as part of the CSV format, and then starts putting everything after it in the last column (ClientIPAddress):

If you have control over the creation of the file, you could use a different FIELDTERMINATOR, like pipe (|). You'll need to pick something that you're sure won't be in the source data, per the docs:

To be usable as a data file for bulk import, a CSV file must comply with the following restrictions:

• Data fields never contain the field terminator.
• Either none or all of the values in a data field are enclosed in quotation marks ("").

UID|customerID|DateT|TypeID|AssociatedID|AttributeID|LandingPage|jsonT|ClientIPAddress
9150|2345|2020-09-01T00:00:36Z|1||859|campaign01.html|{'product': 'string01', 'type': '', 'string011'}|192.168.200.2
9151|2356|2020-09-01T00:02:30Z|1||640|campaign02.html|{'product': 'string02', 'type': '', 'string022'}|192.168.200.1

Then your BULK INSERT statement looks like this:

BULK INSERT dbo.activity
FROM 'C:\temp\activity.csv'
WITH (
    FIRSTROW = 2,
    FIELDTERMINATOR = '|',
    ROWTERMINATOR = '0x0a',
    BATCHSIZE = 1000,
    MAXERRORS = 2
);

If you don't have control over the creation of the export file, you'll unfortunately need to:

• do some custom pre-processing on it to be able to use it with BULK INSERT
• use a different import technology