Detecting the value 0x in a varchar column sourced from Excel

Question

I have a sql table that gets populated via SQLBulkCopy from Excel. The copy down is done using the Microsoft ACE drivers.

I had a problem with one particular file - when it was loaded down to sql, some of the columns (which appear empty in excel) contained an odd value.

For example, running this sql:

SELECT
    CONVERT(VARBINARY(10),MyCol),
    LEN(MyCol)
FROM MyTab

would return

0x, 0

i.e. - converting the value in the column to varbinary shows something, but doing length of the varchar shows no length. I realise that the value shown is the stem of a hex value, but its weird that its gets there, and how hard it is to detect.

Obviously I can just clear out the cells in Excel, but I really need to detect this automatically as end users will have the same issue. It is causing issues further down the line when the data gets processed. Its quite hard to trace the problem back from its eventual symptoms to being this issue in the source.

Other than the above conversion to varbinary to output in SSMS I've not come up with a way of detecting these values, either in Excel or via a SQL script to remove them.

Any ideas?

Answer 1

As Martin points out above, 0x is what you get when you convert an empty string. eg:

SELECT CONVERT(VARBINARY(10),'')

So the problem of detecting it obviously goes away.

I have to assume that there is some rubbish in the excel cell, that is being filtered out in the process of the write down by either the ACE driver or the SQLBulkCopy. Because there was something in the field originally, the value written is empty instead of null.

In order to make sure that everything is consistent in the data we'll need to do a post process to switch all empty values to nulls so that the next lots of scripts work.

Answer 2

This may help you:

-- Conversion from hex string to varbinary:

DECLARE @hexstring VarChar(MAX);
SET @hexstring = 'abcedf012439';
SELECT  CAST('' AS XML).Value('xs:hexBinary( substring(sql:variable("@hexstring"),     sql:column("t.pos")) )', 'varbinary(max)')
FROM (SELECT CASE SubString(@hexstring, 1, 2) WHEN '0x' THEN 3 ELSE 0 END) AS t(pos)
GO

-- Conversion from varbinary to hex string:

DECLARE @hexbin VarBinary(MAX); 
SET @hexbin = 0xabcedf012439; 
SELECT '0x' + CAST('' AS XML).Value('xs:hexBinary(sql:variable("@hexbin") )', 'varchar(max)'); 
GO

One method is to add a new column, convert the data, drop the old column and rename the new column to the old name.