SQL: break up one row into many (normalization)

Question

I am in middle of upgrading from a poorly designed legacy database to a new database. In the old database there is tableA with fields Id and Commodities. Id is the primary key and contains an int and Commodities contains a comma delimited list.

TableA:

id   | commodities
1135 | fish,eggs,meat    
1127 | flour,oil  

In the new database, I want tableB to be in the form id, commodity where each commodity is a single item from the comma delimited list in tableA.

TableB:

id   | commodity
1135 | fish  
1135 | eggs   
1135 | meat  
1127 | flour  
1127 | oil    

I have a function, functionA, that when given an id, a list, and a delimiter, returns a table with an id and item field. How can I use this function to turn the two fields from tableA into tableB?

(Note: I had trouble figuring out what to title this question. Please feel free to edit the title to make it more accurately reflect the question!)

Here is the function code:

ALTER  FUNCTION dbo.functionA
(
@id int,
@List VARCHAR(6000),
@Delim varchar(5)
)
RETURNS
@ParsedList TABLE
(
id int, 
item VARCHAR(6000)
)
AS
BEGIN
DECLARE @item VARCHAR(6000), @Pos INT
SET @List = LTRIM(RTRIM(@List))+ @Delim
SET @Pos = CHARINDEX(@Delim, @List, 1)
WHILE @Pos > 0
BEGIN
SET @item = LTRIM(RTRIM(LEFT(@List, @Pos - 1)))
IF @item <> ''
BEGIN
INSERT INTO @ParsedList (id, item)
VALUES (@id, CAST(@item AS VARCHAR(6000)))
END
SET @List = RIGHT(@List, LEN(@List) - @Pos)
SET @Pos = CHARINDEX(@Delim, @List, 1)
END
RETURN
END

Answer 1

Here's the link I posted as a comment:

http://www.sqlteam.com/article/parsing-csv-values-into-multiple-rows

Answer 2

You need a way to split and process the string in TSQL, there are many ways to do this. This article covers the PROs and CONs of just about every method:

Arrays and Lists in SQL Server 2000 and Earlier

You need to create a split function. This is how a split function can be used:

SELECT
    *
    FROM YourTable                               y
    INNER JOIN dbo.yourSplitFunction(@Parameter) s ON y.ID=s.Value

[I prefer the number table approach to split a string in TSQL](Arrays and Lists in SQL Server 2000 and Earlier) but there are numerous ways to split strings in SQL Server, see the previous link, which explains the PROs and CONs of each.

For the Numbers Table method to work, you need to do this one time table setup, which will create a table Numbers that contains rows from 1 to 10,000:

SELECT TOP 10000 IDENTITY(int,1,1) AS Number
    INTO Numbers
    FROM sys.objects s1
    CROSS JOIN sys.objects s2
ALTER TABLE Numbers ADD CONSTRAINT PK_Numbers PRIMARY KEY CLUSTERED (Number)

Once the Numbers table is set up, create this split function:

CREATE FUNCTION inline_split_me (@SplitOn char(1),@param varchar(7998)) RETURNS TABLE AS
   RETURN(SELECT substring(@SplitOn + @param + ',', Number + 1,
                    charindex(@SplitOn, @SplitOn + @param + @SplitOn, Number + 1) - Number - 1)
                 AS Value
          FROM   Numbers
          WHERE  Number <= len(@SplitOn + @param + @SplitOn) - 1
            AND  substring(@SplitOn + @param + @SplitOn, Number, 1) = @SplitOn)

GO 

You can now easily split a CSV string into a table and join on it:

select * from dbo.inline_split_me(';','1;22;333;4444;;') where LEN(Value)>0

OUTPUT:

Value
----------------------
1
22
333
4444

(4 row(s) affected)

to make you new table use this:

--set up tables:
create table TableA (id int, commodities varchar(8000))
INSERT TableA VALUES (1135,'fish,eggs,meat')
INSERT TableA VALUES (1127,'flour,oil')

Create table TableB (id int, commodities varchar(8000))

--populate TableB
INSERT TableB
    (id, commodities)
SELECT
    a.id,c.value
    FROM TableA    a
        CROSS APPLY dbo.inline_split_me(',',a.commodities) c

 --show tableB contents:
select * from TableB

OUTPUT:

id          commodities
----------- -------------
1135        fish
1135        eggs
1135        meat
1127        flour
1127        oil

(5 row(s) affected)

EDIT after Conrad Frix comment about SQL Server 2000 not supporting CROSS APPLY

this will do the same:

INSERT TableB
        (id, commodities)
    SELECT 
        a.id,NullIf(SubString(',' + a.commodities + ',' , number , CharIndex(',' , ',' + a.commodities + ',' , number) - number) , '')
        FROM TableA            a
            INNER JOIN Numbers n ON 1=1
        WHERE SubString(',' + a.commodities + ',' , number - 1, 1) = ',' 
        AND CharIndex(',' , ',' + a.commodities + ',' , number) - number > 0
        AND number <= Len(',' + a.commodities + ',') 

and is based on the code from the link in the answer by @Rup. It basically removes the function call and does the split in the main query (using a similar Numbers table split), so no need for a CROSS APPLY.

Answer 3

You write a SQL batch that loops through table A and inserts into table b the results of your function call.

Answer 4

Call me lazy, but I'd pull the combined rows out of the database, split them, then reinsert the split rows. This kind of thing seems kind of unnatural for SQL...

Answer 5

SSIS has a pretty handy Unpivot transform if that's available to you.

Answer 6

create table Project (ProjectId int, Description varchar(50));
insert into Project values (1, 'Chase tail, change directions');
insert into Project values (2, 'ping-pong ball in clothes dryer');

create table ProjectResource (ProjectId int, ResourceId int, Name varchar(15));
insert into ProjectResource values (1, 1, 'Adam');
insert into ProjectResource values (1, 2, 'Kerry');
insert into ProjectResource values (1, 3, 'Tom');
insert into ProjectResource values (2, 4, 'David');
insert into ProjectResource values (2, 5, 'Jeff');

-- a bit of SQL magic involving XML and voila
SELECT *, 
   (SELECT Name + ' ' AS [text()] 
    FROM ProjectResource pr 
    WHERE pr.ProjectId = p.ProjectId
    FOR XML PATH ('')) AS ResourceList 
FROM Project p

The output of this will be :

ProjectId   Description                       ResourceList
1           Chase tail, change directions     Adam Kerry Tom 
2           ping-pong ball in clothes dryer   David Jeff