DTS Package Terminates because of Duplicate Key Row

Question

We have an old DTS Package that our SQL 2000 Server uses to push Employee records out to machines on our manufacturing floor.

Recently, we upgraded one of the machines, and it now is running SQL 2008 Express.

We have reconfigured the DTS Package to push the Employee records out to this new Server, but now we are getting this error message:

FETCH_EMPLOYEES:

The statement has been terminated. Cannot insert duplicate key row in object 'dbo.Users' with unique index 'IX_tblUsers_OpID'.

If I remote into our SQL 2000 Server, I can Right-Click to execute each step of the DTS Package in succession with NO errors.

So, I log onto this machine's SQL 2008 Express instance to see if I can figure anything out.

Now I am looking at the FETCH_EMPLOYEES stored procedure:

PROCEDURE [dbo].[FETCH_EMPLOYEES] AS
DECLARE @OpID varchar(255)
DECLARE @Password varchar(50)
DECLARE Employee_Cursor CURSOR FOR
SELECT OpID, Password
FROM dbo.vw_Employees
OPEN Employee_Cursor
FETCH NEXT FROM Employee_Cursor
INTO @OpID,@Password
WHILE @@FETCH_STATUS = 0
BEGIN
insert into dbo.Users (OpID,Password,GroupID) 
VALUES (@OpID,@Password,'GROUP01')
FETCH NEXT FROM Employee_Cursor
INTO @OpID,@Password
END
CLOSE Employee_Cursor
DEALLOCATE Employee_Cursor

I don't really understand Cursors, but I can tell that the data is being pulled from a view called vw_Employees and being inserted into the table dbo.Users.

The view vw_Employees is simple:

SELECT DISTINCT FirstName + ' ' + LastName AS OpID, Num AS Password
FROM         dbo.EmployeeInfo
WHERE     (Num IS NOT NULL) AND (FirstName IS NOT NULL)
      AND (LastName IS NOT NULL) AND (Train IS NULL OR Train <> 'EX')

So, now it seems the problem must be from the table dbo.Users.

I did not see anything particularly attention getting with this, so I scripted this table using a CREATE TO Query Editor and got this information that I don't really understand:

CREATE TABLE [dbo].[Users](
[ID] [int] IDENTITY(1,1) NOT NULL,
[OpID] [nvarchar](255) NOT NULL,
[Password] [nvarchar](50) NOT NULL,
[GroupID] [nvarchar](10) NOT NULL,
[IsLocked] [bit] NOT NULL,
    CONSTRAINT [PK_tblUsers] PRIMARY KEY CLUSTERED 
(
[ID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

ALTER TABLE [dbo].[Users]  WITH CHECK ADD  CONSTRAINT [FK_tblUsers_tblGroups] FOREIGN KEY([GroupID])
REFERENCES [dbo].[Groups] ([GroupID])
GO

ALTER TABLE [dbo].[Users] CHECK CONSTRAINT [FK_tblUsers_tblGroups]
GO

ALTER TABLE [dbo].[Users] ADD  CONSTRAINT [DF_tblUsers_IsLocked]  DEFAULT ((0)) FOR [IsLocked]
GO

OK, I feel the problem is somewhere in this table definition, but I don't really understand what it is doing (after creating the basic table).

It has a CONSTRAINT section with lots of variables I do not understand, then it is altering these tables to add FOREIGN KEY and CONSTRAINTS.

My Question: Could someone help me understand what the error is telling me (other than there is some duplicate key violation).

What column could be throwing a duplicate key violation?

Did I include enough data and screenshots?

UPDATE:

Based on Comments, it sounds like this screenshot is needed.

In the Users table, there is a list of Indexes, and one called IX_tblUsers_OpID says it is Unique and Non-Clustered.

I think we have eliminated duplicate Op_ID values on our source data table EmployeeInfo by finding all of them with this script:

select num as 'Op_ID', count(num) as 'Occurrences'
from employeeInfo
group by num
having 1<count(num);

This should have gotten rid of all of my duplicates. Right?

We purchase manufacturing machines that come configured with PCs for storing local data. They supply these script I have posted up, so I cannot comment on why they picked what they did. We just run a job that pulls the data onto our server.

Answer 1

Having columns with unique values has always been of high value on any dataset. This constrain can be added to any column, or index.

The error you receive is very clear and very specific. It literally gives the answer.

The statement has been terminated. Cannot insert duplicate key row in object 'dbo.Users' with unique index 'IX_tblUsers_OpID'.

It says "NO duplicates.... UNIQUE index..." then it tells you the name of the constrain "IX_tblUsers_OpID".

Now keeping that in mind, you are trying to insert in that column values you craft on the fly by concatenating two strings; name, plus last name.

What are the chances to come up with two of them being "John Smith"? High, very high!

Possible solutions:

• You may remove the constrain and allow duplicates.
• Modify the query so the values that tries to insert are -indeed- unique.
• Use 'WITH (IGNORE_DUP_KEY = ON)' Reference: index_option (Transact-SQL)

Answer 2

Another guy here at work found this hidden feature, which solves the immediate problem but could cause other unknown issues.

In the Users table designer view, we can Right-Click on the OpID column, select Indexes/Keys..., locate this created IX_tblUsers_OpID key and change it's Is Unique value:

That seemed to have made it so that the DTS Package will run, and that is what we have going on right now.

I went back to the original EmployeeInfo table on our SQL 2000 Server to check for duplicate OpID values using this script:

select FirstName + ' ' + LastName as 'OpID',
       Count(FirstName + ' ' + LastName) as 'Occurrences'
from EmployeeInfo
group by FirstName + ' ' + LastName
having 1 < count(FirstName + ' ' + LastName)

...but there were no records returned.

I'm not sure why the DTS Package was failing or why we had to turn off the Unique feature.

If anyone, at some time down the road, comes up with a better fix for this, please post!