MSSQL Remove duplicate records

MSSQL Remove duplicate records from a table leaving a single unique record.

The situation arose where we had multiple records for items where there should be only a single record. We have JobIDs which will have multiple unique TaskIDs. We found we several (lots) of JobIDs which had 2 of the same TaskIDs. It's OK to have many TaskIDs in each job, there just can't be 2 of the same TaskIDs per JobID. Many TaskIDs to one JobID and one unique TaskID to one JobID.

First we find which TaskIDs are duplicated.

view plain print about
1Select DISTINCT TaskID FROM (
2select JobID, TaskID, (*) as Count from JobTable
3Where TaskID IS NOT NULL
4Group by JobID, TaskID HAVING COUNT(*) >
1) X

We are going to remove the 2nd entry for each duplicate TaskID. The original TaskID is correct in this case. The 2nd TaskID is superfluous and will have a greater Identity Key, they were added at a later time. Let's pull up the duplicate TaskIDs with the larger Identity Key (ID).

view plain print about
1SELECT T1.ID
2FROM JobTable T1
3JOIN
4(SELECT JobID,TaskID , MAX(ID) AS ID FROM JobTable
5GROUP BY JobID,TaskID
6Having COUNT(*) >
1) AS T2
7ON T1.TaskID = T2.TaskID AND T1.Qbid = T2.QBID AND T1.ID = T2.ID

This pulls up what we want, so lets get rid of them.

view plain print about
1DELETE JobTable WHERE ID in (
2SELECT T1.ID
3FROM JobTable T1
4JOIN
5(SELECT JobID,TaskID , MAX(ID) AS ID FROM JobTable
6GROUP BY JobID,TaskID
7Having COUNT(*) >
1) AS T2
8ON T1.TaskID = T2.TaskID AND T1.Qbid = T2.QBID AND T1.ID = T2.ID
9)

Done! Now we have unique TaskIDs for each JobID.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
BlogCFC was created by Raymond Camden. This blog is running version 5.9.7. Contact Blog Owner