Duplikate entfernen mit Caveats
-
02-07-2019 - |
Frage
Ich habe eine Tabelle mit rowID, Längengrad, Breitengrad, business, URL, Titel. Dies könnte wie folgt aussehen:
rowID | long | lat | businessName | url | caption
1 20 -20 Pizza Hut yum.com null
Wie kann ich all die Duplikate löschen, aber nur den einen halten, die eine URL (erste Priorität) hat, oder den halten, die eine Beschriftung hat, wenn der andere nicht über eine URL (zweite Priorität) und löschen Sie die Ruhe?
Lösung
Hier ist meine Looping-Technik. Dies wird wahrscheinlich abgelehnt bekommen für nicht Mainstream sein -. Und ich bin cool mit, dass
DECLARE @LoopVar int
DECLARE
@long int,
@lat int,
@businessname varchar(30),
@winner int
SET @LoopVar = (SELECT MIN(rowID) FROM Locations)
WHILE @LoopVar is not null
BEGIN
--initialize the variables.
SELECT
@long = null,
@lat = null,
@businessname = null,
@winner = null
-- load data from the known good row.
SELECT
@long = long,
@lat = lat,
@businessname = businessname
FROM Locations
WHERE rowID = @LoopVar
--find the winning row with that data
SELECT top 1 @Winner = rowID
FROM Locations
WHERE @long = long
AND @lat = lat
AND @businessname = businessname
ORDER BY
CASE WHEN URL is not null THEN 1 ELSE 2 END,
CASE WHEN Caption is not null THEN 1 ELSE 2 END,
RowId
--delete any losers.
DELETE FROM Locations
WHERE @long = long
AND @lat = lat
AND @businessname = businessname
AND @winner != rowID
-- prep the next loop value.
SET @LoopVar = (SELECT MIN(rowID) FROM Locations WHERE @LoopVar < rowID)
END
Andere Tipps
Diese Lösung wird präsentiert von „Sachen, die ich auf Stack-Überlauf gelernt habe“ in der letzten Woche:
DELETE restaurant
WHERE rowID in
(SELECT rowID
FROM restaurant
EXCEPT
SELECT rowID
FROM (
SELECT rowID, Rank() over (Partition BY BusinessName, lat, long ORDER BY url DESC, caption DESC ) AS Rank
FROM restaurant
) rs WHERE Rank = 1)
Warnung: Ich habe nicht getestet auf einer realen Datenbank
Set-basierte Lösung:
delete from T as t1
where /* delete if there is a "better" row
with same long, lat and businessName */
exists(
select * from T as t2 where
t1.rowID <> t2.rowID
and t1.long = t2.long
and t1.lat = t2.lat
and t1.businessName = t2.businessName
and
case when t1.url is null then 0 else 4 end
/* 4 points for non-null url */
+ case when t1.businessName is null then 0 else 2 end
/* 2 points for non-null businessName */
+ case when t1.rowID > t2.rowId then 0 else 1 end
/* 1 point for having smaller rowId */
<
case when t2.url is null then 0 else 4 end
+ case when t2.businessName is null then 0 else 2 end
)
delete MyTable
from MyTable
left outer join (
select min(rowID) as rowID, long, lat, businessName
from MyTable
where url is not null
group by long, lat, businessName
) as HasUrl
on MyTable.long = HasUrl.long
and MyTable.lat = HasUrl.lat
and MyTable.businessName = HasUrl.businessName
left outer join (
select min(rowID) as rowID, long, lat, businessName
from MyTable
where caption is not null
group by long, lat, businessName
) HasCaption
on MyTable.long = HasCaption.long
and MyTable.lat = HasCaption.lat
and MyTable.businessName = HasCaption.businessName
left outer join (
select min(rowID) as rowID, long, lat, businessName
from MyTable
where url is null
and caption is null
group by long, lat, businessName
) HasNone
on MyTable.long = HasNone.long
and MyTable.lat = HasNone.lat
and MyTable.businessName = HasNone.businessName
where MyTable.rowID <>
coalesce(HasUrl.rowID, HasCaption.rowID, HasNone.rowID)
zu einer anderen Antwort Ähnliche, aber Sie basiert löschen möchten auf Zeilennummer anstatt Rang. Mischen Sie mit allgemeinen Tabellenausdrücken auch:
;WITH GroupedRows AS
( SELECT rowID, Row_Number() OVER (Partition BY BusinessName, lat, long ORDER BY url DESC, caption DESC) rowNum
FROM restaurant
)
DELETE r
FROM restaurant r
JOIN GroupedRows gr ON r.rowID = gr.rowID
WHERE gr.rowNum > 1
Wenn möglich, können Sie homogenisieren, dann Duplikate entfernen?
Schritt 1:
UPDATE BusinessLocations
SET BusinessLocations.url = LocationsWithUrl.url
FROM BusinessLocations
INNER JOIN (
SELECT long, lat, businessName, url, caption
FROM BusinessLocations
WHERE url IS NOT NULL) LocationsWithUrl
ON BusinessLocations.long = LocationsWithUrl.long
AND BusinessLocations.lat = LocationsWithUrl.lat
AND BusinessLocations.businessName = LocationsWithUrl.businessName
UPDATE BusinessLocations
SET BusinessLocations.caption = LocationsWithCaption.caption
FROM BusinessLocations
INNER JOIN (
SELECT long, lat, businessName, url, caption
FROM BusinessLocations
WHERE caption IS NOT NULL) LocationsWithCaption
ON BusinessLocations.long = LocationsWithCaption.long
AND BusinessLocations.lat = LocationsWithCaption.lat
AND BusinessLocations.businessName = LocationsWithCaption.businessName
Schritt 2: Duplikate entfernen.