Finding duplicate XML nodes and minimize XML strcutre and size

https://stackoverflow.com/questions/19208961

30-06-2022
|

Question

I have the current XML which contains many duplicates.

<XML version="1.0">
  <body>
    <optionTree>
      <device modelDescription="20 Slot MX2000 Chassis, Base with 1 RE, SFBs, Fan Trays, AC Power" ProductId="1542671" modelCatalogNum="MX2020-BASE-AC" price="450000" quantity="1" userRemovable="1">
        <optionItem modelDescription="MPC Slot 0" slotIndex="1" isSingleSelected="1">
          <device modelDescription="2xTrio Chipset Enhanced MPC, 1588v2, port queuing, price includes full scale L2/L2.5 and reduced scale L3 features" ProductId="1540947" modelCatalogNum="MX-MPC2E-3D-P" price="70000">
            <optionItem modelDescription="MIC Slot 1" slotIndex="1" isSingleSelected="1">
              <device modelDescription="20x10/100/1000 MIC for MX, requires optics sold separately" ProductId="334748" modelCatalogNum="MIC-3D-20GE-SFP" price="9000">
                <optionItem modelDescription="SFP Slot 1" slotIndex="1" isSingleSelected="1">
                  <device modelDescription="SFP capable of support 10/100/1000 speeds" ProductId="291564" modelCatalogNum="SFP-1GE-FE-E-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable supporting 1000BASE-EX Gigabit Ethernet Optic Module, 40km." ProductId="205359" modelCatalogNum="SFP-GE40KM" price="2500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-SX Gigabit Ethernet Optic Module" ProductId="51804" modelCatalogNum="SFP-1GE-SX" price="500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-T Gigabit Ethernet Module (uses Cat 5 cable)" ProductId="51805" modelCatalogNum="SFP-1GE-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LH Gigabit Ethernet Optic Module" ProductId="205257" modelCatalogNum="SFP-1GE-LH" price="5995" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LX Gigabit Ethernet Optic Module" ProductId="51803" modelCatalogNum="SFP-1GE-LX" price="995" />
                </optionItem>
                <optionItem modelDescription="SFP Slot 2" slotIndex="2" isSingleSelected="1">
                  <device modelDescription="SFP capable of support 10/100/1000 speeds" ProductId="291564" modelCatalogNum="SFP-1GE-FE-E-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable supporting 1000BASE-EX Gigabit Ethernet Optic Module, 40km." ProductId="205359" modelCatalogNum="SFP-GE40KM" price="2500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-SX Gigabit Ethernet Optic Module" ProductId="51804" modelCatalogNum="SFP-1GE-SX" price="500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-T Gigabit Ethernet Module (uses Cat 5 cable)" ProductId="51805" modelCatalogNum="SFP-1GE-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LH Gigabit Ethernet Optic Module" ProductId="205257" modelCatalogNum="SFP-1GE-LH" price="5995" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LX Gigabit Ethernet Optic Module" ProductId="51803" modelCatalogNum="SFP-1GE-LX" price="995" />
                </optionItem>
                <optionItem modelDescription="SFP Slot 3" slotIndex="3" isSingleSelected="1">
                  <device modelDescription="SFP capable of support 10/100/1000 speeds" ProductId="291564" modelCatalogNum="SFP-1GE-FE-E-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable supporting 1000BASE-EX Gigabit Ethernet Optic Module, 40km." ProductId="205359" modelCatalogNum="SFP-GE40KM" price="2500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-SX Gigabit Ethernet Optic Module" ProductId="51804" modelCatalogNum="SFP-1GE-SX" price="500" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-T Gigabit Ethernet Module (uses Cat 5 cable)" ProductId="51805" modelCatalogNum="SFP-1GE-T" price="395" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LH Gigabit Ethernet Optic Module" ProductId="205257" modelCatalogNum="SFP-1GE-LH" price="5995" />
                  <device modelDescription="Small Form Factor Pluggable 1000Base-LX Gigabit Ethernet Optic Module" ProductId="51803" modelCatalogNum="SFP-1GE-LX" price="995" />
                </optionItem>
              </device>
            </optionItem>
          </device>
        </optionItem>
      </device>
    </optionTree>
   </body>
</XML>

I want to minimize this XML and remove all duplicated XML nodes.

I want to create a code does which does the following:

Loops through this XML and finds duplicate "Device" elements.
for each duplicate node, remove all of its attribute except for "ProductId" attribute.
Create a legend in the bottom as follows:
A new element called "Devices" will be created and contain full details of all the removed XML nodes.

final XML should look like this:

<XML version="1.0">
  <body>
    <optionTree>
      <device modelDescription="20 Slot MX2000 Chassis, Base with 1 RE, SFBs, Fan Trays, AC Power" ProductId="1542671" modelCatalogNum="MX2020-BASE-AC" price="450000" quantity="1" userRemovable="1">
        <optionItem modelDescription="MPC Slot 0" slotIndex="1" isSingleSelected="1">
          <device modelDescription="2xTrio Chipset Enhanced MPC, 1588v2, port queuing, price includes full scale L2/L2.5 and reduced scale L3 features" ProductId="1540947" modelCatalogNum="MX-MPC2E-3D-P" price="70000">
            <optionItem modelDescription="MIC Slot 1" slotIndex="1" isSingleSelected="1">
              <device modelDescription="20x10/100/1000 MIC for MX, requires optics sold separately" ProductId="334748" modelCatalogNum="MIC-3D-20GE-SFP" price="9000">
                <optionItem modelDescription="SFP Slot 1" slotIndex="1" isSingleSelected="1">
                  <device ProductId="291564" />
                  <device ProductId="205359" />
                  <device ProductId="51804" />
                  <device ProductId="51805" />
                  <device ProductId="205257" />
                  <device ProductId="51803" />
                </optionItem>
                <optionItem modelDescription="SFP Slot 2" slotIndex="2" isSingleSelected="1">
                  <device ProductId="291564" />
                  <device ProductId="205359" />
                  <device ProductId="51804" />
                  <device ProductId="51805" />
                  <device ProductId="205257" />
                  <device ProductId="51803" />
                </optionItem>
                <optionItem modelDescription="SFP Slot 3" slotIndex="3" isSingleSelected="1">
                  <device ProductId="291564" />
                  <device ProductId="205359" />
                  <device ProductId="51804" />
                  <device ProductId="51805" />
                  <device ProductId="205257" />
                  <device ProductId="51803" />
                </optionItem>
              </device>
            </optionItem>
          </device>
        </optionItem>
      </device>
    </optionTree>
    <devices>
      <device ProductId="291564" modelDescription="SFP capable of support 10/100/1000 speeds" modelCatalogNum="SFP-1GE-FE-E-T" price="395" />
      <device ProductId="205359" modelDescription="Small Form Factor Pluggable supporting 1000BASE-EX Gigabit Ethernet Optic Module, 40km." modelCatalogNum="SFP-GE40KM" price="2500" />
      <device ProductId="51804" modelDescription="Small Form Factor Pluggable 1000Base-SX Gigabit Ethernet Optic Module" modelCatalogNum="SFP-1GE-SX" price="500" />
      <device ProductId="51805" modelDescription="Small Form Factor Pluggable 1000Base-T Gigabit Ethernet Module (uses Cat 5 cable)" modelCatalogNum="SFP-1GE-T" price="395" />
      <device ProductId="205257" modelDescription="Small Form Factor Pluggable 1000Base-LH Gigabit Ethernet Optic Module" modelCatalogNum="SFP-1GE-LH" price="5995" />
      <device ProductId="51803" modelDescription="Small Form Factor Pluggable 1000Base-LX Gigabit Ethernet Optic Module" modelCatalogNum="SFP-1GE-LX" price="995" />
    </devices>
   </body>
</XML>

I have already found a code that finds duplicate XML nodes, However I want to do more than just find those duplicates as I described above.

Solution

A quick and dirty solution that should solve your problem:

//find duplicates
var duplicates = doc.Descendants("device")
                    .GroupBy(n => n.ToString())
                    .Where(g => g.Count() > 1)
                    .ToArray();

//remove attributes on all duplicates
foreach (var attribute in duplicates.SelectMany(g => g)
                                    .SelectMany(el => el.Attributes()
                                                        .Where(attribute => attribute.Name != "ProductId"))
                                    .ToArray())
{
    attribute.Remove();
}

//add new element
doc.Descendants("body")
   .Single()
   .Add(new XElement("devices", duplicates.Select(g => XElement.Parse(g.Key))));

Basically the duplicates IGrouping collection uses elements's string representation as a key and holds all duplicated nodes.

Licensed under: CC-BY-SA with attribution

Not affiliated with StackOverflow