A quick duckduckgo search for "xml::simple memory leak" turned up the documentation for Yahoo::Search, which claims:
XML::Simple uses XML::Parser under the hood, and at least on the systems I've tested it, XML::Parser suffers from a crippling memory leak that makes it very undesirable.
It appears possible (perhaps even likely) that the memory leak may not be in your code at all.