Not sure about this but give a try.
DELETE FROM persoonburgstaat person where (person.ingangsdatum, person identifier) in
(select p.ingangsdatum, p.identifier from persoonburgstaat p
left join p.persoon per where per.id_persoon = :persoonID
AND per.id_persoonburgerlijkestaa > p.identifier)
the left join will make the outer query
But to be more sure post PersoonEntity entity as I think " id_persoonburgerlijkestaa " is the name of the column not the property and query will fail based on that.