Bit of a hack, but it should work with what you already have:
$file_array = "XXX_backup*.trn", "XXX_backup*.bak", "XXY_backup*.trn", "XXY_backup*.bak"
$FileNames = $server_files | select -ExpandProperty Name
$FileCheck = ('$Filenames ' + ($file_array | foreach {"-notlike '$_'"}) -join ' ')
Since you're already using wildcards in your file specs, this works with that by creating a command which filters the file name array through a series of -notlike operators, one for each file spec.
$FileCheck
$Filenames -notlike 'XXX_backup*.trn' -notlike 'XXX_backup*.bak' -notlike 'XXY_backup*.trn' -notlike 'XXY_backup*.bak'
Each one will filter out the names that match the file spec and pass the rest onto the next -notlike operator. Whatever falls out the end didn't match any of them
At this point $FileCheck is just a string, and you'll need to use Invoke-Expression to run it.
$ExtraFiles = Invoke-Expression $FileCheck
and then $ExtraFiles should contain the names of all the files that did not match any of the file specs in $file_array. Since the filters are being created from $file_array, there's no other maintenance to do if you add or remove filespecs from the array.