I'm not sure if the notifications you are putting out are for the same event, if so you can do what is called "stacking notifications" like Gmail does when you get a second or third email. It's all in the one notification and therefore will go away as one. This is only compatible above 4.1.
This is the code Google gives along with the stacking. This any help?
mNotificationManager =
(NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
// Sets an ID for the notification, so it can be updated
int notifyID = 1;
mNotifyBuilder = new NotificationCompat.Builder(this)
.setContentTitle("New Message")
.setContentText("You've received new messages.")
.setSmallIcon(R.drawable.ic_notify_status)
numMessages = 0;
// Start of a loop that processes data and then notifies the user
...
mNotifyBuilder.setContentText(currentText)
.setNumber(++numMessages);
// Because the ID remains unchanged, the existing notification is
// updated.
mNotificationManager.notify(
notifyID,
mNotifyBuilder.build());