You can call ISession.Evict
to remove objects from the session. My memory is that this affects only the object passed as the argument, not the entire object's graph, so changes to child collections are not evicted.
My suspicion is that your unit-of-work is too long. It's unusual to make changes to an entity that you do not want persisted to the database.
Oh, and the SaveOrUpdate call is unnecessary.