What you have pointed out is correct, the deleted node should be replace by either its in order successor which is the left most node in the right sub-tree or its in-order predecessor which is the right most node in the left sub-tree. This allows the tree to be traversed correctly. Most binary search tree data structures allow the deletion to be performed either way but in some cases special cases you might want to implement deletion such that the tree remains balanced.
More details and sample code is available on Wikipedia.