You need to delete the segments individually, in the same manner you delete the manifest file.
Keep in mind there is no way to atomically delete a dynamic large object. Even if you delete the manifest file first, the weak consistency model used by the API does not require the effects of two delete operations to appear to users (e.g. in future Get Object calls) in the order they were sent.