The timestamp assigned to the request coming in to the proxy is what will ultimately decide which "last" write wins.
If you have a long running upload and issue the delete during, the tombstone will have a later timestamp and will eventually take precedence even if the upload finishes after.
When using the container versioning feature, overwriting in a versioned container will cause the object data to be COPY'd off the current tip before the PUT data is sent to the storage node with the assigned timestamp. For deletes in a versioned container the "previous version" is discovered at the time the overwriting request is made and subject to eventual consistency in the container listing, but is only deleted once it has been copied into the current location for the object.
More information about object versioning is available here:
http://docs.openstack.org/developer/swift/overview_object_versioning.html