Swiftstack - Containers not getting removed

Question

Even after deleting containers and objects directly from file system, Swift is listing the containers when executed GET command on it. However, if we try to delete the container with DELETE command then 404: Not Found error message is returned. Please explain whether there is something wrong or is there some kind of cache?

Answer 1

I think the problem came from deleting the containers and/or objects directly from the file system.

Swift's methods for handling write requests for object and container have to be very careful to ensure all the distributed index information remains eventually consistent. Direct modification of the file system is not sufficient. It sounds like the container databases got removed before they had a chance to update the account databases listings - perhaps manually unlinked before all of the object index information was removed?

Normally after a delete request the containers have to hang around for awhile as "tombstones" to ensure the account database gets updated correctly.

As a work around you could recreate them (with a POST) and then re-issue the DELETE; which should successfully allow the DELETE of the new empty containers and update the account database listing directly.

(Note: the container databases themselves, although empty, will still exist on disk as tombstones until the the reclaim_age passes)