This question was specific to my problem, but the general problem here is how to properly use containment for jQuery UI Draggable.
Solution
Since my selector was the same for all the elements I wanted to be draggable they also must have the same containment parameter. If you want to have different containment parameters you must have unique selectors otherwise the first element you call draggable() on will have its containment parameter overridden and this odd behavior will occur.