When vertex container policy is defined as vecS, setS or listS, the vertex_descriptor becomes ["value_type" pointed to by] iterator in corresponding container. The type vertex_iterator is usually an iterator type in this container.
It means access to a vertex is always O(1). Adding and removing a new vertex will cost O(logN) in case of setS and O(1) in case of listS or vecS. Removing a vertex will invalidate some other vertices in case of vecS but not listS or setS. For details see the section Iterator and Descriptor Stability/Invalidation at the adjacency graph page.
Regarding your other question. If you want to keep a (pointer) to a data directly on a vertex, you add it as a vertex property.
Quite often you need some kind of one-to-one mapping between your data and graph vertices. It can be convenient to keep direct map as, say, boost::unordered_set and inverse map as a property on each vertex.