I believe your confusion lies in the artificial coupling between database row identity and the notion of identity associated with an entity in DDD. They are certainly related because an entity will have a corresponding identity represented in the database as an identity column. However, just because something has an identity as a database row it doesn't mean that the object has identity in the DDD sense.
In you address example, the value comprising the address VO would typically be stored in the same row as the customer entity. In this way, the address is a value object because it isn't stored in a row of its own and has no identity. When you update an address, you alter the value of the address property on the customer entity, which in turn reflects in the database row.
There are cases where a value object would be stored in its own row. For example, in the stereotypical sales order model, an order is an entity (aggregate root) and line items are value objects. While line items are VOs, in the relational model, they are stored in their own table and may very well have primary keys. In the domain model however, the VOs are tied to the order entity and don't have identity outside of that entity.