The physicsBody property of an SKNode (or subclass of) is nil by default. Yes give the buildingStructure a physics body and correctly configure it to achieve the desired behaviour.
To get the desired behaviour between two physics bodies, first understand that there are 'collisions' and there are 'contacts'. SKPhysicsBody has two properties named collisionBitMask and contactTestBitMask.
For collisionBitMask the default value is 0xFFFFFFFF (all bits set), and for contactTestBitMask the default value is 0x00000000 (all bits cleared). You can see that by defaults every physics body will collide with others but there will be no contact.
A collision is where you want the physics simulation to affect the body in question, where a contact is where you want to be notified so you can do something. So this will call didBeginContact:
I suggest setting the collisionBitMask for both bodies to 0 (so their physics is not affected and pass through one another). Then set the contactTestBitMask property of character to that of the building like this-
character.physicsBody.contactTestBitMask = categoryBitManager.buildingStructureCategory;