This should work OK. I set up an example project to do the same thing. I have three buttons with equal width constraints between button one and button two. Then the same between button two and button three. Button one has a trailing edge constraint relative to it's parent view.
Note that between the second and third button, Interface Builder didn't let me do this in the main window. Instead, I had to do it using the document list on the left.
The result can be seen below:
Here's the link to the project:
https://github.com/MaxMacleod/ThreeButtonConstraintTest1
Couple of caveats. First, this is an iOS rather than an OS X project (I'm an iOS guy!). However the principles should be the same. Secondly, this doesn't pinpoint the exact reason why your project isn't working. However, if you can compare what this sample project does against yours, we can figure it out. I'll then update this answer. Better still if you could make your project available, I'd be happy to take a look.