All keyboard/mouse interactions are sent to the first responder
This is incorrect. From the Cocoa Event Handling Guide:
The window object … dispatches most events to the NSView object associated with the user action in an NSResponder message such as mouseDown: or keyDown:. …
The object receiving an event message differs slightly by type of event. For mouse and tablet events, the NSWindow object dispatches the event to the view over which the user pressed the mouse or stylus button. It dispatches most key events to the first responder of the key window.
This distinction between the paths of mouse and key events is repeated in the relevant sections specific to those types of events.
A button "knows" when it's been clicked because it is the thing which was clicked. The window figures out what was under the mouse cursor using the -[NSView hitTest:]
method.