So I found out what I was doing wrong. I was changing the modalPresentationStyle to custom on the view controller that was being popped onto the navigation controller when I should have been setting it on the navigation controller itself. I added this line to the settingsButtonClicked method above and it worked properly.
navigationController.modalPresentationStyle = UIModalPresentationCustom;