Answering this myself in case anyone needs it.
You can use HideFormPage("Button Name") to hide the orphaned button placed in the ribbon by a custom form. There is no need to loop through the ribbon and find it.
Application.ActiveInspector.HideFormPage("Button Name");