All that needed to be done was to configure the Estimation Statistic, specifically for Time Tracking.
It needs to be switched from "Remaining Estimate and Time Spent" to "None"(which also means it defaults to your Estimation Statistic for Time Tracking, in my case "Story Points")
https://support.atlassian.com/jira-software-cloud/docs/configure-estimation-and-tracking/