So that GUI is SourceTree, by Atlassian. And it was correct. The issue was that I had other branches and tags referencing the "old history". So naturally, it sticks around!
Think in terms of Back to the Future :)
So, "you can't rewrite history" is partly correct, it just depends on your perspective.
Doc Brown: Obviously the time continuum has been disrupted, creating a new temporal event sequence resulting in this alternate reality.
Marty: English, Doc!
Doc Brown: Here. Here, let me demonstrate. Let's say that this line represents time. [draws straight line and points to places] Here's the present 1985, the future and the past. Obviously, somewhere in the past the timeline skewed down into this tangent (branch) [draws new line and writes 1985A] creating an alternate 1985 (master). Alternate to you, me, and Einstein, but reality for everyone else. Recognize this? [shows Blast from the Past bag (tag)] It’s the bag the sports book came in; I know because the receipt (tag) was still inside. I found them in the time machine...along with this! (more tags)
Yay! Git is BTTF!
Moral: Delete the references and it will cease to exist!