There are two parts to the solution I found for this.
Just to be clear, I was using the text (HTML) editor, not the WYSIWYG editor.
Firstly, in one of the other instances where there was this L shaped character, there was a single space after the
and before the next word. That single space was some weird sort of character, because it took two backspaces to delete it. And the same happened when I copied it into notepad - it looked like a single space, but took two backspaces to delete it. Very weird, because I've never seen notepad have special characters like that before.
However that same technique didn't solve this specific instance shown above. I remove all extra spaces, so it was just pure HTML code, but for some reason it still had this character showing. So in the end I changed the width of the H1 tag in CSS so that it wrapped where I wanted it to, and then removed the
altogether. That got rid of the weird L shaped character.