I think you need to write
#add the current branch in yellow
export PS1='$(__git_ps1 "[\[\e[38;5;220m\]%s\[\x1b[0m\]]")'
#make the rest of the text in the prompt gray
PS1+="\[\e[38;5;245m\]\w$ "
#set colors back to default
PS1+="\[\033[0m\]"
So you have \[
and \]
surrounding all the colour directives.