I'm fairly certain you can't get raw mode input easily without os syscalls. Have a look at the answers to Read a character from standard input in Go (without pressing Enter) .
It seems that the generally accepted solution is to use termbox-go which apparently has fairly good cross platform capability.