Thanks to @romainl comment, I looked up completeopt, and I found the answer Make Vim completion popup menu work just like in an IDE. It has a bunch of other related tips. To make the behaviour as described in the question,
:set completeopt=longest,menuone