Will this key work on multiple phones using my app?
Yes, One Application with proper Map-Key can works on many devices. You dont need to create device wise map-key.
Do I have to create an individual Google Maps API key for each new phone that downloads my app from the store?
No. One Key is enough.