You need to specify the public IPs if you want to send emails to the internet.
If you are sending emails to another mail server on the LAN then you might have to add the private IPs as well.
I suggest to use a testing utility to check your SPF records.