First off as @David Jashi said, do it at night.
but more specifically do it at a non-peak time.
Will you have any roll-back in place just in case it does not come back up?
You could use the second server as a carbon copy then do the upgrade on the non-live server, test it and then swap the DNS records, this will provide no down-time what so ever.