The Fortigate has an option to "enable NAT" on a policy, which doesn't mean translate the addresses (it does that for you anyway), but does mean it alters some packets replacing the remote IP with the gateway IP.
Make sure that "Enable NAT" is disabled, and $_SERVER['REMOTE_ADDR']
will work as expected.