The health of connection from the Load Balancer to the server is determined by the way in which your Health Check is set up.
Try and set it up differently.
eg
Use a TCP based Health Check rather than a HTTP based one, and change the thresholds.
If you see some different behaviour, you'll know that the Health Check is the issue.