There are load balancers out there that do this. The two, that I definitely know can do it, are F5's 'Big-IP LTM' (using the OneConnect feature) and Shaka Technologies 'Ishlangu Load Balancer ADC'.
F5's description for OneConnect:
The BIG-IP system OneConnect feature can increase network throughput by efficiently managing connections created between the BIG-IP system and back-end pool members. The OneConnect feature works with HTTP Keep-Alives to allow the BIG-IP system to minimize the number of server-side TCP connections by making existing connections available for reuse by other clients.
Shaka Technologies description for Connection Multiplexing:
The Ishlangu Load Balancer ADC's connection multiplexing capability improves web application performance and decreases server load by reducing the concurrent connections and connection rate on back-end servers.
Connection Multiplexing reuses TCP connections to each server for multiple clients. With fewer open connections and fewer connections opening/closing results in lower resource consumption per server. Minimizing resource utilization per server enables server consolidation therefore allowing operational cost savings.