From my understanding, if you switch from in-proc to state server the idle timeout (in IIS) setting won't have an effect on your session state timeout.
There will still be worker processes that may be terminated in the application pool if there is no activity (if the idle timeout is passed) but the session state (i.e. the user session and application session values) will be maintained beyond this. Your session timeout should just be controlled by the timeout value set in the configuration (from here) i.e.
<configuration>
<system.web>
<sessionState mode="StateServer"
stateConnectionString="tcpip=SampleStateServer:42424"
cookieless="false"
timeout="20"/>
</system.web>
</configuration>
Inproc is faster than StateServer as your session data needs to be serialised / deserialised when it's stored. It may also be stored on a separate machine which may introduce some latency. But of course there are the advantages of State Server i.e. Session state persistence between application restarts (app pool recycling), state can be shared across multiple servers in a web-farm.
This question also discusses pros and cons of using the State Server mode.