This should be possible using tmux configuration
set-titles [on | off ] Attempt to set the window title using the \e]2;...\007 xterm code if the terminal appears to be an xterm. This option is off by default. Note that elinks will only attempt to set the window title if the STY environment variable is set.
set-titles-string string String used to set the window title if set-titles is on. Character sequences are replaced as for the status-left option.
The item in question is #S Ta Session name
.
Check the man page.
To set this for a the session currently connected/active:
$ tmux set-option -s set-titles on
$ tmux set-option -s set-titles-string '#S'