This has kinda split into 2 questions really, the original workaround for conflicting assembly versions and then the issue of assembly redirection not working. Since it looks like assembly redirection is the answer to the original question (as pointed out by Steven V in comments) I will mark question as answered and open another one on why the implementation of assembly redirection is not working.
Thanks to Steven V for nudging me in the right direction.