with the help of this question: Cannot pick an SSH client when installing TortoiseGIT
I was able to resolve the issue by right clicking, going down to 'TortoiseGit > Settings' then into 'Network' and changing the SSH client to 'ssh.exe' without any path.
I could then properly clone my repo.
Hope this helps someone in future. What a pain that was.