It looks like the simplest fix is to return a value at the end:
retry {
System.IO.File.Move("a", "b")
return ()
}
If you look at how computation expressions are de-sugared your code seems to be converted into
retry.Run(retry.Delay(fun () -> System.IO.File.Move("a", "b"); retry.Zero()))
this causes the exception to be thrown during the evaluation. If you return a value this will not happen.