Your English description suggests you want:
apply( df, 1, which.min)
But the answer you give is not formatted as a vector and is not the correct answer if the above interpretation is correct. Oh wait, you were expecting rownumbers.
as.matrix(apply( d, 1, which.min))
[,1]
1 1
2 4
3 3
4 3
5 1
6 1
7 2
8 2
9 4
10 4
11 1
12 3
13 1
14 1
15 3
16 3
17 2
18 3
19 2
20 4