I've the missing step and concept and solved this problem. The chapter is correct. But some concepts are not very comprehensively explained.
The input to Ma's book's flow is Euclidean homography whereas I fed in a Projective homography which results in a wrong answer.