After struggled for hours I manage to solved this issue.
I think I have too many debug gems.
debugger (1.4.0)
debugger-linecache (1.1.2)
debugger-ruby_core_source (1.2.0)
ruby-debug-base19x (0.11.30.pre10)
ruby-debug-ide (0.4.17.beta16)
linecache19 (0.5.12)
After removed the following gems it working fine and the breakpoints just stop at the line I want.
debugger (1.4.0)
debugger-linecache (1.1.2)
debugger-ruby_core_source (1.2.0)
Thanks.