Per this post, it seems like a rather complicated problem: https://forum.jquery.com/topic/chrome-bug-or-how-do-i-prevent-a-form-field-to-scroll-the-container-when-focused
I have managed similar issues by manually managing the tabindex attribute of the inputs. This removes the unwanted pseudo-scrolling, but also removes the ability to tab to inputs outside the viewport. If you are willing to live with that, then you can easily add some code to your click handlers to add or remove tabindexes:
$(inputEl).attr('tabindex', -1); // removes tabbing
$(inputEl).attr('tabindex', 1); // restores tabbing
Remove the tab indexes from inputs outside the viewport, and restore them for visible inputs.