Instead of using replaceWith, I used the JWPlayer load function coupled with a play command on my button. That way, the player would be initialized already when the seek function was read. Here's my code:
<script type="text/javascript">
function watchSermon(URL,sermonTime){
jwplayer('player').load([{
file:URL
}]);
jwplayer('player').play()
jwplayer('player').seek(sermonTime)
};
</script>
<button class="button-watch" onclick="javascript:watchSermon('{video_url}','{sermon_start}')">Watch sermon</button>