You are most likely suffering from rounding errors. The PTS in TS MUST be perfect or many players will attempt to resynchronize playback with the reference clock , this will often appears as dropped samples, or inserted silence.
Make sure your starting PTS is accurate by counting samples and converting to to 90khz. Do your multiply before your divide e.g. (sampleCount * 90000) / sampleRate
(NOT sampleCount * ( 90000 / sampleRate )
, and make sure you are using a 64bit integer to avoid integer overflows. Or better yet, use av_rescale from libavutil.