It took a while, but this is the answer: https://stackoverflow.com/a/16903982/805882
packet.pts = av_rescale_q(packet->pts, inStream->time_base, outStream->time_base);
packet.dts = av_rescale_q(packet->dts, inStream->time_base, outStream->time_base);