There are 1,152 samples per a frame, so if your chunk is a fixed number of N
frames, then your chunk is a fixed length of N*1152
samples. To turn that into milleseconds, you will need to find the sample rate from the frame header.
You just need a MP3 parser. Here is the source code for a full blown decoder: https://bitbucket.org/portalfire/pymp3 including frame header parsing code, which is really all you need.
Here is more documentation on the format: http://www.codeproject.com/Articles/8295/MPEG-Audio-Frame-Header