Create an array of all the file types you want Shadowbox to play included avi,flv etc
var FileTypesToPlay = { flv:['flv'],
qtwmp:['avi', 'mpg', 'mpeg'],
qt:['dv', 'mov', 'moov', 'movie', 'mp4'],
};
And write the Shadowbox.open like that
Shadowbox.open({
content: 'Your File Path Url',
players: [ 'flv', 'qt', 'qtwmp']
ext: '+FileTypesToPlay+'
});
Read the Shadowbox documentation also : http://www.shadowbox-js.com/usage.html