Sorry to say, to do that, you WILL have to work with symbols. However, this is an easy fix.
Select the object you want to color, click Modify--> Convert to Symbol. Give it a good name, and set it to MovieClip. Click OK.
Now, in the Library panel, double-click the icon next to your symbol name. Color as you wish.
There are multiple ways to put this on your timeline. You could replace each instance of that drawing with the symbol. Or, better, you could create a new layer with only one keyframe (and regular frames after that to the end of your animation. Place your object on that layer, and then remove all the old versions of that drawing.
NOTE: You will need to be mindful of where in the stack you put layers, as that sets the z-index of everything on that layer.