I think you can use machine learning methods for color classification. Just train classifier (e.g. SVM or neural network) on hear color samples (you can take color histogram as input vector), and some classes (color names). I think it will be good method fo deal with light changes, noise, shadows, etc. And I think it'll be better use some advanced color space L*a*b* or HSV.
May be you will need include in input vector some reference historgam (for get data about lighting conditions). You can get it from face for example (not the best solution).