Without looking at your file I'd say the most likely cause is that the cell does contain a number which is then formatted as a date. This is perfectly valid in Excel, in fact all dates are stored as numbers.
Use the following to check if a cell contains a date:
DateTime date;
if (cell.TryGetValue(out date))
{
// Got a date
}