I thought you were crazy until I tried it myself in a minimal PivotViewer control and got the same error!
After some more experimentation, it seems the problem isn't with negative numbers but rather with a mantissa (the decimal portion) containing four or five leading zeroes, depending on the value of the first non-zero digit.
Values with four leading zeroes like 0.00001, -1.00002, 2.00003, -3.00004 will all cause your error. Mantissas .00005 through 0.00009 do work, presumably because the values are being rounded up, e.g. 0.00005 "=" 0.0001.
The same problem is described here: (http://social.msdn.microsoft.com/Forums/en-US/silverlightpivotviewer/thread/03f0e782-f2d8-4c70-86f2-93560a8d6151/)
Their workaround: round your decimals to four places and you should be fine. If your data are in a super-tight range where you need that degree of precision in PivotViewer, I'd multiply by e.g. 1000 and somehow indicate that the values are scaled.
PS This behavior certainly seems buggy - I'd guess the magic four-decimal limit is related to the fact that, when dragging sliders in a Numeric property's histogram in the filter panel, it displays text indicating the current value of the slider - to four decimal places' precision. Perhaps throwing a fifth decimal place at it makes it go boom?