The answer is WHATEVER YOU DO DONT USE RREF in java. Converting to reduced echelon form turns out to have lots of comparisons to 0. If the value is 0 we do one thing. If the value is very close to 0, but not quite 0, we do something completely different (like divide by the value). This means one unstable algorithm.
Instead we can use QR Decomposition, which happens to be implemented in colt.