I think you mistake the function of a feColorMatrix. You don't get a single output color from a colormatrix, the output is dependent on the combination of R , G, and B colors.
Read the web platform docs on feColorMatrix.
You can play with sepia settings using this color matrix demo I built.