java.lang.Number doesn't implement “+” or any other operators?

Question

I'm creating a class which is supposed to be able to be used with an array of any type of number (float, int, etc), so here is one method I have:

// T extends Number
public synchronized T[] average() {
    Number[] ret = new Number[queue[0].length];
    for (int i = 0; i < ret.length; ++i) {
        for (int j = 0; j < size; ++j) {
            ret[i] += queue[j][i]; // WTF ERROR?!
        }
        ret[i] /= size; // WTF ERROR?!
    }
    return (T[])ret;
}

Except this won't compile because "Number" doesn't implement the "+=" or "/=" operators. Event worse, java's Number class doesn't implement even the most basic operators like "+" or "-"! How can I make a method which returns the average of an array of Numbers if java won't let me compile it because it thinks that numbers can't be added?!?!