ranking tuples with ties

Question

I am writing an extension of the Wilcoxon rank-sums test, which requires that I write the basic function of this test first This also means I can not use SciPy for this exercise.

I have the basic skeleton code there, but I am having trouble averaging the rank of ties. Here is my code:

#read in data
m1 = [0,0,0,0,0,2,3,3,3,4,4,5,6,10,10,10,11,12,15,15,15,20,22,25,25,27,30]
w1 = [0,0,0,0,0,0,1,3,3,3,3,7,8,8,19,20,27,30]

#convert to tuples, incl where they came from
m1t = []
for m in m1:
    m1t.append((m, "m1"))
w1t = []
for w in w1:
    w1t.append((w, "w1"))

all1t = m1t + w1t #combine

all1ts = sorted(all1t, key=lambda tup: tup[0]) #sort

all1tsr = [row+(i,) for i,row in enumerate(all1ts,0)] #rank

#revert to back to original grouping
m1r = [i for i in all1tsr if i[1]=="m1"]
w1r = [i for i in all1tsr if i[1]=="w1"]

and here's the current output:

>>> all1tsr[:15]
[(0, 'm1', 0),
 (0, 'm1', 1),
 (0, 'm1', 2),
 (0, 'm1', 3),
 (0, 'm1', 4),
 (0, 'w1', 5),
 (0, 'w1', 6),
 (0, 'w1', 7),
 (0, 'w1', 8),
 (0, 'w1', 9),
 (0, 'w1', 10),
 (1, 'w1', 11),
 (2, 'm1', 12),
 (3, 'm1', 13),
 (3, 'm1', 14)]

Element 1 of eachtuple is the value by which they are sorted, element 2 is just an identifier, and element 3 is the rank when sorted by element 1. There are 10 observations with "0" as element 1, and right now they are all assigned ascending ranks but I want to somehow average these ranks (assign all to have a rank of 5).

In other words I want this:

[(0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (1, 'w1', 11),
 (2, 'm1', 12),
 (3, 'm1', 13.5),
 (3, 'm1', 13.5)]

All feedback is welcomed, thanks

Answer 1

For starters, I'm going to get all1ts in a much shorter way:

import itertools

all1ts = sorted(itertools.chain(((m, "m1") for m in m1),
                                ((w, "w1") for w in w1)))

all1tsr = [row+(i,) for i,row in enumerate(all1ts)]

Then I'm going to use itertools.groupby, which is basically designed for doing things like this.

groups = []
for _, group in itertools.groupby(all1tsr, lambda x: x[0]):
  group = list(group)
  rank = sum(x[2] for x in group) / len(group)
  groups.extend((val, identifier, rank) for val, identifier, _ in group)

Running on your test data gives me this result:

[(0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'm1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (0, 'w1', 5),
 (1, 'w1', 11),
 (2, 'm1', 12),
 (3, 'm1', 16),
 (3, 'm1', 16),
 (3, 'm1', 16),
 (3, 'w1', 16),
 (3, 'w1', 16),
 (3, 'w1', 16),
 (3, 'w1', 16),
 (4, 'm1', 20),
 (4, 'm1', 20),
 (5, 'm1', 22),
 (6, 'm1', 23),
 (7, 'w1', 24),
 (8, 'w1', 25),
 (8, 'w1', 25),
 (10, 'm1', 28),
 (10, 'm1', 28),
 (10, 'm1', 28),
 (11, 'm1', 30),
 (12, 'm1', 31),
 (15, 'm1', 33),
 (15, 'm1', 33),
 (15, 'm1', 33),
 (19, 'w1', 35),
 (20, 'm1', 36),
 (20, 'w1', 36),
 (22, 'm1', 38),
 (25, 'm1', 39),
 (25, 'm1', 39),
 (27, 'm1', 41),
 (27, 'w1', 41),
 (30, 'm1', 43),
 (30, 'w1', 43)]

Which I think is what you want.