For arrays of length 2, this will work.
num2.GroupBy(a => a[0])
.Select(g => new { A0 = g.Key, A1 = g.GroupBy(a => a[1]) })
.SelectMany(a => a.A1.Select(a1 => new { Pair = new int[] { a.A0, a1.Key }, Count = a1.Count() }));
I think that should give you optimal performance; you could also try an .AsParallel()
clause after your first Select statement.
This strategy (grouping successively by the n-th element of the arrays) generalises to arrays of arbitrary length:
var dim = 2;
var tuples = num2.GroupBy(a => a[0])
.Select(g => new Tuple<int[], List<int[]>>(new [] { g.Count(), g.Key }, g.Select(a => a.Skip(1).ToArray()).ToList()));
for (int n = 1; n < dim; n++)
{
tuples = tuples.SelectMany(t => t.Item2.GroupBy(list => list[0])
.Select(g => new Tuple<int[], List<int[]>>(new[] { g.Count() }.Concat(t.Item1.Skip(1)).Concat(new [] { g.Key }).ToArray(), g.Select(a => a.Skip(1).ToArray()).ToList())));
}
var output = tuples.Select(t => new { Arr = string.Join(",", t.Item1.Skip(1)), Count = t.Item1[0] })
.OrderByDescending(o => o.Count)
.ToList();
which generates an output of
Arr = "35, 44", Count = 2
Arr = "200, 22", Count = 2
Arr = "35, 33", Count = 1
Arr = "200, 2", Count = 1
Arr = "3967, 11", Count = 1
in your example. I'll let you test it for higher dimensions. :)
You should be able to parallelise these queries without too much difficulties, as the successive groupings are independent.