Is there a standard class for an infinitely nested defaultdict?

Question

Does anyone know if there's a standard class for an infinitely nestable dictionary in Python?

I'm finding myself repeating this pattern:

d = defaultdict(lambda: defaultdict(lambda: defaultdict(int)))
d['abc']['def']['xyz'] += 1

If I want to add "another layer" (e.g. d['abc']['def']['xyz']['wrt']), I have to define another nesting of defaultdicts.

To generalize this pattern, I've written a simple class that overrides __getitem__ to automatically create the next nested dictionary.

e.g.

d = InfiniteDict(('count',0),('total',0))
d['abc']['def']['xyz'].count += 0.24
d['abc']['def']['xyz'].total += 1
d['abc']['def']['xyz']['wrt'].count += 0.143
d['abc']['def']['xyz']['wrt'].total += 1

However, does anyone know of a pre-existing implementation of this idea? I've tried Googling, but I'm not sure what this would be called.

Answer 1

You can derive from defaultdict to get the behavior you want:

class InfiniteDict(defaultdict):
   def __init__(self):
      defaultdict.__init__(self, self.__class__)

class Counters(InfiniteDict):
   def __init__(self):
      InfiniteDict.__init__(self)                                               
      self.count = 0
      self.total = 0

   def show(self):
      print "%i out of %i" % (self.count, self.total)

Usage of this class would look like this:

>>> d = Counters()
>>> d[1][2][3].total = 5
>>> d[1][2][3].show()
0 out of 5
>>> d[5].show()
0 out of 0

Answer 2

This lends itself naturally to a recursive definition.

>>> import collections
>>> def nested_dd():
...     return collections.defaultdict(nested_dd)
...
>>> foo = nested_dd()
>>> foo
defaultdict(<function nested_dd at 0x023F0E30>, {})
>>> foo[1][2]=3
>>> foo[1]
defaultdict(<function nested_dd at 0x023F0E30>, {2: 3})
>>> foo[1][2]
3

Answer 3

I think this one-liner is a nearly perfect solution:

>>> from collections import defaultdict
>>> infinite_defaultdict = lambda: defaultdict(infinite_defaultdict)
>>> d = infinite_defaultdict() 
>>> d['x']['y']['z'] = 10

by Raymond Hettinger on Twitter (https://twitter.com/raymondh/status/343823801278140417)

Answer 4

The ideal solution, inspired by sth's answer:

from collections import defaultdict

class InfiniteDict(defaultdict):
   def __init__(self, **kargs):
      defaultdict.__init__(self, lambda: self.__class__(**kargs))
      self.__dict__.update(kargs)

d = InfiniteDict(count=0, total=0)
d['abc']['def'].count += 0.25
d['abc']['def'].total += 1
print d['abc']['def'].count
print d['abc']['def'].total
d['abc']['def']['xyz'].count += 0.789
d['abc']['def']['xyz'].total += 1
print d['abc']['def']['xyz'].count
print d['abc']['def']['xyz'].total

Answer 5

In case after eight years you are still thinking about how to get this with a one-liner:

from collections import defaultdict

t = defaultdict(lambda: defaultdict(t.default_factory))

Answer 6

This is close:

class recursivedefaultdict(defaultdict):
    def __init__(self, attrFactory=int):
        self.default_factory = lambda : type(self)(attrFactory)
        self._attrFactory = attrFactory
    def __getattr__(self, attr):
        newval = self._attrFactory()
        setattr(self, attr, newval)
        return newval

d = recursivedefaultdict(float)
d['abc']['def']['xyz'].count += 0.24  
d['abc']['def']['xyz'].total += 1  

data = [
    ('A','B','Z',1),
    ('A','C','Y',2),
    ('A','C','X',3),
    ('B','A','W',4),
    ('B','B','V',5),
    ('B','B','U',6),
    ('B','D','T',7),
    ]

table = recursivedefaultdict(int)
for k1,k2,k3,v in data:
    table[k1][k2][k3] = v

It's not quite what you want, since the most deeply nested level does not have your default 0 values for 'count' or 'total'.

Edited: Ah, this works now - just needed to add a __getattr__ method, and this does what you want.

Edit 2: Now you can define other factory methods for the attributes, besides ints. But they all have to be the same type, can't have count be float and total be int.