I figured this out. My problem was I was doing this:
rv = {}
rv['val'] = ExtensionType()
rv['val'].c_attr = val
This doesn't work, because rv['val'] is a python object now, so you can't access the cdef attrs. You need to use an intermediate cdef, like this
cdef ExtensionType tmpvar
rv = {}
tmpvar = ExtensionType()
tmpvar.c_attr = val
rv['val'] = tmpvar