Python read-only lists using the property decorator

Question

Short Version

Can I make a read-only list using Python's property system?

Long Version

I have created a Python class that has a list as a member. Internally, I would like it to do something every time the list is modified. If this were C++, I would create getters and setters that would allow me to do my bookkeeping whenever the setter was called, and I would have the getter return a const reference, so that the compiler would yell at me if I tried to do modify the list through the getter. In Python, we have the property system, so that writing vanilla getters and setters for every data member is (thankfully) no longer necessary.

However, consider the following script:

def main():

    foo = Foo()
    print('foo.myList:', foo.myList)

    # Here, I'm modifying the list without doing any bookkeeping.
    foo.myList.append(4)
    print('foo.myList:', foo.myList)

    # Here, I'm modifying my "read-only" list.
    foo.readOnlyList.append(8)
    print('foo.readOnlyList:', foo.readOnlyList)


class Foo:
    def __init__(self):
        self._myList = [1, 2, 3]
        self._readOnlyList = [5, 6, 7]

    @property
    def myList(self):
        return self._myList

    @myList.setter
    def myList(self, rhs):
        print("Insert bookkeeping here")
        self._myList = rhs

    @property
    def readOnlyList(self):
        return self._readOnlyList


if __name__ == '__main__':
    main()

Output:

foo.myList: [1, 2, 3]
# Note there's no "Insert bookkeeping here" message.
foo.myList: [1, 2, 3, 4]
foo.readOnlyList: [5, 6, 7, 8]

This illustrates that the absence of the concept of const in Python allows me to modify my list using the append() method, despite the fact that I've made it a property. This can bypass my bookkeeping mechanism (_myList), or it can be used to modify lists that one might like to be read-only (_readOnlyList).

One workaround would be to return a deep copy of the list in the getter method (i.e. return self._myList[:]). This could mean a lot of extra copying, if the list is large or if the copy is done in an inner loop. (But premature optimization is the root of all evil, anyway.) In addition, while a deep copy would prevent the bookkeeping mechanism from being bypassed, if someone were to call .myList.append() , their changes would be silently discarded, which could generate some painful debugging. It would be nice if an exception were raised, so that they'd know they were working against the class' design.

A fix for this last problem would be not to use the property system, and make "normal" getter and setter methods:

def myList(self):
    # No property decorator.
    return self._myList[:]

def setMyList(self, myList):
    print('Insert bookkeeping here')
    self._myList = myList

If the user tried to call append(), it would look like foo.myList().append(8), and those extra parentheses would clue them in that they might be getting a copy, rather than a reference to the internal list's data. The negative thing about this is that it is kind of un-Pythonic to write getters and setters like this, and if the class has other list members, I would have to either write getters and setters for those (eww), or make the interface inconsistent. (I think a slightly inconsistent interface might be the least of all evils.)

Is there another solution I am missing? Can one make a read-only list using Python's property system?

Answer 1

You could have method return a wrapper around your original list -- collections.Sequence might be of help for writing it. Or, you could return a tuple -- The overhead of copying a list into a tuple is often negligible.

Ultimately though, if a user wants to change the underlying list, they can and there's really nothing you can do to stop them. (After all, they have direct access to self._myList if they want it).

I think that the pythonic way to do something like this is to document that they shouldn't change the list and that if the do, then it's their fault when their program crashes and burns.

Answer 2

despite the fact that I've made it a property

It does not matter if it's a property, you are returning a pointer to the list so you can modify it.

I'd suggest creating a list subclass and overriding append and __add__ methods

Answer 3

The proposed solutions of returning a tuple or subclassing list for the return, seem like nice solutions, but i was wondering whether it wouldn't be easier to subclass the decorator instead? Not sure it this might be a stupid idea:

• using this safe_property protects against accidental sending mixed API signals (internal immutable attributes are "protected" against all operations, while the for mutable attributes, some operations are still allowed with the normal property builtin)
• advantage: easier to use than to implement custom return types everywhere -> easier to internalize
• disadvantage: necessity to use different name

class FrozenList(list):

    def _immutable(self, *args, **kws):
        raise TypeError('cannot change object - object is immutable')

    pop = _immutable
    remove = _immutable
    append = _immutable
    clear = _immutable
    extend = _immutable
    insert = _immutable
    reverse = _immutable


class FrozenDict(dict):

    def _immutable(self, *args, **kws):
        raise TypeError('cannot change object - object is immutable')

    __setitem__ = _immutable
    __delitem__ = _immutable
    pop = _immutable
    popitem = _immutable
    clear = _immutable
    update = _immutable
    setdefault = _immutable


class safe_property(property):

    def __get__(self, obj, objtype=None):
        candidate = super().__get__(obj, objtype)
        if isinstance(candidate, dict):
            return FrozenDict(candidate)
        elif isinstance(candidate, list):
            return FrozenList(candidate)
        elif isinstance(candidate, set):
            return frozenset(candidate)
        else:
            return candidate


class Foo:

    def __init__(self):
        self._internal_lst = [1]

    @property
    def internal_lst(self):
        return self._internal_lst

    @safe_property
    def internal_lst_safe(self):
        return self._internal_lst


if __name__ == '__main__':

    foo = Foo()

    foo.internal_lst.append(2)
    # foo._internal_lst is now [1, 2]
    foo.internal_lst_safe.append(3)
    # this throws an exception

Very much interested in other opinions on this as i haven't seen this implemented somewhere else.

Answer 4

This can be accomplished by using the Sequence type hint, which unlike list is non-modifiable:

from typing import Sequence

def foo() -> Sequence[int]:
    return []

result = foo()
result.append(10)
result[0] = 10

Both mypy and pyright will give an error when trying to modify a list that is hinted with Sequence:

$ pyright /tmp/foo.py
  /tmp/foo.py:7:8 - error: Cannot access member "append" for type "Sequence[int]"
    Member "append" is unknown (reportGeneralTypeIssues)
  /tmp/foo.py:8:1 - error: "__setitem__" method not defined on type "Sequence[int]" (reportGeneralTypeIssues)
2 errors, 0 warnings, 0 informations 

Python itself however ignores those hints, so a little must be taken to make sure that either of those type checkers are run regularly or part of the build process.

There is also a Final type hint, that acts similar to C++ const, it however only provides protection of the variable in which the list reference is stored, not for the list itself, so it's not useful here, but might be of use in other situations.

Answer 5

Why is a list preferable to a tuple? Tuples are, for most intents and purposes, 'immutable lists' - so by nature they will act as read-only objects that can't be directly set or modified. At that point one simply needs to not write said setter for that property.

>>> class A(object):
...     def __init__(self, list_data, tuple_data):
...             self._list = list(list_data)
...             self._tuple = tuple(tuple_data)
...     @property
...     def list(self):
...             return self._list
...     @list.setter
...     def list(self, new_v):
...             self._list.append(new_v)
...     @property
...     def tuple(self):
...             return self._tuple
... 
>>> Stick = A((1, 2, 3), (4, 5, 6))
>>> Stick.list
[1, 2, 3]
>>> Stick.tuple
(4, 5, 6)
>>> Stick.list = 4 ##this feels like a weird way to 'cover-up' list.append, but w/e
>>> Stick.list = "Hey"
>>> Stick.list
[1, 2, 3, 4, 'Hey']
>>> Stick.tuple = 4
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>> 

Answer 6

The two main suggestions seem to be either using a tuple as a read-only list, or subclassing list. I like both of those approaches.

Returning a tuple from the getter, or using a tuple in the first place, prevents one from using the += operator, which can be a useful operator and also triggers the bookkeeping mechanism by calling the setter. However, returning a tuple is a one-line change, which is nice if you would like to program defensively but judge that adding a whole other class to your script might be unnecessarily complicated.

Here is a script that illustrates both approaches:

import collections


def main():

    foo = Foo()
    print('foo.myList:', foo.myList)

    try:
        foo.myList.append(4)
    except RuntimeError:
        print('Appending prevented.')

    # Note that this triggers the bookkeeping, as we would like.
    foo.myList += [3.14]
    print('foo.myList:', foo.myList)

    try:
        foo.readOnlySequence.append(8)
    except AttributeError:
        print('Appending prevented.')
    print('foo.readOnlySequence:', foo.readOnlySequence)


class UnappendableList(collections.UserList):
    def __init__(self, *args, **kwargs):
        data = kwargs.pop('data')
        super().__init__(self, *args, **kwargs)
        self.data = data

    def append(self, item):
        raise RuntimeError('No appending allowed.')


class Foo:
    def __init__(self):
        self._myList = [1, 2, 3]
        self._readOnlySequence = [5, 6, 7]

    @property
    def myList(self):
        return UnappendableList(data=self._myList)

    @myList.setter
    def myList(self, rhs):
        print('Insert bookkeeping here')
        self._myList = rhs

    @property
    def readOnlySequence(self):
        # or just use a tuple in the first place
        return tuple(self._readOnlySequence)


if __name__ == '__main__':
    main()

Output:

foo.myList: [1, 2, 3]
Appending prevented.
Insert bookkeeping here
foo.myList: [1, 2, 3, 3.14]
Appending prevented.
foo.readOnlySequence: (5, 6, 7)

---

_{This answer was posted as an edit to the question Python read-only lists using the property decorator by the OP ngb under CC BY-SA 3.0.}