What is the highest number Python 2.* x86 (i.e.32bit) id() function on Windows x64 can return?

Question

What is the highest number Python 2.6's x86 id() function can return?

I presume the ceiling must be the amount of memory any 32-bit application can see which must be: 2³², i.e. 4294967296?

(Which would be fine as I have to fit this value in a C# type for which UInt32 or the CLS-compliant Int64 would suffice, which it the reason for my concern, though irrelevant to the question.)

But what if a I am running on Windows x64 with more than 2GB of memory say 32GB of memory - is it possible for Python's id() function, even though the Python interpreter itself is x86, to return a value higher then 2³²???

I am guessing it comes down to the Python interpreter's view of the machine - I imagine WoW64 translates 64-bit memory addresses to 32-bit addresses - but I am only guessing - I need to be sure!

Answer 1

Assuming the documentation for id() is correct and CPython (x86) returns the address of the object, the maximum value that can be returned is 2³²-1 (4294967295). This can be demonstrated in a simple C program:

#include <stdio.h>

int main(void) {
    void* p = 0;          // specify a pointer at 0
    p = ~p;               // invert all bits
    uintptr_t x = p;      // convert to an integer type fo the same size
    printf("%lu\n", x);
    return 0;
}

This is true regardless of the underlying OS, an application compiled with a 32-bit pointer type can only specify addresses between 0 and 2³²-1. Note that the address visible in the application may not correspond to a particular physical memory address, due to the use of virtual memory.

Answer 2

From the linked documentation:

Returns (...) an integer (or long integer)

In Python 2.x on amd64, a long integer is an integer larger than 64 bits. In any case, in Python, integers are unbounded. Therefore, id can return arbitrarily long values.

If you must know a definite maximum value, you may assume that the available memory on your platform is an upper boundary for the size of the integer you get. Therefore, I'd specify 2²³² for 32 bit, and 2²⁶⁴ for 64 architectures. In the case of an x86 python implementation, one can therefore place an upper boundary at 2²³² with reasonable confidence.

cpython (the most popular python implementation) will indeed return a memory addresses (builtin_id in Python/bltinmodule.c):

static PyObject * builtin_id(PyObject *self, PyObject *v) {
    return PyLong_FromVoidPtr(v);
}

This will be a 32 bit/64 bit value, but the behavior is an implementation detail, as explicitly stated in the docs. By definition, a programmer must not rely on implementation details.

I strongly doubt that there is a legitimate use case for ever using IDs, much less transferring them to another program. Instead, you should use a custom object table, or just transfer a set. If you intend to use Python in conjunction with C#, ironpython allows you to do so in the same code.

Answer 3

def func(n):
    max=0L
    for i in range(n):
        x=id(i)
        if x > max : max=x
    return max

for i in range(32):
    x=0L
    try:
        x=func(2**i)
        print '{0: 10}{1: 15}{2: 15}'.format(i,2**i,x)
    except:
        print '{0: 10}{1: 15}{2:>15}'.format(i,2**i,'error')

i think in practical the max value the id() can achieve is 2**32 but in reality it never reaches such high number. I tried the above code in the 32 bit architecture . please let me know if i made any mistake in understanding your question.

4294967296L is the 2^32 and max id() i reached is 1460876200.

Answer 4

Your guesses seem reasonable, but I have no idea if they're correct.

If you need to rely on behaviour that is an implementation detail (i.e. isn't specified in the docs), and you need to be sure, then I guess the only thing to do is read the source code for the version you're using and find out what it does.

Answer 5

Windows will run an application in the environment (32 or 64 bit) that it was compiled for, not the CPU it is running on. E.g., an x64 computer running Windows 7 will run 32-bit python in a 32-bit environment.

So, running python compiled for x86 will always use a 32-bit address space, which means that id() will always return a value that maps to a unique 32-bit pointer. (Note that the implementation may do some weird stuff like salting the pointer or using 64-bits or some crazy stuff like that.)