Does Python have a bitfield type?

Question

I need a compact representation of an array of booleans, does Python have a builtin bitfield type or will I need to find an alternate solution?

Answer 1

Bitarray was the best answer I found, when I recently had a similar need. It's a C extension (so much faster than BitVector, which is pure python) and stores its data in an actual bitfield (so it's eight times more memory efficient than a numpy boolean array, which appears to use a byte per element.)

Answer 2

If you mainly want to be able to name your bit fields and easily manipulate them, e.g. to work with flags represented as single bits in a communications protocol, then you can use the standard Structure and Union features of ctypes, as described at How Do I Properly Declare a ctype Structure + Union in Python? - Stack Overflow

For example, to work with the 4 least-significant bits of a byte individually, just name them from least to most significant in a LittleEndianStructure. You use a union to provide access to the same data as a byte or int so you can move the data in or out of the communication protocol. In this case that is done via the flags.asbyte field:

import ctypes
c_uint8 = ctypes.c_uint8

class Flags_bits(ctypes.LittleEndianStructure):
    _fields_ = [
            ("logout", c_uint8, 1),
            ("userswitch", c_uint8, 1),
            ("suspend", c_uint8, 1),
            ("idle", c_uint8, 1),
        ]

class Flags(ctypes.Union):
    _fields_ = [("b", Flags_bits),
                ("asbyte", c_uint8)]

flags = Flags()
flags.asbyte = 0xc

print(flags.b.idle)
print(flags.b.suspend)
print(flags.b.userswitch)
print(flags.b.logout)

The four bits (which I've printed here starting with the most significant, which seems more natural when printing) are 1, 1, 0, 0, i.e. 0xc in binary.

Answer 3

You should take a look at the bitstring module, which has recently reached version 2.0. The binary data is compactly stored as a byte array and can be easily created, modified and analysed.

You can create BitString objects from binary, octal, hex, integers (big or little endian), strings, bytes, floats, files and more.

a = BitString('0xed44')
b = BitString('0b11010010')
c = BitString(int=100, length=14)
d = BitString('uintle:16=55, 0b110, 0o34')
e = BitString(bytes='hello')
f = pack('<2H, bin:3', 5, 17, '001') 

You can then analyse and modify them with simple functions or slice notation - no need to worry about bit masks etc.

a.prepend('0b110')
if '0b11' in b:
    c.reverse()
g = a.join([b, d, e])
g.replace('0b101', '0x3400ee1')
if g[14]:
    del g[14:17]
else:
    g[55:58] = 'uint:11=33, int:9=-1'

There is also a concept of a bit position, so that you can treat it like a file or stream if that's useful to you. Properties are used to give different interpretations of the bit data.

w = g.read(10).uint
x, y, z = g.readlist('int:4, int:4, hex:32')
if g.peek(8) == '0x00':
    g.pos += 10

Plus there's support for the standard bit-wise binary operators, packing, unpacking, endianness and more. The latest version is for Python 2.7 and 3.x, and although it's pure Python it is reasonably well optimised in terms of memory and speed.

Answer 4

I use the binary bit-wise operators !, &, |, ^, >>, and <<. They work really well and are implemented directly in the underlying C, which is usually directly on the underlying hardware.

Answer 5

Represent each of your values as a power of two:

testA = 2**0
testB = 2**1
testC = 2**3

Then to set a value true:

table = table | testB

To set a value false:

table = table & (~testC)

To test for a value:

bitfield_length = 0xff
if ((table & testB & bitfield_length) != 0):
    print "Field B set"

Dig a little deeper into hexadecimal representation if this doesn't make sense to you. This is basically how you keep track of your boolean flags in an embedded C application as well (if you have limitted memory).

Answer 6

The BitVector package may be what you need. It's not built in to my python installation, but easy to track down on the python site.

https://pypi.python.org/pypi/BitVector for the current version.

Answer 7

NumPy has a array interface module that you can use to make a bitfield.

Answer 8

If your bitfield is short, you can probably use the struct module. Otherwise I'd recommend some sort of a wrapper around the array module.

Also, the ctypes module does contain bitfields, but I've never used it myself. Caveat emptor.

Answer 9

If you want to use ints (or long ints) to represent as arrays of bools (or as sets of integers), take a look at http://sourceforge.net/projects/pybitop/files/

It provides insert/extract of bitfields into long ints; finding the most-significant, or least-significant '1' bit; counting all the 1's; bit-reversal; stuff like that which is all possible in pure python but much faster in C.

Answer 10

For mostly-consecutive bits there's the https://pypi.org/project/range_set/ module which is API compatible to Python's built-in set. As the name implies, it stores the bits as begin/end pairs.