Fastest way to sum two lists of items with each other in python

Question

I have a bit of a strange request that I'm looking to solve with utmost efficiency; I have two lists list_1 and list_2, which are both the same length and will both only ever contain integers greater than or equal to 0. I want to create a new list list_3 such that every element i is the sum of the elements at position i from list_1 and list_2. In python, this would suffice:

list_3 = [list_1[i] + list_2[i] for i in range(len(list_1))]

However, there is a catch. For every i such that 0 <= i < len(list_1), if the item at position i (i.e. list_1[i]) is 0, then the sum of list_1[i] and list_2[i] should also be zero.

What would be the most efficient way to do this? I have to perform this operation on list with 323 elements in it and it needs to be for a game, so it should be able to run easily 60 times per second whilst allowing a lot of extra time for other things to calculate in the game. I was wondering if there might be any fancy numpy way of doing it but I'm not well versed enough with numpy to know for sure.

EDIT:

In terms of simply summing two elements, some common expressions are:

list_3 = [list_1[i] + list_2[i] for i in range(len(list_1))]
list_3 = [sum(t) for t in zip(list_1, list_2)]
list_3 = numpy.add(list_1, list_2)

EDIT 2:

I know of conditional list comprehensions, but I'm wondering if there is a faster method than that.

EDIT 3:

Here are some of the timings of methods given:

>>> import timeit
>>> setup='''
import random
list_1 = [random.randint(0, 323) for i in range(323)]
list_2 = [random.randint(0, 323) for i in range(323)]
'''
>>> timeit.timeit('list_3 = [list_1[i] + list_2[i] if list_2[i] else 0 for i in range(len(list_1))]', setup=setup, number=1)
6.005677381485953e-05
>>> timeit.timeit('list_3 = [x + y if y else 0 for x, y in zip(list_1, list_2)]', setup=setup, number=1)
3.604091037417601e-05

Anything faster?

EDIT 4:

Here is an explanation as to what I need this for: I'm working on a video game that requires a system of checking the state of certain keys on the keyboard from time to time. The way the system needs to work is that the longer a key is pressed down, the higher a counter for said key increases. Once that key is released, the counter is set back to 0. This needs to be done for all keys, not just a select few. It's currently a bottleneck compared to the rest of the program, according to cProfile.

Here is the code that generates the state of each key in the keyboard (it uses pygame to grab the key states):

class KeyState:
    """
    An object representing the state of the keyboard input at a given frame.

    The KeyState is a global replacement for pygame's event system (or
    pygame.keys.get_pressed()). It provides a simple interface for updating
    and retreiving the states of keys in real time.

    To retreive and store the current key information, simply call
    the update() method. To retreive the given information about a
    key, use the get_state(key) method where key is any pygame key
    (i.e. pygame.K_RSHIFT, etc.).
    """

    def __init__(self):
       self.current_frame = pygame.key.get_pressed()

    def update(self):
        """
        Retreive the current key state data.
        """
        new_frame = pygame.key.get_pressed()
        self.current_frame = [state + new_frame[i] if new_frame[i] else 0 for i, state in enumerate(self.current_frame)]

    def get_state(self, key, strict=True):
        """
        Retreive the current state of a given key.

        >= 1 - Pressed
        0    - Unpressed
        """
        try: 
            return self.current_frame[key]
        except KeyError:
            if strict:
                raise

Answer 1

the longer a key is pressed down, the higher a counter for said key increases

Unless your users have 300 fingers, they likely would only be pressing up to ten keys at a time. You can register for keydown and keyup events; save the frame counter or the return value of time()/clock() on the array when a key is down; and when a key is up or when you need to find the current value of the key, subtract the differences. This will reduce the number of loops to around 10 rather than 300. Note that depending on the system, time()/clock() may be a syscall, which can be slow, so using frame counter may be preferable.

counter = 0
keys = {}
while True:
    for event in pygame.event.get() :
        if event.type == pygame.KEYDOWN :
            keys[event.key] = counter
        elif event.type == pygame.KEYUP :
            diff[event.key] = keys.pop(event.key) - counter
    counter += 1

But I highly doubt that this is the bottleneck of your game.

Answer 2

This will work nicely in Python 3.x

list3 = [x+y if x and y else 0 for x, y in zip(list1, list2)]

Or, if you're using Python 2.x:

import itertools as it
list3 = [x+y if x and y else 0 for x, y in it.izip(list1, list2)]

Answer 3

Setup:

import numpy as np
import random
list_1 = [random.randint(0, 323) for i in range(323)]
list_2 = [random.randint(0, 323) for i in range(323)]
array_1 = np.random.randint(0,323,323)
array_2 = np.random.randint(0,323,323)

Original timings:

%timeit list_3 = [list_1[i] + list_2[i] if list_2[i] else 0 for i in range(len(list_1))]
10000 loops, best of 3: 62.8 µs per loop

%timeit list_3 = [list_1[i] + list_2[i] if list_2[i] else 0 for i in range(len(list_1))]
10000 loops, best of 3: 62.3 µs per loop

Oscar Lopez's solutions:

%timeit list3 = [x+y if x and y else 0 for x, y in zip(list_1, list_2)]
10000 loops, best of 3: 60.7 µs per loop

import itertools as it

%timeit list3 = [x+y if x and y else 0 for x, y in it.izip(list_1, list_2)]
10000 loops, best of 3: 50.5 µs per loop

Dawg's np.vectorize solution:

vector_func=np.vectorize(lambda e1, e2: e1+e2 if e1 and e2 else 0)

%timeit vector_func(array_1,array_2)
10000 loops, best of 3: 121 µs per loop

The numpy solution:

%timeit out = array_1 + array_2; out[(array_1==0) & (array_2==0)] = 0
100000 loops, best of 3: 11.1 µs per loop

The issue here is if you choose to use list the numpy solution is in fact slower.

%%timeit
array_1 = np.array(list_1)
array_2 = np.array(list_2)
out = array_1 + array_2
out[(array_1==0) & (array_2==0)] = 0
10000 loops, best of 3: 84.8 µs per loop

The numpy solution will be the fastest, but you need to use numpy arrays at the very start.

Answer 4

list3 = [x + y for x, y in zip(list1, list2)]

Answer 5

Here is an example that illustrates both a regular sum and a sum with a conditional, using zip:

>>> list_1 = [1,2,0,4,9]
>>> list_2 = [6,7,3,1,0]
>>> list_3 = [sum(v) for v in zip(list_1, list_2)] # regular sum
>>> list_3
[7, 9, 3, 5, 9]
>>> list_3 = [sum(v) if 0 not in v else 0 for v in zip(list_1, list_2)] # neither list_1[i] nor list_2[i] should be 0
>>> list_3
[7, 9, 0, 5, 0]
>>> list_3 = [sum(v) if v[0] != 0 else 0 for v in zip(list_1, list_2)] # list_1[i] should not be 0
>>> list_3
[7, 9, 0, 5, 9]

As for speed, any sane solution will perform about the same -- although to save memory you could consider using a generator expression rather than a list if the rest of your code allows it. You should go for whatever is most legible.

Answer 6

You can use a conditional expression:

li1=[1,2,3,4,0,5,6]
li2=[1,2,3,0,5,6,7]

print [ li1[i] + li2[i]  if li1[i] and li2[i] else 0 for i in range(len(li1))]
# [2, 4, 6, 0, 0, 11, 13]

For Numpy (which your question is tagged) you can use vectorize:

>>> a1=np.array([1,2,3,4,0,5,6])
>>> a2=np.array([1,2,3,0,5,6,7])
>>> vector_func=np.vectorize(lambda e1, e2: e1+e2 if e1 and e2 else 0)
>>> vector_func(a1,a2)
array([ 2,  4,  6,  0,  0, 11, 13])

Or, if you prefer a function to a lambda:

>>> def vector(e1, e2):
...    if e1 and e2: return e1+e2
...    return 0
...
>>> vector_func=np.vectorize(vector)
>>> vector_func(a1,a2)
array([ 2,  4,  6,  0,  0, 11, 13])

Using your timing code, the vectorize solution is faster:

import timeit
import numpy as np
import random

a1=np.array([random.randint(0, 323) for i in range(323)])
a2=np.array([random.randint(0, 323) for i in range(323)])
vector_func=np.vectorize(lambda e1, e2: e1+e2 if e1 and e2 else 0)

n=10000

setup='''
from __main__ import a1, a2, vector_func
'''
print timeit.timeit('a3 = [a1[i] + a2[i] if a2[i] else 0 for i in range(len(a1))]', setup=setup, number=n)
print timeit.timeit('a3 = [x + y if y else 0 for x, y in zip(a1, a2)]', setup=setup, number=n)
print timeit.timeit('a3=vector_func(a1,a2)', setup=setup, number=n)

Prints:

2.25640797615
1.97595286369
0.993543148041

Numpy is slower vs pure Python though if you just have ints and no other reason to use numpy.