Python: Why Is List Comprehension Slower Than For Loop

Essentially these are the same functions - except list comprehension uses sum instead of x=0; x+= since the later is not supported. Why is list comprehension compiled to something

Solution 1:

You are using a generator expression in your list comprehension:

sum(samples[i-j] for j in range(n))

Generator expressions require a new frame to be created each time you run one, just like a function call. This is relatively expensive.

You don't need to use a generator expression at all; you only need to slice the samples list:

sum(samples[i - n + 1:i + 1])

You can specify a second argument, a start value for the sum() function; set it to 0.0 to get a float result:

sum(samples[i - n + 1:i + 1], 0.0)

Together these changes make all the difference:

>>> from timeit import timeit
>>> import random
>>> testdata = [i*random.random() for i in range(1000)]
>>> def slow_moving_average(samples, n=3):
...     return [float(sum(samples[i-j] for j in range(n)))/n for i in range(n-1, len(samples))]
... 
>>> def fast_moving_average(samples, n=3):
...     return [sum(samples[i - n + 1:i + 1], 0.0) / n for i in range(n-1, len(samples))]
... 
>>> def verbose_moving_average(samples, n=3):
...     l =[]
...     for i in range(n-1, len(samples)):
...         x = 0.0
...         for j in range(n): 
...             x+= samples[i-j]
...         l.append(x / n)
...     return l
... 
>>> timeit('f(s)', 'from __main__ import verbose_moving_average as f, testdata as s', number=1000)
0.9375386269966839
>>> timeit('f(s)', 'from __main__ import slow_moving_average as f, testdata as s', number=1000)
1.9631599469939829
>>> timeit('f(s)', 'from __main__ import fast_moving_average as f, testdata as s', number=1000)
0.5647804250038462