Python List Of Lists Vs Numpy

So, I have a snippet of a script: lol = [] latv1 = 0 latv2 = 0 latv3 = 0 #Loop a for a in range(100): #Refresh latv2 after each iteration of loop a latv2 = 0 #Loop b

Solution 1:

Approach #1

To create the NumPy equivalent of the posted code and have NumPy array as output, you could additionally make use of itertools, like so -

from itertools import product

out = np.zeros((N**3,4),dtype=int)
out[:,:3] = list(product(np.arange(N), repeat=3))

Please note that it would be N = 100 to make it equivalent to the posted code.

Approach #2

Another potentially faster approach based on purely NumPy and using it's vectorized broadcasting capabilities could be suggested like so -

out= np.zeros((N**3,4),dtype=int)
out[:,:3] = (np.arange(N**3)[:,None]/[N**2,N,1])%N

I would think this to be faster than the previous itertools based one, because that created a list of tuples that are to be set into a NumPy array. We will test this theory out in the next section.

---

Runtime test

In [111]: def itertools_based(N):
     ...:     out= np.zeros((N**3,4),dtype=int)
     ...:     out[:,:3] = list(product(np.arange(N), repeat=3))
     ...:     returnout
     ...: 
     ...: def broadcasting_based(N):
     ...:     out= np.zeros((N**3,4),dtype=int)
     ...:     out[:,:3] = (np.arange(N**3)[:,None]/[N**2,N,1])%N
     ...:     returnoutIn [112]: N =20In [113]: np.allclose(itertools_based(N),broadcasting_based(N)) # Verify results
Out[113]: TrueIn [114]: %timeit itertools_based(N)
100 loops, best of3: 7.42 ms per loop

In [115]: %timeit broadcasting_based(N)
1000 loops, best of3: 1.23 ms per loop

Now, let's time just the creation of list of tuples of those iterated elements and put it against the NumPy based one -

In [116]: %timeit list(product(np.arange(N), repeat=3))
1000 loops, best of 3: 746 µs per loop

In [117]: %timeit (np.arange(N**3)[:,None]/[N**2,N,1])%N
1000 loops, best of 3: 1.09 ms per loop

Well, so the creation part for the itertools-based one is faster now, as predicted/thought out earlier! So, if you are happy with the first three columns as output and them being list of tuples, then go with itertools.