Creating many arrays at once

Question

I am currently working with hundreds of files, all of which I want to read in and view as a numpy array. Right now I am using os.walk to pull all the files from a directory. I have a for loop that goes through the directory and will then create the array, but it is not stored anywhere. Is there a way to create arrays "on the go" or to somehow allocate a certain amount of memory for empty arrays?

Answer 1

Just append them to a list as you go:

lists = []

for dirpath, dirnames, filenames in os.walk(...):
    lists.append(...)

Answer 2

Python's lists are dynamic, you can change their length on-the-fly, so just store them in a list. Or if you wanted to reference them by name instead of number, use a dictionary, whose size can also change on the fly.