memory-mapping in python using numpy error

Question

OUT_DIR = '/media/sf_3dAnalysis/simMatrix/'  
SIM_FILE = 'similarity.npy'

data = np.lib.format.open_memmap(OUT_DIR+SIM_FILE, mode='w+', dtype='float32', shape=(len(filelist),len(filelist)))
del data

So I get the following error message when running this code... mmap.error: [Errno 22] Invalid argument. I really don't understand what I am doing wrong. I am running this in a Linux VM if that's relevant. Also, what's particularly curious is the matrix is created after the code runs, but it still crashes saying the argument is invalid which makes no sense as to why it would be creating the matrix when it says the argument is invalid.

Is there anything special I need to do to get memory mapping to work on a linux machine versus windows and mac? Because it is working fine on my mac and windows machine. I guess I should specify even more, is there some setting or something that needs to be set-up in a virtual machine to have memory mapping working? Because I tried it on a computer running Linux normally, and it worked.

Answer 1

So I fixed my problem guys. I created a local copy of the matrix on the Virtual Machine. I then moved that copy to a shared folder. Here is the code illustrating that.

#create local copy
data = np.memmap(SIM_FILE, dtype='float32', mode='w+', 
          shape=(len(filelist),len(filelist)))
#move local copy to shared folder
os.system('mv' + " ~/Desktop/" + SIM_FILE + " " + OUT_DIR ) 

Answer 2

I was unable to replicate your error with the example given above.

mmap.error: [Errno 22] Invalid argument is the error code from the low-level call to the libc mmap routine see http://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html

mmap returns the address of the new mapping, or -1 for an error.

Possible errors include:

EINVAL Either address was unusable, or inconsistent flags were given.

I guess it's an out of memory condition because you are trying to allocate a too large block, that does not fit into the VM virtual memory space.