Looking at the source of Linux kernel, you can get EINVAL for:
- Passing in an invalid mode value. Either out of range of "inconsistent" (using both static and relative nodes at the same time)
- invalid maxnode (> number of bits in a page -> 32K on x86).
- Various other problems with nodemask.
- Not having one of
MPOL_MF_STRICT | MPOL_MF_MOVE | MPOL_MF_MOVE_ALL
start
is not page-aligned.start+len
when page-aligned = start. [that is, your len is not at least one byte]start+len
<start
- that is, negative length.- policy =
MPOL_DEFAULT
andnodes
isn't empty orNULL
. - quoting comment from source "MPOL_PREFERRED cannot be used with MPOL_F_STATIC_NODES or MPOL_F_RELATIVE_NODES if the nodemask is empty (local allocation). All other modes require a valid pointer to a non-empty nodemask.
My guess would be on start
is not page-aligned.
This code works for me:
#include <numaif.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#define ASSERT(x) do { if (!(x)) do_assert(#x,(long)(x), __FILE__, __LINE__); } while(0)
static void do_assert(const char *expr, long expr_int, const char *file, int line)
{
fprintf(stderr, "ASSERT failed %s (%d) at %s:%d\n",
expr, expr_int, file, line);
perror("Error if present:");
exit(1);
}
int main()
{
size_t num_items = 6156000;
double *A = valloc(num_items * sizeof(double));
ASSERT(A != NULL);
int res;
unsigned long nodemask;
size_t page_size = sysconf(_SC_PAGESIZE);
size_t objs_per_page = page_size/sizeof(A[0]);
ASSERT(page_size%sizeof(A[0])==0);
size_t split_three=num_items/3;
size_t aligned_size=(split_three/objs_per_page)*objs_per_page;
size_t remnant=num_items-(aligned_size*3);
size_t piece = aligned_size;
printf("A[0]=%p\n", &A[0]);
printf("A[%d]=%p\n", piece, &A[aligned_size]);
printf("A[%d]=%p\n", 2*piece, &A[2*piece]);
nodemask=1;
res = mbind(&A[0],piece*sizeof(double),MPOL_BIND,&nodemask,64,MPOL_MF_MOVE);
ASSERT(res ==0);
nodemask=1;
res = mbind(&A[aligned_size],piece*sizeof(double),MPOL_BIND,&nodemask,64,MPOL_MF_MOVE);
ASSERT(res ==0);
nodemask=1;
res = mbind(&A[aligned_size*2],(piece+remnant)*sizeof(double),MPOL_BIND,
&nodemask,64,MPOL_MF_MOVE);
ASSERT(res == 0);
}
Note that I'm using "nodemask=1" on all allocations, since I've only got one quad-core processor in my machine, so no other nodes to bind to - which also gives EINVAL
. I take it you actually have more than one node in your system.
I also moved the "remnant" from the A[]
to piece+remnant
size for the last mbind
call.