implement striping algorithm C++

Question

Hi I am having trouble implementing a striping algorithm. I am also having a problem loading 30000 records in one vector, I tried this, but it is not working.

The program should declare variables to store ONE RECORD at a time. It should read a record and process it then read another record, and so on. Each process should ignore records that "belong" to another process. This can be done by keeping track of the record count and determining if the current record should be processed or ignored. For example, if there are 4 processes (numProcs = 4) process 0 should work on records 0, 4, 8, 12, ... (assuming we count from 0) and ignore all the other records in between.`

   Residence res;
int numProcs = 4;
int linesNum = 0;
int recCount = 0;
int count = 0;

while(count  <=  numProcs)
{
    while(!residenceFile.eof())
   {
        ++recCount;
      //distancess.push_back(populate_distancesVector(res,foodbankData));
        if(recCount % processIS  == linesNum)
        {
           residenceFile >> res.x >>res.y;
           distancess.push_back(populate_distancesVector(res,foodbankData));
       }
      ++linesNum;

   }
    ++count;
}

Update the code

Residence res;
int numProcs = 1;
int recCount = 0;



    while(!residenceFile.eof())
    {
        residenceFile >> res.x >>res.y;

        //distancess.push_back(populate_distancesVector(res,foodbankData));
        if ( recCount == processId)//process id
        {
            distancess.push_back(populate_distancesVector(res,foodbankData));
        }
        ++recCount;
        if(recCount == processId )
            recCount = 0;
    }

update sudo code

 while(!residenceFile.eof())
    {
        residenceFile >> res.x >>res.y;


        if ( recCount % numProcs == numLines)
        {
            distancess.push_back(populate_distancesVector(res,foodbankData));
        }
        else
          ++numLines
        ++recCount
    }

Answer 1

You have tagged your post with MPI, but I don't see any place where you are checking a processor ID to see which record it should process.

Pseudocode for a solution to what I think you're asking:

While(there are more records){
If record count % numProcs == myID
   ProcessRecord
else
   Increment file stream pointer forward one record without processing
Increment Record Count
}

If you know the # of records you will be processing beforehand, then you can come up with a cleverer solution to move the filestream pointer ahead by numprocs records until that # is reached or surpassed.

Answer 2

A process that will act on records 0 and 4 must still read records 1, 2 and 3 (in order to get to 4).

Also, while(!residenceFile.eof()) isn't a good way to iterate through a file; it will read one round past the end. Do something like while(residenceFile >> res.x >>res.y) instead.

As for making a vector that contains 30,000 records, it sounds like a memory limitation. Are you sure you need that many in memory at once?

EDIT:

Look carefully at the updated code. If the process ID (numProcs) is zero, the process will act on the first record and no other; if it is something else, it will act on none of them.

EDIT:

Alas, I do not know Arabic. I will try to explain clearly in English.

You must learn a simple technique, before you attempt a difficult technique. If you guess at the algorithm, you will fail.

First, write a loop that iterates {0,1,2,3,...} and prints out all of the numbers:

int i=0;
while(i<10)
{
  cout << i << endl;
  ++i;
}

Understand this before going farther. Then write a loop that iterates the same way, but prints out only {0,4,8,...}:

int i=0;
while(i<10)
{
  if(i%4==0)
    cout << i << endl;
  ++i;
}

Understand this before going farther. Then write a loop that prints out only {1,5,9,...}. Then write a loop that reads the file, and reports on every record. Then combine that with the logic from the previous exercise, and report on only one record out of every four.

Start with something small and simple. Add complexity in small measures. Develop new techniques in isolation. Test every step. Never add to code that doesn't work. This is the way to write code that works.