Read multiple .dat files by GPU

Question

I understand that reading files by GPU is inefficient task as it's faced by the slowest part of the system, that is, IO. However, I came up with another approach by using the CPU for files reading and let the processing burden be handled by the GPU. I wrote the following code in C++ but I'm stuck at the integration point, that is, how to make GPU handle these files after they've been read by the CPU. In other words, what is the set off point of C++-amp to be added and integrated with the code? or should I rewrite the whole code from the scratch?

{/* this code to read multiple .dat files from the directory that contains the implementation (from my account of stackoverflow) */

#include <Windows.h>
#include <ctime>
#include <stdint.h>
#include <iostream>
using std::cout;
using std::endl;
#include <fstream>
using std::ifstream;

#include <cstring>

/* Returns the amount of milliseconds elapsed since the UNIX epoch. Works on both
 * windows and linux. */

uint64_t GetTimeMs64()
{


 FILETIME ft;
 LARGE_INTEGER li;

 /* Get the amount of 100 nano seconds intervals elapsed since January 1, 1601 (UTC) and copy it
  * to a LARGE_INTEGER structure. */
 GetSystemTimeAsFileTime(&ft);
 li.LowPart = ft.dwLowDateTime;
 li.HighPart = ft.dwHighDateTime;

 uint64_t ret;
 ret = li.QuadPart;
 ret -= 116444736000000000LL; /* Convert from file time to UNIX epoch time. */
 ret /= 10000; /* From 100 nano seconds (10^-7) to 1 millisecond (10^-3) intervals */

 return ret;

}


const int MAX_CHARS_PER_LINE = 512;
const int MAX_TOKENS_PER_LINE = 20;
const char* const DELIMITER = "|";

int main()
{
  // create a file-reading object
  uint64_t a = GetTimeMs64();
  cout << a << endl;
  HANDLE h;
WIN32_FIND_DATA find_data;
h = FindFirstFile( "*.dat", & find_data );
if( h == INVALID_HANDLE_VALUE ) {
    cout<<"error"<<endl;

}
do {
    char * s = find_data.cFileName;

    ifstream fin;
  fin.open(s); // open a file
  if (!fin.good()) 
    return 1; // exit if file not found

  // read each line of the file
  while (!fin.eof())
  {
    // read an entire line into memory
    char buf[MAX_CHARS_PER_LINE];
    fin.getline(buf, MAX_CHARS_PER_LINE);

    // parse the line into blank-delimited tokens
    int n = 0; // a for-loop index

    // array to store memory addresses of the tokens in buf
    const char* token[MAX_TOKENS_PER_LINE] = {}; // initialize to 0

    // parse the line
    token[0] = strtok(buf, DELIMITER); // first token
    if (token[0]) // zero if line is blank
    {
      for (n = 1; n < MAX_TOKENS_PER_LINE; n++)
      {
    token[n] = strtok(0, DELIMITER); // subsequent tokens
        if (!token[n]) break; // no more tokens
  }
}

    // process (print) the tokens
    for (int i = 0; i < n; i++) // n = #of tokens
      cout << "Token[" << i << "] = " << token[i] << endl;
    cout << endl;
  }
            // Your code here
} while( FindNextFile( h, & find_data ) );
FindClose( h );



  uint64_t b = GetTimeMs64();
  cout << a << endl;
  cout << b << endl;
  uint64_t c = b - a;
  cout << c << endl;

  system("pause");
}

Answer 1

There is no way to handle the files for GPU. As you assumed CPU handles IO. So you need to store your read information in memory, send it to the GPU, compute there and etc.

One of the good ways to work with files is to archive (with GPU) your information.

So you read file with CPU, extract > compute > archive with GPU, and store it with CPU.

UPD.

(CPU IO READ from file (should be already archived information))  to -> main memory
(CPU SEND) to -> GPU global memory from main memory

(GPU EXTRACT (if archived))
(GPU COMPUTE (your work here))
(GPU ARCHIVE)

(CPU RETRIEVE) to -> main memory from GPU global memory
(CPU IO WRITE to file)