accessing in C++ a 'virtual file' created by Bash process substitution

Question

I have a C++ executable that, in normal use, accepts a file name as an argument option in the following manner:

executable -i myFile.txt

I want to use Bash process substitution to create a 'virtual file' and send information (simple, line by line data) to this executable in the following manner:

executable -i <(echo "${myData}")

However, my C++ program is not accessing the information when I use this process substitution. The main file reading section of code in the C++ program is the following:

ifstream file1 (fileName1);
string line;
int currentLineNumber = 0;
if (verboseFlag == 1) {cout << "reading data from file " << fileName1 << "..." << endl;}
while (getline (file1, line)){
    currentLineNumber++;
    if (verboseFlag == 1) {cout << "line " << currentLineNumber << ": ";}
    istringstream linestream(line);
    string item;
    int itemNumber = 0;
    while (getline (linestream, item, ',')){
        itemNumber++;
        if (verboseFlag == 1) {cout << "item " << itemNumber << ": " << item << " ";}
        // data
            if (itemNumber == 1) {x[currentLineNumber]=atof(item.c_str());}
            if (itemNumber == 2) {y[currentLineNumber]=atof(item.c_str());}
    }
}
file1.close();

Could you point me in the right direction on solving this reading problem? Is there some better approach that would work for both normal file reading and process substitution 'file' reading?

I'm new to this process substitution and I very much appreciate any assistance on this.

---

EDIT:

Following some comments, what follows is a minimal working example illustrating the problem I am encountering:

// definition of standard input/output stream objects
    #include <iostream>
// manipulate strings as though they were input/output streams
    #include <sstream>
// input and output operations
    #include <stdio.h>
// file input and output operations
    #include <fstream>
// manipulate C strings and arrays
    #include <string.h>
// classify and transform individual characters
    #include <ctype.h>
// Standard General Utilities Library
    #include <stdlib.h>
// getopts (handle command line options and arguments)
    #include <unistd.h>
// sstream (handle conversion from char* to double)
    #include <sstream>

using namespace std;

double returnDoubleFromPointerToChar(const char *cText){
    std::stringstream ss ( cText );
    double dText = 0;
    ss >> dText;
    return dText;
}

int returnNumberOfLinesInFile(const char *fileName1){
    int lineCount = 0;
    string line;
    ifstream file1(fileName1);
    while (std::getline(file1, line))
        ++lineCount;
    file1.close();
    return lineCount;
}

int main (int argc, char **argv){
    char *fileName1 = NULL; // input file name  (i) (required input)
    int verboseFlag = 0;        // verbose flag     (v)
    int index; // internal variable
    int c; // internal variable
    opterr = 0;

    // process command line arguments and options
        while ((c = getopt (argc, argv, "i:v")) != -1)
            switch (c){
                case 'i':
                    fileName1 = optarg;
                    break;
                case 'v':
                    verboseFlag = 1;
                    break;
                case '?':
                    if (
                        optopt == 'i'
                    ){
                        fprintf (stderr, "option -%c requires an argument.\n", optopt);
                    }
                    else if (isprint (optopt)){
                        fprintf (stderr, "unknown option `-%c'.\n", optopt);
                    }
                    else {
                        fprintf (stderr, "unknown option character `\\x%x'.\n", optopt);
                    }
                    return 1;
                default:
                    abort ();
        }
        for (index = optind; index < argc; index++) printf ("non option argument %s\n", argv[index]);
    if (verboseFlag == 1){
        cout << endl;
        cout << "input file name: " << fileName1 << endl;
    }
    // Determine the number of lines in the input file.
        int numberOfLinesInInputFile=returnNumberOfLinesInFile(fileName1);
        if (verboseFlag == 1) {cout << "number of lines in input file: " << numberOfLinesInInputFile << endl;}
    // number of data points
        int n=numberOfLinesInInputFile-1;
    // x variable
        double x[n];
    // y variable
        double y[n];
    // Access the data in the input file.
        ifstream file1 (fileName1);
        string line;
        int currentLineNumber = 0;
        if (verboseFlag == 1) {cout << "reading data from file " << fileName1 << "..." << endl;}
        while (getline (file1, line)){
            currentLineNumber++;
            if (verboseFlag == 1) {cout << "line " << currentLineNumber << ": ";}
            istringstream linestream(line);
            string item;
            int itemNumber = 0;
            while (getline (linestream, item, ',')){
                itemNumber++;
                if (verboseFlag == 1) {cout << "item " << itemNumber << ": " << item << " ";}
                // data
                    if (itemNumber == 1) {x[currentLineNumber]=atof(item.c_str());}
                    if (itemNumber == 2) {y[currentLineNumber]=atof(item.c_str());}
            }
            if (verboseFlag == 1) {cout << endl;}
        }
        file1.close();
    return 0;
}

EDIT:

I have added the solution code below (following from a comment by that other guy):

// include WBM C++ library
//  #include "lib_cpp.c"
// definition of standard input/output stream objects
    #include <iostream>
// manipulate strings as though they were input/output streams
    #include <sstream>
// input and output operations
    #include <stdio.h>
// file input and output operations
    #include <fstream>
// manipulate C strings and arrays
    #include <string.h>
// classify and transform individual characters
    #include <ctype.h>
// Standard General Utilities Library
    #include <stdlib.h>
// getopts (handle command line options and arguments)
    #include <unistd.h>
// sstream (handle conversion from char* to double)
    #include <sstream>

using namespace std;

// example usage:
//  ./graph2d -i data.txt -o data.eps -v
//  ./graph2d -i data.txt -o graph.eps -t "training test error versus epochs" -x "epochs" -y "error measure" -v
//  ./graph2d -i data.txt -o graph.eps -t "training test error versus epochs" -x "epochs" -y "error measure" -a 70 -b 50 -c 22 -d 7 -v
//  ./graph2d -i <(echo "${dataForTrainingErrorVersusEpoch}") -o graph.eps -t "training test error versus epochs" -x "epochs" -y "error measure" -v

double returnDoubleFromPointerToChar(const char *cText){
    std::stringstream ss ( cText );
    double dText = 0;
    ss >> dText;
    return dText;
}

int main (int argc, char **argv){
    char *fileName1 = NULL; // input file name  (i) (required input)
    char *fileName2 = NULL; // output file name (o) (required input)
    char *graphTitleMain = NULL;    // graph title      (t)
    char *graphTitleAxisx = NULL;   // graph x axis title   (x)
    char *graphTitleAxisy = NULL;   // graph y axis title   (y)
    double axisyMaximum = DBL_MAX;  // y axis maximum   (a)
    double axisyMinimum = DBL_MAX;  // y axis minimum   (b)
    double axisxMaximum = DBL_MAX;  // x axis maximum   (c)
    double axisxMinimum = DBL_MAX;  // x axis minimum   (d)
    int verboseFlag = 0;        // verbose flag     (v)
    int index; // internal variable
    int c; // internal variable
    opterr = 0;

    // process command line arguments and options
        while ((c = getopt (argc, argv, "i:o:t:x:y:a:b:c:d:v")) != -1)
            switch (c){
                case 'i':
                    fileName1 = optarg;
                    break;
                case 'o':
                    fileName2 = optarg;
                    break;
                case 't':
                    graphTitleMain = optarg;
                    break;
                case 'x':
                    graphTitleAxisx = optarg;
                    break;
                case 'y':
                    graphTitleAxisy = optarg;
                    break;
                case 'a':
                    axisyMaximum = returnDoubleFromPointerToChar(optarg);
                    break;
                case 'b':
                    axisyMinimum = returnDoubleFromPointerToChar(optarg);
                    break;
                case 'c':
                    axisxMaximum = returnDoubleFromPointerToChar(optarg);
                    break;
                case 'd':
                    axisxMinimum = returnDoubleFromPointerToChar(optarg);
                    break;
                case 'v':
                    verboseFlag = 1;
                    break;
                case '?':
                    if (
                        optopt == 'i' ||
                        optopt == 'o' ||
                        optopt == 't' ||
                        optopt == 'x' ||
                        optopt == 'y' ||
                        optopt == 'a' ||
                        optopt == 'b' ||
                        optopt == 'c' ||
                        optopt == 'd'
                    ){
                        fprintf (stderr, "option -%c requires an argument.\n", optopt);
                    }
                    else if (isprint (optopt)){
                        fprintf (stderr, "unknown option `-%c'.\n", optopt);
                    }
                    else {
                        fprintf (stderr, "unknown option character `\\x%x'.\n", optopt);
                    }
                    return 1;
                default:
                    abort ();
        }
        for (index = optind; index < argc; index++) printf ("non option argument %s\n", argv[index]);
    if (verboseFlag == 1){
        cout << endl;
        cout << "input file name: " << fileName1 << endl;
        cout << "output file name: " << fileName2 << endl;
    }
    // x variable
        vector<int> x;
    // y variable
        vector<int> y;
    // Access the data in the input file.
        ifstream file1 (fileName1);
        string line;
        int currentLineNumber = 0;
        if (verboseFlag == 1) {cout << "reading data from file " << fileName1 << "..." << endl;}
        while (getline (file1, line)){
            currentLineNumber++;
            if (verboseFlag == 1) {cout << "line " << currentLineNumber << ": ";}
            istringstream linestream(line);
            string item;
            int itemNumber = 0;
            while (getline (linestream, item, ',')){
                itemNumber++;
                if (verboseFlag == 1) {cout << "item " << itemNumber << ": " << item << " ";}
                // data
                    if (itemNumber == 1) {x.push_back(atof(item.c_str()));}
                    if (itemNumber == 2) {y.push_back(atof(item.c_str()));}
            }
            if (verboseFlag == 1) {cout << endl;}
        }
        file1.close();
        int numberOfLinesInInputFile = currentLineNumber + 1;
    // number of data points
        int n=numberOfLinesInInputFile;
    // graph
        if (verboseFlag == 1){
            cout << "graph main title: " << graphTitleMain << endl;
            cout << "graph x axis title: " << graphTitleAxisx << endl;
            cout << "graph y axis title: " << graphTitleAxisy << endl;
        }
        // Create a new canvas.
            TCanvas *c1 = new TCanvas(graphTitleMain, graphTitleMain); // #u
        // Create a new graph.
            //TGraph *graph = new TGraph(n, x, y);
            TGraph *graph = new TGraph(n, &x[0], &y[0]);
        // Set the graph titles.
            graph->SetTitle(graphTitleMain);
            graph->GetXaxis()->SetTitle(graphTitleAxisx);
            graph->GetYaxis()->SetTitle(graphTitleAxisy);
        // Set the marker styles.
            graph->SetMarkerColor(2); // red
            graph->SetMarkerStyle(kFullCircle); // circle
            graph->SetMarkerSize(1); // default size
        // Set the graph range, if ranges have been specified in command line options.
            if (
                axisyMaximum != DBL_MAX &&
                axisyMinimum != DBL_MAX
            ){
                if (verboseFlag == 1){
                    cout << "graph y axis minimum: " << axisyMinimum << endl;
                    cout << "graph y axis maximum: " << axisyMaximum << endl;
                }
                graph->GetYaxis()->SetRangeUser(axisyMinimum, axisyMaximum);
            }
            if (
                axisxMaximum != DBL_MAX &&
                axisxMinimum != DBL_MAX
            ){
                if (verboseFlag == 1){
                    cout << "graph x axis minimum: " << axisxMinimum << endl;
                    cout << "graph x axis maximum: " << axisxMaximum << endl;
                }
                graph->GetXaxis()->SetRangeUser(axisxMinimum, axisxMaximum);
            }
        // Draw the canvas, then draw the graph and then save the canvas to an image file.
            c1->Draw();
            graph->Draw("ALP");
            // disable ROOT messages
                gErrorIgnoreLevel = 5000;
            if (verboseFlag == 1) {cout << "saving file " << fileName2 << "..." << endl;}
            c1->SaveAs(fileName2);
    if (verboseFlag == 1) {cout << endl;}
    return 0;
}

Answer 1

First of all, I trimmed out two thirds of your program and it still shows the problem. This is significantly closer to minimal:

#include <iostream>
#include <fstream>

using namespace std;

int returnNumberOfLinesInFile(const char *fileName1){
  int lineCount = 0;
  string line;
  ifstream file1(fileName1);
  while (std::getline(file1, line))
    ++lineCount;
  file1.close();
  return lineCount;
}

int main (int argc, char **argv){
  char *fileName1 = argv[1];
  cout << "input file name: " << fileName1 << endl;
  int numberOfLinesInInputFile=returnNumberOfLinesInFile(fileName1);
  cout << "number of lines in input file: " << numberOfLinesInInputFile << endl;

  ifstream file1(fileName1);
  string line;
  cout << "File contents: " << endl;
  while (getline (file1, line)){
    cout << "line: " << line << endl;
  }
  file1.close();
  return 0;
}

The problem here is that you open the file twice. <(process substitution) only runs the command once and streams the result. Bash doesn't take the liberty of running the command again if you want to read output again, since the command could have been doing a lot of other things besides spitting out text.

Make sure your program only opens and reads the contents once, and it'll work. This may require you to rewrite your logic a bit, or just be lazy and read it all into memory at once.

Answer 2

Your code works fine for me (I'm on OS X).

Keep in mind that, unlike real files, "virtual files" are usually pipe endpoints (implemented in bash using file descriptor special files). So, you cannot open, read, and close a virtual file more than once, or you will get nothing the second time around.